Waterlogging video water accumulation area segmentation method, system, device and medium

Abstract

The present disclosure relates to the technical field of computer vision and urban waterlogging monitoring, and particularly relates to an urban waterlogging video waterlogging area segmentation method, system, computer device and computer readable storage medium fusing time sequence frequency domain consistency constraint, waterlogging physical spectrum modeling, frequency band competition selection and adaptive gating decoupling interaction. The method acquires a monitoring video and extracts a continuous T-frame image sequence, performs fast Fourier transform and time sequence frequency domain consistency coding after normalization, constructs a learnable bimodal spectrum template, an overlapping soft frequency band competition selection mechanism and an amplitude-phase gating decoupling interaction mechanism, obtains a waterlogging area segmentation result through multi-scale decoding, and obtains a dynamic water level estimation result and a waterlogging early warning result in combination with a reference object calibration and high-frequency ripple disturbance information. The present disclosure can improve the time sequence stability, boundary accuracy and early warning reliability of waterlogging segmentation in a complex scene.

Description

Technical Field

[0001] This disclosure relates to the fields of computer vision and urban flood monitoring technology, and in particular to a method, system, computer equipment, and computer-readable storage medium for segmenting flooded areas in urban flood videos by integrating time-series frequency domain consistency constraints, physical spectrum modeling of water accumulation, frequency band competition selection, and adaptive gating decoupling interaction. Background Technology

[0002] Urban flood monitoring is a crucial technical component in urban flood control and drainage, traffic operation assurance, and emergency command and dispatch. With the continuous development of urban road video surveillance, underpass monitoring, and flood-prone area monitoring systems, automatic identification and segmentation of waterlogged areas based on surveillance video has become an important technical approach for urban flood monitoring and early warning. Compared to single-point water level sensors, video monitoring can provide a wider range of on-site image information, facilitating the acquisition of waterlogged areas, waterlogged boundaries, road traffic conditions, and scene changes.

[0003] Existing methods for identifying water accumulation in videos are typically based on RGB images, texture features, semantic segmentation networks, or video object segmentation networks. These methods can identify water accumulation areas in a video to some extent, but they primarily rely on color, brightness, texture, edge, and semantic information in the image spatial domain. In complex scenarios such as urban roads, bridges, and underpasses, water accumulation often occurs simultaneously with road surface reflections, building reflections, strong headlight glare, rain streaks, vehicle shadows, and pedestrian disturbances. Relying solely on spatial domain features can easily misclassify slippery surfaces, highly reflective areas, or moving shadows as water accumulation areas. Furthermore, under low light, rainy conditions, and conditions with blurred boundaries, segmentation boundaries may flicker or break.

[0004] Some existing methods attempt to incorporate frequency domain learning or frequency domain filtering to utilize image spectral features to help distinguish between water accumulation and background interference. However, common frequency domain methods are mostly based on single-frame image processing or use fixed low-frequency and high-frequency division to filter the spectrum. These methods struggle to utilize the relatively stable spectral evolution characteristics of water accumulation areas between adjacent frames in a video sequence, and are also difficult to adapt to changes in the spectral distribution of water accumulation under different scenes, lighting conditions, and water surface fluctuations. Furthermore, if frequency band division relies too heavily on preset hard thresholds, it can easily lead to spectral information truncation, making it difficult to simultaneously address the recognition needs of low-frequency reflection areas and mid-to-high-frequency ripple areas.

[0005] Furthermore, the visual representation of water accumulation is not simply a matter of image texture. Water accumulation areas typically exhibit features such as low-frequency specular reflection, local ripple disturbances, and boundary gradations. Among these, the amplitude spectrum more readily reflects the overall structure and energy distribution, while the phase spectrum more readily reflects edge and local geometric changes. Existing amplitude and phase branch fusion methods often employ channel splicing, element-wise addition, or fixed-weight fusion, making it difficult to dynamically adjust the information contribution of different feature branches based on the reflectivity, boundary sharpness, and ripple disturbances in the current scene.

[0006] On the other hand, urban flooding monitoring systems not only need to output waterlogged area segmentation results, but also typically need to further estimate water depth and trigger early warnings. If water level estimation relies solely on the pixel geometry between the waterlogged area and a reference object, without considering the impact of water surface fluctuations, reflection disturbances, and spectral changes on water surface boundary positioning, then water level estimation errors are likely to occur in scenarios with obvious ripples or strong image reflections. Therefore, how to simultaneously utilize temporal information, frequency domain features, and physical priors of the water surface during video waterlogged area segmentation, and stably serve dynamic water level estimation and flooding early warning with the segmentation results, is a technical problem that needs further resolution in the field of intelligent urban flooding monitoring. Summary of the Invention

[0007] (a) Technical problems to be solved

[0008] This disclosure aims to at least partially solve one of the aforementioned technical problems.

[0009] Specifically, this disclosure aims to provide a method, system, device, and medium for waterlogging area segmentation in urban flooding videos based on time-series frequency domain physical modeling, in order to at least partially solve the problems of existing video waterlogging identification methods, which mainly rely on spatial domain image features, have difficulty distinguishing between waterlogged areas and pseudo-waterlogged areas such as reflections, have difficulty utilizing the temporal stability of the waterlogging spectrum in video sequences, lack physical constraints on frequency band division, have insufficiently adaptive amplitude and phase spectrum fusion methods, and have water level estimation that is easily affected by water surface fluctuations.

[0010] (II) Technical Solution According to the first aspect of this disclosure, a method for segmenting waterlogged areas in urban flooding videos based on temporal frequency domain physical modeling is provided. The method includes: Step S1, using surveillance videos of low-lying urban roads, underpasses, or flood-prone areas, extracting and normalizing a continuous T-frame image sequence to obtain a normalized continuous video frame sequence, and determining the initial low-frequency peak and initial high-frequency band of the physical spectrum template based on prior statistical data of the waterlogged spectrum; Step S2, performing a Fast Fourier Transform on the continuous video frames to obtain the complex spectrum, amplitude spectrum, phase spectrum, and spatiotemporal frequency domain tensor, and extracting the corresponding layer features of the encoder through phase correlation calculation, frequency domain warping alignment, and waterlogged frequency domain masking; Step S3, constructing a learnable bi-peak spectrum template characterizing the low-frequency specular reflection peak and high-frequency ripple disturbance band, and matching it with the amplitude spectrum to obtain the physically enhanced amplitude spectrum; Step S4, constructing... Multiple overlapping soft frequency bands are selected, and the enhanced amplitude spectrum after competitive selection is obtained through a differentiable competitive selection mechanism; Step S5: Phase consistency edge features are extracted, and cross-fusion features are obtained through amplitude-gated signals and phase-gated signals; Step S6: Multi-stage upsampling decoding is performed to obtain the waterlogged area segmentation probability map and waterlogged area segmentation results; Step S7: A composite loss function including segmentation loss, decoupling loss, temporal consistency loss, physical constraint loss and edge loss is constructed, and the waterlogged area segmentation model is trained; Step S8: The trained model is used to process the monitoring video to be detected, and the dynamic water level estimation results are obtained by combining the reference calibration information and high-frequency ripple disturbance information, and the urban flooding warning results are output.

[0011] According to a second aspect of this disclosure, a system for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling is provided. This system includes a video acquisition and preprocessing module, a time-series frequency domain consistent coding module, a waterlogged physical spectrum modeling module, a frequency band contention selection module, an adaptive gating decoupling interaction module, a multi-scale physical sensing decoding module, a model training module, and a water level estimation and early warning module.

[0012] According to a third aspect of this disclosure, a computer device is provided. The computer device includes a memory and a processor, the processor executing a computer program stored in the memory to implement the method described above.

[0013] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, implements the method described above.

[0014] (III) Beneficial Effects As can be seen from the above technical solution, this disclosure has at least one of the following beneficial effects compared to the prior art: 1. Improve the temporal stability of water accumulation region segmentation in complex scenarios. In this disclosure, by performing frequency domain transformation on consecutive video frames, the displacement field is estimated by utilizing the phase correlation between the complex spectra of adjacent frames, and frequency domain warping alignment constraints are applied to the amplitude spectrum within the water accumulation frequency domain mask. This enables the model to utilize the relatively stable spectral structure of the water accumulation region between adjacent frames, thereby improving the continuity and stability of the segmentation results in the video sequence.

[0015] 2. Improve the physical rationality of frequency domain feature selection In this disclosure, a learnable bi-peak spectral template characterizing the low-frequency specular reflection peak and the high-frequency ripple disturbance band is constructed, enabling the frequency domain enhancement process to be constrained by the reflection characteristics of the water surface and the ripple disturbance characteristics, thereby improving the interpretability and robustness of the frequency domain modeling.

[0016] 3. Improve frequency band adaptability under different lighting and water surface conditions. In this disclosure, by dividing the frequency domain into multiple overlapping soft bands and using a differentiable competitive selection mechanism to calculate the selection weight of each soft band, it is possible to adaptively focus on the frequency band related to water accumulation identification based on the spectral response in the current video frame.

[0017] 4. Improve the ability to identify water accumulation boundaries and perform amplitude and phase fusion. In this disclosure, edge features are extracted through phase consistency response, and amplitude branch features, phase branch features, and amplitude-phase cross-modulation features are dynamically fused through amplitude gating signals and phase gating signals to improve the recognition capability under conditions of low contrast, strong reflection, and gradual water boundary changes.

[0018] 5. Improve the constraint integrity of segmentation model training. In this disclosure, a composite loss function consisting of segmentation loss, decoupling loss, temporal consistency loss, physical constraint loss, and edge loss is used to simultaneously constrain the model training by pixel classification accuracy, amplitude and phase branch decoupling, adjacent frame spectrum stability, physical spectrum template rationality, and boundary consistency.

[0019] 6. Improve the reliability of dynamic water level estimation and urban flooding early warning. In this disclosure, by combining the waterlogged area segmentation results with the calibration information of a preset reference object, and by using the high-frequency ripple disturbance information in the learnable bi-peak spectrum template to determine the water surface fluctuation correction factor, the water level estimation deviation caused by ripples, reflections or water surface disturbances can be corrected, thereby improving the practical value of the urban flooding early warning results. Attached Figure Description

[0020] Figure 1 This is a flowchart of a method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling, according to an embodiment of this disclosure.

[0021] Figure 2This is a schematic diagram of the technical architecture and data flow of the waterlogged area segmentation method in urban flooding video based on time-series frequency domain physical modeling, as described in this embodiment of the present disclosure. Detailed Implementation

[0022] This disclosure provides a method, system, computer device, and computer-readable storage medium for waterlogging area segmentation based on temporal frequency domain physical modeling in urban flooding videos. The method takes continuous monitoring video frames as input, constructs a spatiotemporal frequency domain representation through Fast Fourier Transform, and establishes temporal spectrum alignment constraints using the phase correlation of the complex spectra of adjacent frames. Simultaneously, a learnable bimodal spectrum template is constructed based on the low-frequency specular reflection and high-frequency ripple disturbance characteristics of the water surface. Segmentation features with physical prior constraints are generated through overlapping soft band competitive selection and amplitude-phase gating decoupling interaction. Furthermore, this disclosure utilizes a multi-scale physical perception decoder to output the waterlogging area segmentation results and combines reference object calibration information and high-frequency ripple disturbance information for water level estimation and correction, thereby improving the stability, boundary accuracy, and early warning reliability of waterlogging area identification in complex urban road scenarios.

[0023] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and accompanying drawings. It should be noted that the following embodiments are used to illustrate the technical solutions of this disclosure and are not intended to limit the scope of protection of this disclosure. For those skilled in the art, any adjustments made to the following parameters, model structure, number of frequency bands, loss weights, threshold conditions, and deployment methods without departing from the concept of this disclosure should be understood as falling within the scope of protection of this disclosure.

[0024] Please refer to Figure 1The waterlogging area segmentation method based on temporal frequency domain physical modeling disclosed herein generally includes steps S1 to S8. Step S1 is used to complete video temporal frame sampling and physical prior initialization; step S2 is used to construct a temporal frequency domain consistent encoder; step S3 is used to construct a waterlogging physical spectrum modeling module; step S4 is used to construct a frequency band competition selection attention mechanism; step S5 is used to construct an adaptive gating decoupling interaction mechanism; step S6 is used to output the waterlogging area segmentation result using a multi-scale physical perception decoder; step S7 is used to train the model using a composite physical-temporal loss function; and step S8 is used to perform dynamic water level physical estimation and early warning. The above steps together form a complete processing chain from monitoring video input, frequency domain feature modeling, physical prior constraints, segmentation result output to water level estimation and early warning. The overall technical solution of this disclosure will be described below according to steps S1 to S8. After step S8 is completed, the titles “Actual Scenario Embodiment 1” to “Actual Scenario Embodiment 5” will be retained, and each embodiment will explain which steps in the aforementioned overall solution and their specific scenario content correspond to each embodiment, so that the preceding general step description and the following specific scenario embodiments will form a clear correspondence.

[0025] I. Video temporal frame sampling and physical prior initialization, step S1 Step S1 is used to generate the timing input required for subsequent frequency domain modeling from the surveillance video and to provide initial priors for the physical spectrum template. This step includes sub-steps S1a to S1d.

[0026] Sub-step S1a: Using surveillance videos of low-lying urban road sections, underpasses, or flood-prone areas, extract continuous T-frame images from the surveillance videos according to a preset sampling rate to obtain a continuous T-frame image sequence.

[0027] Specifically, the surveillance video can come from fixed surveillance cameras on urban roads, surveillance cameras in underpasses, surveillance cameras in bridge areas, or video acquisition equipment at flood-prone locations. The preset sampling rate can be set according to the monitoring scenario and computing resources, for example, 10 to 15 frames per second. In one specific embodiment, the number of consecutive video frames is... This corresponds to a short video window of approximately 0.5 seconds. By employing continuous... Using frames as input, this embodiment can utilize the spectral stability of the water accumulation area in a short video sequence, rather than relying solely on the static spatial features of a single frame image.

[0028] Sub-step S1b: Using the continuous T-frame image sequence, perform size normalization and pixel value normalization on each frame image to obtain a normalized continuous video frame sequence.

[0029] Specifically, each frame of the image can be scaled to a uniform size. ,For example , For the first Original frame image Perform pixel value normalization to obtain the first pixel value. Frame normalized image It satisfies:

[0030] in, For the first Original image frame For the first Frame-normalized image, The mean of the dataset. Let be the standard deviation of the dataset. In one embodiment, and The channel mean and standard deviation obtained from the training dataset can be used, or the standard normalized parameters corresponding to the pre-trained visual model can be used. By normalizing the size and pixel values, this embodiment can reduce the impact of different camera resolutions, exposure levels, and brightness distributions on subsequent frequency domain analysis.

[0031] In sub-step S1c, during the model training phase, training samples are constructed using the continuous T-frame image sequence and the pixel-level water accumulation annotations corresponding to the T-th frame to obtain the training dataset.

[0032] Specifically, the training dataset can be represented as .in, Indicates continuity Frame image sequence, Indicates the first Pixel-level water accumulation annotations corresponding to the frames. When a pixel belongs to a water-filled area, its label value is 1; when a pixel belongs to a non-water-filled background, its label value is 0. A frame can be used as the current frame or the target frame, so that the model can output the water accumulation region segmentation result of the current frame based on the preceding temporal information.

[0033] Sub-step S1d uses the statistical prior of the water accumulation spectrum and the reflection characteristics of the water surface to determine the initial low-frequency peak and initial high-frequency band of the learnable physical spectrum template.

[0034] Specifically, water-filled surfaces typically exhibit a combination of low-frequency specular reflection dominance and high-frequency capillary ripple disturbances in the visual spectrum. Based on this characteristic, the initial center of the low-frequency peak of the learnable physical spectrum template can be set to coordinates close to the origin of the frequency domain, for example... Set the initial center of the high-frequency band to the mid-to-high frequency texture region, for example... The above coordinates can correspond to , The frequency domain coordinates at the image resolution. The initial low-frequency peak is used to initialize the low-frequency specular reflection peak, and the initial high-frequency band is used to initialize the high-frequency ripple disturbance band.

[0035] In this embodiment, step S1 enables subsequent steps to be calculated under the same time window, the same scale, and the same frequency domain reference by continuous frame sampling, unified normalization, and physical prior initialization. At the same time, by setting the initial positions of low-frequency peaks and high-frequency bands for the learnable physical spectrum template, the template learning can start from a reasonable region that conforms to the physical characteristics of water accumulation, reducing the risk of overfitting to non-water accumulation textures or background noise during the training process.

[0036] Those skilled in the art will understand that, in this embodiment, sampling is performed at 10 to 15 frames per second. as well as The advantage of this resolution-based approach is that it can balance short-term temporal continuity and computational efficiency. However, this disclosure is not limited to this. In other embodiments of this disclosure, the sampling rate can be adjusted based on the monitoring video frame rate, device computing power, and the complexity of the road scene. Setting the values ​​and normalized dimensions to other values ​​can also achieve the present disclosure, and has the advantage of adapting to different deployment conditions. This variation is also within the protection scope of the present disclosure.

[0037] II. Constructing a time-frequency domain consistent encoder, step S2 Step S2 is used to convert consecutive video frames into a frequency domain representation and establish temporal spectral consistency constraints through phase correlation and frequency domain warping alignment. This step includes sub-steps S2a to S2g.

[0038] Sub-step S2a: Using the normalized continuous video frame sequence, for the first... Perform a Fast Fourier Transform on the frame-normalized image to obtain the first frame. The complex spectrum of the frame, and the first The complex spectrum of the frame is decomposed into the first... Frame amplitude spectrum and the first Frame phase spectrum.

[0039] Specifically, for the first Frame normalized image Perform a Fast Fourier Transform to obtain the complex spectrum. :

[0040] In frequency domain coordinates At this point, the complex spectrum It can be represented as the real part. and the virtual part The combination of complex spectra. The amplitude spectrum can be calculated from the complex spectrum. and phase spectrum ,in:

[0041]

[0042] Furthermore, by Constructing a spatiotemporal frequency domain tensor from the complex spectrum corresponding to the frame image This spatiotemporal frequency domain tensor can include time dimensions, spatial frequency domain dimensions, and real and imaginary parts. Through this processing, the input video is transformed from a spatial domain image sequence into a spatiotemporal frequency domain representation, providing a foundation for subsequent spectrum alignment, physical template matching, and frequency band selection.

[0043] In sub-step S2b, the cross-power spectrum between adjacent frames is calculated using the complex spectrum corresponding to adjacent frames, and an inverse fast Fourier transform is performed on the cross-power spectrum to obtain the phase-correlated surface.

[0044] Specifically, for adjacent frames and complex spectrum and Calculate the cross power spectrum :

[0045] in, For the first Frame complex spectrum in frequency domain coordinates The complex conjugate at the location, To avoid the smoothing constant of division by zero, the cross-power spectrum was then analyzed. Perform a two-dimensional inverse fast Fourier transform and take the modulus to obtain the phase-correlated surface. :

[0046] The peak positions in the phase-correlated surface reflect the main displacement relationships between adjacent frames. Unlike estimating motion directly in the spatial domain, this embodiment uses the phase relationship between complex spectra to obtain displacement information, which can better capture the stable evolution of the overall reflection structure of the water accumulation area between adjacent frames.

[0047] In sub-step S2c, the peak coordinates of the phase-correlated surface are determined using the phase-correlated surface, and sub-pixel displacement estimation is performed in the neighborhood of the peak coordinates to obtain the displacement estimation results between adjacent frames, generating a dense displacement field; wherein, the displacement estimation results are used to characterize the pixel offset between adjacent frames; and then a frequency domain displacement field is obtained for frequency domain warping alignment of the amplitude spectrum of the next frame.

[0048] Specifically, the phase-dependent surface can be determined through the arg max operation. Peak coordinates To improve the accuracy of displacement estimation, adjustments can be made in the neighborhood of the peak coordinates. Sub-pixel refinement is performed using the centroid method. For example, it can be done using... Calculate subpixel displacement using a neighborhood window:

[0049]

[0050] The displacement estimation results described above are used to characterize the pixel offset between adjacent frames. In scenes with spreading water ripples, passing vehicles, or slight camera shake, these displacement estimation results can provide a basis for amplitude spectrum alignment.

[0051] Furthermore, to handle potential non-rigid local displacements in the waterlogged area, the amplitude spectrum of adjacent frames can be divided into... Several overlapping frequency domain blocks. For example, the size of the frequency domain block can be... The step size can be 32. Phase correlation calculation is performed on each overlapping frequency domain block to obtain the local displacement estimation result corresponding to each overlapping frequency domain block. Then, the local displacement estimation result is upsampled to the original frequency domain resolution through bilinear interpolation to obtain the dense displacement field. :

[0052] in, and The dense displacement field in the frequency domain coordinates are respectively The displacement components are located in two directions. This dense displacement field can be mapped to frequency domain coordinates for frequency domain warping alignment of the amplitude spectrum in the next frame.

[0053] In substep S2d, the displacement estimation results are used to perform bilinear interpolation sampling and frequency domain warping alignment on the amplitude spectrum of the next frame to obtain the aligned amplitude spectrum.

[0054] Specifically, for the first Frame Amplitude Spectrum Frequency domain warping is performed to obtain the aligned amplitude spectrum of the next frame. :

[0055] The amplitude values ​​at non-integer frequency domain coordinates can be obtained through differentiable bilinear interpolation. This process allows the amplitude spectrum of the next frame to be compared with the amplitude spectrum of the current frame after displacement compensation is estimated, thereby reducing the impact of camera jitter, overall water displacement, or local ripple disturbances on timing consistency constraints.

[0056] In sub-step S2e, the segmentation mask output by the network forward propagation of the previous iteration or the segmentation mask predicted by the previous frame is used to perform thresholding and morphological dilation processing on the segmentation mask to obtain the water accumulation frequency domain mask.

[0057] Specifically, during the model training phase, the segmentation mask output from the network forward propagation of the previous iteration can be used. Threshold it, for example, threshold And perform morphological dilation treatment, for example, using Structural elements are used to obtain the water accumulation frequency domain mask during the training phase. To avoid a training logic loop between temporal consistency loss and segmentation prediction, gradient backpropagation of the mask branch can be blocked by stopping gradient operations.

[0058] During the model inference phase, the segmentation mask predicted in the previous frame can be used. After morphological dilation, the water accumulation frequency domain mask for the inference stage is obtained. For the first frame of inference, since there is no prediction result from the previous frame, thresholding can be performed based on the spectral matching response between the physical spectrum template and the input amplitude spectrum to obtain the initialization result of the water accumulation frequency domain mask for the first frame. This setup enables continuous video monitoring without manual annotation.

[0059] Sub-step S2f, using the water accumulation frequency domain mask, the first The amplitude spectrum of the first frame and the amplitude spectrum of the next frame after alignment are used to construct a temporal spectrum alignment constraint to ensure that the amplitude spectra of adjacent frames remain consistent within the water-based frequency domain mask.

[0060] Specifically, time-series spectral alignment loss It can satisfy:

[0061] in, For water accumulation frequency domain masking, The amplitude spectrum of the aligned next frame in the frequency domain coordinates The value at that location, For the first Frame amplitude spectrum in frequency domain coordinates The value at that location. By applying temporal spectrum alignment constraints only within the water accumulation frequency domain mask, this embodiment can avoid excessive interference from whole-frame background motion, vehicle movement, and rain pattern flickering on the constraint terms.

[0062] Subsequently, by iteratively calculating the temporal spectrum alignment loss, a temporal spectrum alignment constraint is constructed to ensure that the amplitude spectrum of the water accumulation domain remains consistent.

[0063] In some embodiments, the time-series spectral alignment constraint can also be combined with the learnable bimodal spectral template generated in step S3. Coupling. Specifically, one can first use... For the Frame Amplitude Spectrum and the amplitude spectrum of the aligned next frame Weighting is performed to obtain the amplitude response after weighting using the physical spectrum template; subsequently, only the... Temporal spectral alignment constraints are applied to frequency bands with weighted energy exceeding a preset threshold. Thus, the temporal consistency loss primarily affects the frequency bands associated with low-frequency specular reflections from accumulated water and high-frequency ripple disturbances, while reducing the impact of rain patterns, high-frequency noise, non-accumulated water textures, and background reflections on the temporal consistency constraints. This process couples the temporal constraints in step S2 with the physical spectrum modeling in step S3, thereby enhancing the physical directivity of the frequency domain constraints.

[0064] In sub-step S2g, the complex spectrum, amplitude spectrum, phase spectrum, aligned amplitude spectrum and water-accumulated frequency domain mask are used to extract multi-layer frequency domain features through a time-series frequency domain consistency encoder to obtain the corresponding layer features of the encoder.

[0065] Specifically, the temporal frequency domain consistency encoder can employ a convolutional encoder, a Transformer encoder, or a hybrid encoder structure combining convolution and attention. The corresponding layer features of the encoder can serve as skip connection features in the subsequent multi-scale physical perception decoder, in order to preserve frequency domain and temporal information at different scales during the decoding process.

[0066] In this embodiment, step S2 estimates the displacement field through complex spectral phase correlation and uses the displacement field for frequency domain warping alignment of the amplitude spectrum, enabling the model to learn the spectral stability of the water accumulation region between adjacent frames. This structure can suppress inter-frame flicker caused by transient interference such as rain streaks, headlights, and vehicle shadows, and improve the temporal continuity of the video water accumulation segmentation results.

[0067] To verify the timing spectrum alignment constraints The effectiveness was assessed by comparing the complete protocol and the ablation protocol on the same rainy day test sequence. The results are shown in the table below.

[0068] Table 1. Verification Table of the Effectiveness of Temporal Spectral Alignment Constraints Configuration A (Complete Solution) Apply weighting coefficients 0.94 Consistent timing at the boundary, flicker-free Configuration B (ablation scheme) Remove 0.71 Noticeable inter-frame flickering and edge jitter. As shown in Table 1, on the same rainy day test sequence, the inter-frame IoU stability of the segmentation results of the complete scheme is 0.94, while that of the removed scheme is... The ablation scheme was reduced to 0.71. The water accumulation boundary of the ablation scheme showed obvious irregular jittering between adjacent frames, while the boundary of the complete scheme was temporally consistent. This result shows that temporal spectrum alignment constraints can effectively suppress inter-frame flicker caused by rain streaks, vehicle disturbances, and local reflection changes. The reason is that rain streaks appear as high-frequency transient fringes in the spatial domain, corresponding to directional frequency bands with drastic energy jitter in the frequency domain. The rain streak spectrum does not have a stable phase relationship between adjacent frames, and even after frequency domain warping alignment, there are still obvious energy differences, which will be... Natural penalties apply; however, the low-frequency specular reflection spectrum of the water surface has a relatively stable temporal structure, and the displacement field estimated by the phase correlation method can be aligned more accurately, resulting in a smaller loss. Therefore, step S2 can achieve adaptive suppression of transient disturbances.

[0069] Those skilled in the art will understand that in this embodiment, the use of phase correlation, overlapping frequency domain blocks, and bilinear interpolation to generate a dense displacement field has the advantage of balancing global translation and local non-rigid perturbations. However, this disclosure is not limited thereto. In other embodiments of this disclosure, other differentiable displacement estimation methods, other frequency domain block sizes, or other upsampling methods can be used to form a displacement field for amplitude spectrum alignment, which can also achieve this disclosure and has the advantage of adapting to different video resolutions and motion complexities. This variation is also within the protection scope of this disclosure.

[0070] III. Constructing the physical spectrum modeling module for water accumulation, step S3 Step S3 is used to construct a learnable bimodal spectral template based on the physical spectral characteristics of the water accumulation surface, and to use this template to enhance the frequency band response in the amplitude spectrum related to the water accumulation. This step includes sub-steps S3a to S3d.

[0071] Sub-step S3a: Using the initial low-frequency peak and the initial high-frequency band, initialize the template parameters of the learnable bi-peak spectrum template to obtain the initialized template parameters.

[0072] Specifically, the template parameters may include the center frequency of the low-frequency specular reflection peak. The center frequency of the high-frequency ripple disturbance band Bandwidth of low-frequency specular reflection peak Bandwidth of high-frequency ripple disturbance band and amplitude coefficient , In one embodiment, it can be set Or 0.8, Or 0.3, , The center frequency is initialized based on the initial low-frequency peak and initial high-frequency band determined in step S1. These parameters can be updated end-to-end during training.

[0073] Sub-step S3b: Using the initialized template parameters, construct a learnable bimodal spectral template.

[0074] Specifically, bimodal spectral templates can be learned. It can satisfy:

[0075] in, For learnable bimodal spectral templates in frequency domain coordinates The response value at the location; The center frequency of the low-frequency specular reflection peak; The center frequency of the high-frequency ripple disturbance band; The bandwidth of the low-frequency specular reflection peak; The bandwidth of the high-frequency ripple disturbance band; and The amplitude coefficient is used. The low-frequency specular reflection peak is used to characterize the low-frequency energy concentration caused by the relatively smooth reflection of the water surface, while the high-frequency ripple disturbance band is used to characterize the mid-to-high frequency texture response caused by fine ripples, capillary ripples, or disturbance reflections on the water surface.

[0076] Sub-step S3c involves element-wise multiplication of the learnable bimodal spectral template and the amplitude spectrum to obtain the physically enhanced amplitude spectrum.

[0077] Specifically, no. Amplitude spectrum after frame physical enhancement It can satisfy:

[0078] in, For the first Frame amplitude spectrum This indicates element-wise multiplication. Through this element-wise spectral matching, the frequency band that conforms to the physical spectral characteristics of the water is enhanced, while the background spectrum that does not conform to the physical spectral characteristics of the water is relatively suppressed.

[0079] In sub-step S3d, during model training, physical constraint loss is used to constrain the template parameters, so that the template parameters are updated within a preset range relative to the initial low-frequency peak and the initial high-frequency band, and a learnable bi-peak spectrum template is obtained after training.

[0080] Specifically, physical constraint loss The physical constraint loss can be determined based on the center frequency shift of the low-frequency specular reflection peak, the center frequency shift of the high-frequency ripple disturbance band, the bandwidth constraint term of the low-frequency specular reflection peak, the bandwidth constraint term of the high-frequency ripple disturbance band, and the amplitude coefficient shift. For example, the physical constraint loss may include the following:

[0081] in, and These represent one of the initial frequency domain coordinates for the low-frequency peak and the high-frequency band, respectively. and The initial reference value for the amplitude coefficient is used. Through physical constraint loss, a bimodal spectral template can be learned to retain its learning ability without deviating unrestrainedly from the reasonable spectral distribution of the water surface.

[0082] In this embodiment, step S3 uses a low-frequency specular reflection peak and a high-frequency ripple disturbance band to model the water accumulation spectrum in a bimodal manner, giving the spectrum enhancement process a clear physical meaning. This setting reduces the risk that the model may mistakenly focus on background texture, headlight reflection, or road stains due to reliance solely on data-driven learning, thereby improving the interpretability and cross-scene generalization ability of water accumulation recognition.

[0083] Those skilled in the art should understand that, in this embodiment, the use of a bimodal Gaussian spectral template has the advantage of simultaneously describing the spectral characteristics of both low-frequency reflections and high-frequency ripples in water accumulation with a relatively low number of parameters. However, this disclosure is not limited thereto. In other embodiments of this disclosure, multimodal Gaussian templates, radial basis function templates, learnable frequency domain masks, or other differentiable spectral templates can also be used to represent the prior water accumulation spectrum, achieving the same result and offering the advantage of adapting to more water surface conditions or camera imaging conditions. This variation is also within the scope of protection of this disclosure.

[0084] IV. Constructing a frequency band competitive selection attention mechanism, step S4 Step S4 is used to construct multiple overlapping soft frequency bands on the physically enhanced amplitude spectrum, and adaptively select key frequency bands related to water accumulation identification through a differentiable competitive selection mechanism. This step includes sub-steps S4a to S4i.

[0085] Sub-step S4a uses the physically enhanced amplitude spectrum and frequency domain coordinates, according to the preset number of frequency bands. Build By identifying overlapping soft frequency bands, the frequency band response corresponding to each soft frequency band is obtained.

[0086] Specifically, It is an integer greater than or equal to 3. In one embodiment, The eight soft bands can cover the equivalent frequency range from low to mid-high frequencies, thus taking into account both the low-frequency specular reflection band and the mid-high frequency ripple disturbance band. Overlapping areas can be set between the soft bands to form a soft band transition, avoiding the spectrum truncation caused by traditional hard-threshold low-pass or high-pass filtering.

[0087] Sub-step S4b, using the first Calculate the center frequency and bandwidth corresponding to the first soft band. The soft frequency band in the frequency domain coordinates Frequency band response at that location.

[0088] Specifically, we can first use frequency domain coordinates Calculate the radial frequency value ,For example . No. Band response of a soft band It can satisfy:

[0089] in, For the first The soft frequency band in the frequency domain coordinates The response value at that location, For the first The center frequency of each soft band For the first The bandwidth of a soft band. Multiple The distribution can increase in order from low frequency to mid-high frequency. In one embodiment, the center frequency can be preset based on image resolution and water accumulation spectrum statistics, or it can be adjusted as a learnable parameter. In a specific embodiment, when At that time, the center frequency of the 8 soft bands to The values ​​can be set sequentially to 4, 16, 32, 48, 64, 80, 96, and 112, with units in frequency domain coordinates; the bandwidth of each softband. It can be set to 8 and updated during training as a shared learnable parameter. With the above frequency band center and bandwidth settings, the low-frequency soft band can cover the water surface reflection area, the mid-to-high-frequency soft band can cover the water surface ripple disturbance area, and the overlap between adjacent frequency bands can reduce the information loss caused by hard band truncation.

[0090] Sub-step S4c, using the first The frequency band response of the first soft band and the amplitude spectrum after physical enhancement are weighted and pooled to obtain the first soft band. A soft-band frequency band descriptor.

[0091] Specifically, no. A soft band frequency descriptor It can satisfy:

[0092] in, and These are the height and width of the amplitude spectrum, respectively. For the first The amplitude spectrum of the physically enhanced frame in the frequency domain coordinates The value at that location. Frequency band descriptor. Capable of characterizing the physical enhancement amplitude spectrum in the first Energy response in each soft band.

[0093] Sub-step S4d uses the frequency band descriptors of each soft band to calculate the selection weight of each soft band through a differentiable competitive selection mechanism.

[0094] Specifically, the Gumbel-Softmax mechanism can be used to generate the first... The selection weight of each soft band :

[0095] in, For random noise that follows a Gumbel distribution, This refers to the temperature parameter. During the initial training phase... It can be set to a larger value to maintain differentiable soft selection between different soft bands; in the later stages of training, By gradually decreasing the weights, the selection weight distribution becomes sparser, allowing the model to more clearly focus on a few target frequency bands related to water accumulation identification.

[0096] In sub-step S4e, the selection weights of each soft band, the frequency response of each soft band, and the amplitude spectrum after physical enhancement are used to perform a weighted summation to obtain the enhanced amplitude spectrum after competitive selection.

[0097] Specifically, the enhancement spectrum after competitive selection It can satisfy:

[0098] in, For the first The selection weight of each soft band This indicates element-wise multiplication. Therefore, the responses of the physically enhanced amplitude spectrum in different soft bands are competitively weighted, resulting in a more focused frequency domain representation.

[0099] In sub-steps S4f to S4i, the number of frequency bands can be... Set to 8, and distribute the center frequencies of the 8 overlapping soft bands from low to mid-high frequencies in an increasing manner; adjacent soft bands can be set to have overlapping regions to reduce spectral truncation caused by hard thresholding; the temperature parameter can be adjusted during training. The selection weights are gradually reduced to allow the soft band selection to converge from differentiable soft selection to sparse band selection. Alternatively, the selection weights of each soft band can be used to form a band selection probability distribution, and the target band can be determined based on this distribution. The target band may include at least one of a low-frequency specular reflection correlation band and a mid-to-high-frequency ripple disturbance correlation band.

[0100] In this embodiment, step S4 uses an overlapping soft frequency band and a differentiable competitive selection mechanism to enable the model to automatically select key frequency bands based on the spectral response of the current scene. This setting avoids the limitations of fixed low-frequency and high-frequency hard threshold division, adapts to spectral drift under different lighting conditions, different water depths, and different water surface ripple conditions, thereby improving the flexibility and stability of water accumulation spectrum identification.

[0101] Those skilled in the art should understand that, in this embodiment, the following is employed: The soft-band and Gumbel-Softmax competitive selection mechanism has the advantage of enabling sparse band selection while ensuring differentiable training. However, this disclosure is not limited to this; in other embodiments of this disclosure, the number of soft bands... The center frequency set, bandwidth parameters, and temperature annealing strategy can all be adjusted according to the scene, image resolution, and training data, thus achieving the same result as this disclosure. Furthermore, this variation has the advantage of adapting to different monitoring deployment conditions, and it is also within the protection scope of this disclosure.

[0102] V. Construct an adaptive gating decoupling interaction mechanism, step S5 Step S5 is used to adaptively fuse the structural information in the amplitude spectrum and the edge information in the phase spectrum, and to maintain the frequency band specificity of different branches through decoupling loss. This step includes sub-steps S5a to S5e, and may further include decoupling loss calculation.

[0103] In sub-step S5a, the phase-consistent response is calculated using the phase spectrum and multi-scale amplitude response to obtain the phase-consistent edge response map.

[0104] Specifically, phase consistency response It can satisfy:

[0105] in, For the first Frame at pixel position Phase consistency response at the location, For the first Each scale at pixel position The amplitude response at that point, For the first Each scale at pixel position Phase deviation at that point For the first Scale weights for each scale For the first Noise threshold at each scale, is the numerical stability constant. The phase consistency response is relatively robust to changes in brightness, making it suitable for edge feature extraction in low-light, headlight reflection, and water boundary gradient scenarios.

[0106] In sub-step S5b, feature fusion is performed using the phase-consistent edge response map and the phase spectrum to obtain phase-consistent edge features.

[0107] Specifically, phase consistency edge features It can satisfy:

[0108] in, For the first Frame phase spectrum, Indicates channel cascading. This indicates a 3x3 convolution process. By fusing the phase-consistent edge response and phase spectrum, this embodiment can generate edge features that are more sensitive to water accumulation boundaries, local ripple edges, and reflective region boundaries.

[0109] In sub-step S5c, global average pooling and gated mapping are performed using the enhanced amplitude spectrum and phase consistency edge features after competitive selection to obtain amplitude-gated signals and phase-gated signals.

[0110] Specifically, amplitude gating signal and phase-gated signal It can satisfy:

[0111]

[0112] Wherein, GAP represents global average pooling. and For learnable weights, and For bias, This represents the Sigmoid function. The amplitude-gated signal is used to adjust the information contribution of the amplitude branch characteristic, and the phase-gated signal is used to adjust the information contribution of the phase branch characteristic.

[0113] In sub-step S5d, amplitude branch features, phase branch features, and amplitude-phase cross-modulation features are generated using amplitude-gated signals, phase-gated signals, enhanced amplitude spectra after competitive selection, and phase consistency edge features, respectively. These features are then dynamically fused to obtain cross-fused features.

[0114] Specifically, cross-integration features It can satisfy:

[0115] in, This represents a one-to-one convolution process. In the above formula, the first term corresponds to the amplitude branch feature, the second term to the phase branch feature, and the third term to the amplitude-phase cross-modulation feature. Through this structure, the model can adaptively switch between three modes—structure-dominated, edge-dominated, and collaborative interaction—based on the input content.

[0116] Sub-step S5e: When both the amplitude-gated signal and the phase-gated signal are less than the preset gating threshold, the contribution of the amplitude-phase cross-modulation feature to the cross-fusion feature is enhanced.

[0117] Specifically, in scenarios with complex lighting, unclear water boundaries, or significant local ripples, individual amplitude structure information and phase edge information may not be significantly dominant. In such cases, enhancing the contribution of cross-modulation features can preserve the nonlinear interaction information between the amplitude spectrum and the phase spectrum, thereby improving the discrimination ability in complex scenes.

[0118] In one specific embodiment, when local overexposure occurs due to nighttime vehicle headlight illumination, the amplitude spectrum is easily affected by brightness saturation, while the phase spectrum remains relatively stable to changes in illumination. In this situation, an adaptive gating decoupling interaction mechanism can adjust the amplitude gating signal to a lower value and the phase gating signal to a higher value, for example... , This reduces the contribution of amplitude branching features and enhances the contribution of phase branching features. When rain streaks, reflections, and boundary blurring coexist, it makes... and When both are below 0.5, the amplitude-phase cross-modulation characteristics are enhanced to preserve the model. and The nonlinear interaction information between them. Therefore, step S5 can dynamically adjust the contribution ratio of amplitude information, phase information, and cross-modulation information according to scene changes such as nighttime overexposure, strong reflection, rain streaks, and blurred boundaries.

[0119] Furthermore, during model training, the enhanced amplitude spectrum after competitive selection can be used. Phase consistency edge features Calculate decoupling loss :

[0120] in, This is a learnable low-pass filter constructed from low-frequency side soft band weights. This is a learnable high-pass filter constructed from high-frequency soft-band weights. A decoupling loss is used to encourage the amplitude branch to retain low-frequency structural information and the phase branch to retain high-frequency edge information, reducing feature redundancy between the two branches.

[0121] In this embodiment, step S5 achieves adaptive fusion of amplitude spectrum structure information and phase spectrum edge information through phase consistency edge features, bidirectional gating signals, and amplitude-phase cross-modulation terms. This setting can overcome the problem that simple splicing, element-by-element addition, or fixed-weight fusion methods are difficult to adapt to complex scenes, and improve the segmentation reliability under conditions of nighttime, reflection, low contrast, and gradual water boundary changes.

[0122] Those skilled in the art should understand that, in this embodiment, global average pooling and the Sigmoid function are used to generate the gated signal, which has the advantages of simple structure and low computational overhead. However, this disclosure is not limited thereto. In other embodiments of this disclosure, attention pooling, channel attention, spatial attention, or Transformer attention can also be used to generate the gated signal, which can also achieve the present disclosure and has the advantage of enhanced local region adaptability. Such variations are also within the protection scope of this disclosure.

[0123] VI. Perform physical-guided upsampling using a multi-scale physical sensing decoder, step S6 Step S6 is used to recover the cross-fused features into a high spatial resolution water accumulation region segmentation probability map, and to continuously introduce physical spectrum template constraints during the decoding process. This step includes sub-steps S6a to S6d.

[0124] In sub-step S6a, cross-fusion features are used as decoding input features, and corresponding encoder layer features are used as skip connection features to obtain upsampled fusion features.

[0125] Specifically, cross-integration features This includes the comprehensive frequency domain features after amplitude branching, phase branching, and amplitude-phase cross-modulation. Encoder corresponding layer features. It can provide frequency domain and temporal context at different scales. By using skip connections, the detailed information preserved at the encoding end is combined with the high-level semantic information at the decoding end, which can improve the boundary recovery capability.

[0126] Sub-step S6b uses learnable bimodal spectral templates of different scales to perform multi-stage upsampling decoding guided by physical templates on the features to be upsampled and fused, thereby obtaining the decoded features at each stage.

[0127] Specifically, a progressive four-stage upsampling approach can be adopted, with each stage performing a doubling of the upsampling. Stage decoding features It can satisfy:

[0128] in, This indicates double upsampling. For the encoder corresponding layer features, For the first Scale-learnable bimodal spectral template This indicates a 3x3 convolution process. It can be obtained from a learnable bimodal spectral template at the original scale through bilinear interpolation or downsampling. By introducing physical spectral templates at different scales at each decoding stage, this embodiment enables physical priors to be integrated throughout the segmentation result recovery process.

[0129] In sub-step S6c, the final stage decoding features, temporal average edge features, and temporal average physical enhancement amplitude spectrum are used for fusion prediction to obtain a water accumulation area segmentation probability map.

[0130] Specifically, the probability map for segmenting waterlogged areas. It can satisfy:

[0131] in, For the final stage of decoding features, For the first Frame phase consistency edge features For the first Amplitude spectrum after frame physical enhancement This represents the Sigmoid function. By fusing temporally averaged edge features and temporally averaged physical augmentation amplitude spectrum, the model can simultaneously utilize edge stability and physical spectrum stability when outputting a segmentation probability map.

[0132] In sub-step S6d, the water accumulation region segmentation probability map is used for thresholding to obtain the water accumulation region segmentation result.

[0133] Specifically, it can be Pixels larger than a preset threshold are identified as water accumulation areas, while pixels smaller than or equal to the preset threshold are identified as non-water accumulation areas, thus obtaining a binarized water accumulation area segmentation result. The preset threshold can be 0.5, or it can be adjusted according to the application scenario and false alarm control requirements.

[0134] In this embodiment, step S6, through skip-layer connections, multi-stage upsampling, and multi-scale physical template guidance, ensures that the water accumulation area segmentation results are subject to prior constraints from the physical spectrum while preserving spatial details. This setup improves the segmentation stability of small-area water accumulation, water accumulation with gradually changing boundaries, and complex reflective areas.

[0135] Those skilled in the art should understand that the four-stage, double-upsampling structure employed in this embodiment has the advantage of being suitable for gradually restoring the original resolution from low-resolution deep features. However, this disclosure is not limited thereto. In other embodiments of this disclosure, three-stage, five-stage, or other upsampling ratio decoding structures can also be used, as well as combinations of transposed convolution, subpixel convolution, or interpolated convolution, which can also achieve the present disclosure and have the advantage of adapting to different network backbones and resolution requirements. Such variations are also within the protection scope of this disclosure.

[0136] 7. Model training using the composite physics-temporal loss function, step S7 Step S7 is used to jointly constrain segmentation accuracy, amplitude-phase decoupling, temporal spectrum stability, physical template rationality, and boundary consistency during the training phase. This step includes sub-steps S7a to S7f.

[0137] In sub-step S7a, the segmentation loss is calculated using the water accumulation region segmentation results and the actual water accumulation annotations to obtain the segmentation loss value used to constrain the pixel classification results of the water accumulation region.

[0138] Specifically, segmenting losses This can include Dice loss and cross-entropy loss. Dice loss is used to improve the overlap between the foreground water accumulation area and the ground truth annotation, while cross-entropy loss is used to provide pixel-by-pixel classification constraints. By combining the two, it is possible to balance the overall region integrity and pixel-level classification accuracy even when the water accumulation area is uneven.

[0139] In one specific embodiment, segmentation loss It can satisfy:

[0140]

[0141]

[0142] in, pixel position Predicted probability of water accumulation in the area. pixel position The actual water accumulation at the location is indicated. and These are the image height and width, respectively. is the numerical stability constant.

[0143] In sub-step S7b, the enhanced amplitude spectrum and phase consistency edge features after competitive selection are used to calculate the decoupling loss, and the decoupling loss value used to constrain the frequency band specificity of the amplitude branch and phase branch is obtained.

[0144] Specifically, decoupling loss The form described in step S5 above can be used to constrain the amplitude branch and the phase branch respectively by using a learnable low-pass filter and a learnable high-pass filter, so that the amplitude branch is biased towards low-frequency structural information and the phase branch is biased towards high-frequency edge information.

[0145] In sub-step S7c, the aligned amplitude spectrum, amplitude spectrum, and water accumulation frequency domain mask are used to calculate the temporal consistency loss, and the temporal consistency loss value used to constrain the spectral stability of the water accumulation region in adjacent frames is obtained.

[0146] Specifically, the timing consistency loss can be implemented using step S2. This loss is used to ensure that the amplitude spectra of adjacent frames within the water accumulation frequency domain mask are aligned and remain consistent. This loss enables the model to learn the stable spectral characteristics of the water accumulation region in the video time sequence.

[0147] In sub-step S7d, the physical constraint loss is calculated using the template parameters of the learnable bimodal spectrum template and the corresponding initialization parameters to obtain the physical constraint loss value used to constrain the template parameters to be within a physically reasonable range.

[0148] Specifically, physical constraint loss Constraints can be imposed on the low-frequency peak center, high-frequency band center, low-frequency peak bandwidth, high-frequency band bandwidth, and amplitude coefficient to prevent the template parameters from deviating from the reasonable spectral range of the water surface.

[0149] In sub-step S7e, the predicted edge is generated using phase-consistent edge features, and the real edge is generated using real water accumulation annotations. The edge loss is calculated to obtain the edge loss value used to constrain the consistency of the water accumulation boundary.

[0150] Specifically, predicted edges can be generated from phase-consistent edge features through convolution and sigmoid activation, while true edges can be generated from true water accumulation annotations through Canny edge detection, Sobel operator, or other edge detection methods. Edge loss. A binary cross-entropy between predicted edges and true edges can be used. In one specific embodiment, the predicted edges... It can satisfy:

[0151] Edge loss It can satisfy:

[0152] in, Mark the true edge at pixel position The value at that location, To predict the edge at the pixel location The probability value at the location. By using edge loss constraints, the model can better maintain the continuity and clarity of water accumulation boundaries when outputting water accumulation area segmentation results.

[0153] In sub-step S7f, a composite loss function is constructed using segmentation loss, decoupling loss, temporal consistency loss, physical constraint loss, and edge loss. The model parameters are then updated based on the composite loss function to obtain the trained water accumulation area segmentation model.

[0154] Specifically, the composite loss function It can satisfy:

[0155] in, , , and These are the weighting coefficients for the corresponding loss terms. In one embodiment, It can be 0.1. It can be 0.1 or 0.2. It can be 0.05. It can be 0.1 or 0.2. By updating the network parameters through backpropagation and the optimizer, a trained water accumulation region segmentation model can be obtained.

[0156] In a specific training embodiment, a UFVD dataset can be constructed, which includes four scene categories: sunny, rainy, nighttime, and bright light or shadow. Each scene category can include 500 video clips, each 5 seconds long, containing 60 frames of images at a frame rate of 12 FPS. The dataset can be divided into training, validation, and test sets in an 8:1:1 ratio. Annotation information can include pixel-level water accumulation masks, water accumulation boundary contours, and reference object calibration information. Training samples can be in the form of long... Sliding time window construction, for example Adjacent training samples overlap by 4 frames. The input image can be scaled to... The mean and standard deviation of the data were normalized using the ImageNet standard.

[0157] In the above training example, the AdamW optimizer can be used for model training, and the initial learning rate can be set to... The model employs a cosine annealing learning rate strategy; the batch size can be set to 4; the maximum number of training epochs can be set to 100; and the early stopping patience value can be set to 10, meaning training stops when the validation set metric no longer improves for 10 consecutive epochs. During training, the model's mIoU gradually increases from an initial 0.42 to 0.787, triggering early stopping at the 87th epoch. The above training configuration and convergence process demonstrate that the composite loss function disclosed herein can be stably optimized within a finite number of training epochs, and enables the model to converge simultaneously in terms of region segmentation, temporal stability, and boundary recovery.

[0158] In this embodiment, step S7 constrains the model at different levels using a five-element composite loss function, ensuring that the model not only pursues accurate pixel classification but also focuses on temporal stability, physical rationality, amplitude-phase decoupling, and edge consistency. This setting improves the model's overall robustness in rainy, nighttime, strong reflective, and complex traffic disturbance scenarios.

[0159] To verify the overall performance advantage of our published method, PTF-Net, we conducted comparative experiments with existing mainstream video segmentation methods on a rainy day subset of our self-built UFVD dataset. This rainy day subset includes 50 video segments with a total of 2500 frames. All methods were trained on the same training set, with a unified input resolution. Time series dimension Only the timing method uses timing inputs. The comparison with the baseline method is shown in the table below.

[0160] Table 2. Baseline Comparison Methods DeepLabV3+ Single-frame image segmentation Spatial CNN baseline, ResNet-101 backbone SegFormer Single-frame image segmentation Transformer-based spatial segmentation baseline STCN Video object segmentation Temporal propagation networks utilize the mask of the previous frame. XMem Video object segmentation Long-sequence memory network SF-UNet Frequency domain segmentation Single-frame frequency domain U-Net baseline, a relatively close existing technology in this field. The quantitative results for sunny, rainy, and nighttime scenarios are shown in the table below.

[0161] Table 3. Quantitative Results of Different Methods DeepLabV3+ 72.3 58.1 51.4 60.6 0.62 SegFormer 75.8 61.3 54.2 63.8 0.65 STCN 78.5 65.7 58.9 67.7 0.78 XMem 79.2 66.4 59.5 68.4 0.80 SF-UNet 76.4 63.1 55.8 65.1 0.68 This public method (PTF-Net) 82.1 74.3 67.8 74.7 0.94 As shown in Tables 2 and 3, compared with single-frame spatial domain segmentation methods, video object segmentation methods, and single-frame frequency domain segmentation methods, the method disclosed in this paper achieves higher mIoU in sunny, rainy, and nighttime scenes, and reaches an inter-frame IoU stability index of 0.94. This result indicates that the temporal spectrum alignment in step S2, the physical spectrum modeling in step S3, the frequency band competition selection in step S4, the amplitude and phase gating decoupling in step S5, and the composite loss training in step S7 collectively improve the model's segmentation accuracy and temporal stability in complex weather, low-light, and water surface disturbance scenarios.

[0162] Those skilled in the art should understand that the combination of Dice loss, cross-entropy loss, decoupling loss, temporal consistency loss, physical constraint loss, and edge loss in this embodiment has the advantage of providing a more complete constraint dimension. However, this disclosure is not limited thereto. In other embodiments of this disclosure, the combination of loss terms or loss weights can be adjusted according to the training data scale, the degree of class imbalance, and the quality of boundary labels, which can also achieve the present disclosure and has the advantage of adapting to different training tasks. Such variations are also within the protection scope of this disclosure.

[0163] 8. Dynamic water level physical estimation and early warning, step S8 Step S8 is used to further convert the waterlogged area segmentation results into dynamic water level estimation results, and output an urban flooding warning based on the water level estimation results. This step includes sub-steps S8a to S8e.

[0164] Sub-step S8a: The trained water accumulation area segmentation model is used to process the surveillance video to be detected, and the water accumulation area segmentation result corresponding to the surveillance video to be detected is obtained.

[0165] Specifically, during the inference phase, the system can maintain a length of The frame buffer will be the most recent The trained water accumulation area segmentation model is input from the monitoring video to be detected. The water accumulation area segmentation probability map and water accumulation area segmentation result are obtained through the processing flow corresponding to steps S1 to S6.

[0166] In sub-step S8b, the water accumulation area segmentation results and preset reference calibration information are used to determine the pixel height difference corresponding to the water accumulation area, and the pixel scale is mapped to the physical scale based on the pixel height difference to obtain the initial water level estimation result.

[0167] Specifically, the pre-set reference object can be a water level gauge, traffic sign, curbstone, tunnel wall markings, or other fixed reference objects with a known actual height. Let the actual height of the reference object be... The reference pixel height is The vertical coordinate of the top of the reference object is The vertical coordinate of the top of the waterlogged area is Then the water level can be estimated by the pixel ratio.

[0168] Sub-step S8c uses high-frequency ripple disturbance information in the learnable bimodal spectral template to determine the surface ripple correction factor.

[0169] Specifically, the high-frequency ripple disturbance information may include at least one of the bandwidth, energy, or center frequency of the high-frequency ripple disturbance band. In one embodiment, the information can be based on the bandwidth of the high-frequency ripple disturbance band. Determine the water surface fluctuation correction factor with the preset normalization constant. . These can be normalization coefficients obtained from the training set calibration, for example... It can also be adaptively determined based on different camera pitch angles, reference distances, and the degree of water surface disturbance. The larger the high-frequency bandwidth, the more obvious the water surface ripples or local disturbances, and the more necessary the water level estimation needs to be corrected.

[0170] In sub-step S8d, the initial water level estimation result is corrected using a water surface fluctuation correction factor to obtain the dynamic water level estimation result.

[0171] Specifically, the dynamic water level estimation results It can satisfy:

[0172] in, This is the result of dynamic water level estimation. To pre-set the actual height of the reference object, To set the pixel height of the reference object, The top ordinate of the preset reference object. The vertical coordinate of the top of the waterlogged area is . For water surface fluctuation correction factor, This represents the bandwidth of the high-frequency ripple disturbance band. It should be noted that when the vertical axis of the image coordinate system is opposite to the actual height direction, the pixel height difference in the above formula can be expressed using absolute difference or equivalent transformation based on the coordinate direction to ensure that the water level estimation result is positive.

[0173] In one specific embodiment, if the initial water level estimate obtained solely based on the linear mapping from pixel to physical scale is 75cm, and the bandwidth of the high-frequency ripple disturbance band is... If it converges to 18.7, then... At that time, the dynamic water level estimate after water surface ripple correction was approximately 66.3 cm. Compared to the uncorrected linear mapping result, the corrected water level decreased by approximately 8.7 cm, corresponding to approximately 13.1% compensation for false water level height. This embodiment illustrates that when water surface ripples cause the water accumulation boundary to float, flicker, or mislocate, correction based on high-frequency ripple disturbance information can reduce water level estimation errors.

[0174] Sub-step S8e compares the dynamic water level estimation result with the preset water level threshold, and outputs the corresponding level of urban flooding warning result based on the comparison result.

[0175] Specifically, three levels of alerts can be set: yellow, orange, and red. For example, when... A yellow alert will be issued at any time; when An orange alert will be issued at any time; when A red alert will be issued in a timely manner. The alert result can be pushed to the city flood control platform, emergency command terminal, road maintenance terminal or other alert receiving equipment.

[0176] In this embodiment, step S8 combines the water accumulation area segmentation results with reference object calibration information, and further uses high-frequency ripple disturbance information to correct the water level estimation results, enabling the video water accumulation segmentation results to serve dynamic water level assessment and graded early warning. This setting can reduce the risk of falsely high or low water levels caused by water surface ripples, reflections, and boundary jitter, and improve the engineering reliability of urban flooding early warning results.

[0177] To verify the real-time performance and stability of this disclosure in continuous monitoring scenarios, multi-scenario continuous operation tests were conducted in a laboratory environment. The test conditions were: 72 hours of continuous inference; the input source consisted of four channels of pre-recorded 1080p video played in a loop, with H.264 encoding, and included 20 segments each for sunny days, rainy days, and nighttime; the hardware environment consisted of an Intel Xeon W-2235 @3.8GHz, an NVIDIA RTX 3090, and 64GB DDR4; the software environment consisted of PyTorch 1.12, TensorRT 8.4, and CUDA 11.6; the batch size was 1. Performance metrics are shown in the table below.

[0178] Table 4. Real-time Performance and Stability Indicators of the Project Average processing latency 35.7ms / frame After a 1000-frame warm-up, the GPU kernel execution time for the subsequent 5000 frames is recorded. Throughput 28.0 FPS Total frames / Total time, including I / O and post-processing Peak memory usage 6.8GB nvidia-smi monitoring maximum value 72-hour stability No crashes / no memory leaks Record system status every 6 hours. As shown in Table 4, this disclosure can achieve a throughput of 28.0 FPS under single-channel frame-by-frame inference conditions, with an average processing latency of 35.7 ms / frame, which can meet the real-time processing requirements of road video surveillance. At the same time, no crashes or memory leaks occurred during 72 hours of continuous operation, indicating that the method has the continuous operational stability required for engineering deployment.

[0179] Furthermore, in a simulated rainstorm video, the water level rose from 10cm to 45cm, with a video duration of 8 minutes and a frame rate of 12FPS. When the system detects that the water level has reached the orange warning threshold of 30cm, it can complete the audible and visual alarm within 3 seconds and push the notification to the preset duty terminal. This delay includes inference delay, post-processing and threshold judgment, network push, and terminal response, which can meet the requirements for timely warning response in urban flood monitoring.

[0180] Those skilled in the art should understand that, in this embodiment, the water surface fluctuation correction factor is determined using the high-frequency ripple disturbance bandwidth, which has the advantage of directly utilizing the water accumulation physical spectrum information learned in step S3. However, this disclosure is not limited thereto. In other embodiments of this disclosure, the water surface fluctuation correction factor can also be determined based on the high-frequency band energy, the high-frequency center frequency, the timing fluctuation amplitude, or a combination of the above characteristics. This can also achieve the present disclosure and has the advantage of adapting to different water surface disturbance conditions. This variation is also within the protection scope of this disclosure.

[0181] IX. Corresponding Scenario Examples It should be noted that the following embodiments are used to further illustrate the specific implementation of the aforementioned steps under different application scenarios, training conditions or engineering conditions, and do not change the data flow and processing logic between steps S1 to S8.

[0182] Real-world scenario example 1: Physical spectrum template initialization and adaptive learning in a sunny day scenario This embodiment provides an explanation of physical spectrum template initialization and adaptive learning in a sunny scenario, corresponding to steps S1 and S3 in the aforementioned overall scheme.

[0183] Specifically, in step S1, in clear weather monitoring scenarios such as low-lying urban roads, riverside roads, or underpasses, road surveillance cameras are used to collect road video. The surveillance video can be captured as short time-series video clips at a sampling rate of 12 frames per second, and five consecutive frames are extracted from these clips to obtain a continuous video frame sequence. The original resolution of each frame can be... Each frame of the image is scaled using bilinear interpolation to... Then, pixel value normalization is performed. Pixel value normalization can use the standard normalization parameters of the ImageNet pre-trained model, i.e. , And satisfy:

[0184] in, For the first Original image frame For the first Frame-normalized image, The channel mean. The standard deviation of the channel is given. Through the above processing, a normalized continuous video frame sequence can be obtained, providing input data with a uniform scale and pixel distribution for subsequent frequency domain transformation and physical spectrum template learning.

[0185] In the physical prior initialization of step S1, the theoretical reflection characteristics of the water surface can be calculated based on the Fresnel reflection equation. For the visible light band, the Fresnel reflectivity of the water-air interface... It can be represented as:

[0186] in, The refractive index of air, Let be the refractive index of water. Angle of incidence The angle of refraction. In one specific embodiment, , , The reflectivity can be obtained. This indicates that the surface of the water in a sunny setting exhibits predominantly low-frequency specular reflection, accompanied by high-frequency texture disturbances caused by water ripples. Based on this physical optical characteristic and the corresponding spectral distribution pattern, the initial frequency domain position of the learnable bimodal spectral template can be preset: the center of the low-frequency specular reflection peak is... The center of the high-frequency capillary corrugation band is The above coordinates are based on The statistical distribution of spectral energy in the water accumulation area at the image resolution is determined, but it does not represent the absolute physical frequency.

[0187] In step S3, a learnable physical spectrum template is constructed using the initial low-frequency peak and initial high-frequency band described above. Specifically, a bimodal spectrum template can be initialized. It satisfies:

[0188] in, The center frequency of the low-frequency specular reflection peak. The center frequency of the high-frequency ripple disturbance band. The bandwidth of the low-frequency specular reflection peak. This refers to the bandwidth of the high-frequency ripple disturbance band. and This refers to the amplitude coefficient. In one specific embodiment, the initial parameter can be set to... , , , Template parameters It can include , , , , , , and And it updates adaptively during training.

[0189] Furthermore, the learnable bimodal spectral template is multiplied element-wise with the amplitude spectrum obtained in step S2 to obtain the physically enhanced amplitude spectrum:

[0190] in, For the first Frame amplitude spectrum This indicates element-wise multiplication. The above treatment can enhance the low-frequency specular reflection response and high-frequency ripple disturbance response corresponding to water accumulation areas on sunny days, while suppressing irrelevant spectral responses from road surface texture, background noise, and non-water accumulation reflective areas.

[0191] This practical scenario illustrates that, under clear skies with relatively stable illumination and minimal rain ripple interference, the learnable bimodal spectral template can learn the typical distribution of low-frequency reflections and high-frequency ripples from a relatively clear water reflection spectrum. This template not only provides the foundation for the physical enhancement amplitude spectrum in step S3, but also offers reusable physical spectral priors for subsequent temporal spectral constraints in rainy scenarios, frequency band competition selection in nighttime scenarios, and water surface fluctuation correction in dynamic water level estimation.

[0192] Example 2: Time-series frequency domain consistency constraints in rainy weather scenarios This embodiment provides an explanation of performing time-series frequency domain consistency constraints in a rainy scenario, corresponding to step S2 in the aforementioned overall scheme.

[0193] Specifically, in step S2, the continuous video frames of the rainy day are... Perform Fast Fourier Transform on each frame to obtain the complex spectrum of adjacent frames. and And further obtain the amplitude spectrum , and phase spectrum , For adjacent frames and Calculate the cross power spectrum:

[0194] Subsequently, an inverse fast Fourier transform was performed on the cross-power spectrum to obtain the phase correlation surface:

[0195] The peak coordinates of the phase-correlated surface are used to estimate the main displacements between adjacent frames. In rainy scenes, due to factors such as water ripple diffusion, raindrop disturbance, passing vehicles, and slight camera shake, local non-rigid displacements may exist in the water accumulation area. Therefore, the amplitude spectrum can be divided into multiple overlapping frequency domain blocks, for example, a block size of... The step size is 32; phase correlation calculation is performed on each frequency domain block to obtain the local displacement estimation result, and the dense displacement field is obtained by bilinear interpolation upsampling. :

[0196] Based on this dense displacement field, frequency domain warping alignment is performed on the amplitude spectrum of the next frame:

[0197] During the training phase, the segmentation mask output from the network forward propagation of the previous iteration can be used. After thresholding and Morphological dilatation yields a water accumulation frequency domain mask Gradient backpropagation of the mask branch is blocked by stopping gradient operations. During inference, the water accumulation frequency domain mask can be obtained by morphological dilation of the previous frame's prediction segmentation mask. For the first frame of inference, a physical spectrum template can be used... The matching response to the input amplitude spectrum initializes the water accumulation frequency domain mask.

[0198] Using water accumulation frequency domain mask, the first The temporal spectrum alignment loss can be constructed from the frame amplitude spectrum and the aligned amplitude spectrum of the next frame:

[0199] In some implementations... It can also be used with learnable bimodal spectral templates. Coupling, only for Constraints are applied to frequency bands with weighted energy greater than a preset threshold, so that the timing consistency loss mainly affects the frequency bands related to low-frequency specular reflection from water accumulation and high-frequency ripple disturbance.

[0200] This practical scenario embodiment illustrates that in rainy weather, rain streaks appear as high-frequency transient fringes in the spatial domain and correspond to directional frequency bands with drastically fluctuating energy in the frequency domain, exhibiting no stable phase relationship between adjacent frames. In contrast, the low-frequency specular reflection spectrum of the water surface possesses a relatively stable temporal structure, allowing for accurate alignment of the displacement field estimated through phase correlation. Therefore, the temporal spectrum alignment loss can adaptively suppress rain streaks, vehicle disturbances, and localized reflection changes. Table 1 presents the complete solution and removal methods in this embodiment. The results of the stability comparison between the subsequent ablation schemes.

[0201] Example 3: Frequency band contention selection and gating decoupling in low-light nighttime scenarios This embodiment provides an explanation of frequency band contention selection and gating decoupling in low-light nighttime scenarios, corresponding to steps S4 and S5 in the aforementioned overall scheme.

[0202] Specifically, in step S4, for the complex spectral response formed by reflections from vehicle headlights, streetlights, building glass curtain walls, and road surface water at night, the physically enhanced amplitude spectrum is first used. Construction of frequency domain coordinates Several overlapping soft frequency bands. In one specific embodiment, The center frequencies of the 8 soft bands to The bandwidths were set sequentially to 4, 16, 32, 48, 64, 80, 96, and 112. It can be set to 8 and used as a shared learnable parameter. The frequency band response of each soft band satisfies:

[0203] Using frequency band response and the amplitude spectrum after physical enhancement Weighted pooling is performed to obtain the frequency band descriptor:

[0204] Then, the soft band selection weights are calculated using a differentiable competitive selection mechanism of the Gumbel-Softmax form:

[0205] in, For random noise that follows a Gumbel distribution, This refers to the temperature parameter. During the initial training phase... Larger, to maintain differentiable soft selection between soft bands; in the later stages of training, The frequency band selection is gradually reduced, causing it to converge towards the sparse target frequency band. Ultimately, the enhanced amplitude spectrum after competitive selection satisfies:

[0206] In step S5, the phase spectrum is used. Calculating phase-consistent response using multi-scale amplitude response:

[0207] Furthermore, the phase-consistent edge response map Phase spectrum By fusing the phase-consistent edge features, we obtain the following:

[0208] Subsequently, based on the enhanced amplitude spectrum after competitive selection Phase consistency edge features Amplitude-gated signals are generated through global average pooling and gating mapping. and phase-gated signal :

[0209]

[0210] Based on the above gated signal, the amplitude branch feature, phase branch feature, and amplitude-phase cross-modulation feature are dynamically fused to obtain the cross-fused feature:

[0211] For example, in overexposed car headlight scenes at night, the amplitude spectrum is easily affected by brightness saturation, and the amplitude gating signal can be reduced to [value missing]. The phase spectrum is relatively stable to changes in illumination, and the phase-gated signal can be increased to [value missing]. When rain streaks, reflections, and blurred boundaries coexist, it makes... and When both are below 0.5, the amplitude-phase cross-modulation characteristics can be enhanced to preserve... and Non-linear interactive information between them.

[0212] Furthermore, the decoupling loss can be calculated using the enhanced amplitude spectrum and phase consistency edge features after competitive selection:

[0213] in, This is a learnable low-pass filter constructed from low-frequency side soft band weights. This is a learnable high-pass filter constructed from the soft band weights on the high-frequency side. This practical scenario embodiment illustrates that steps S4 and S5 can adaptively determine the target frequency band and dynamically adjust the contribution ratios of amplitude, phase, and cross-modulation information under conditions of low light, strong reflection, overexposed vehicle lights, and blurred water boundaries at night, thereby improving the ability to identify water accumulation area boundaries.

[0214] Example 4: Model Training and Loss Function Configuration This embodiment provides an explanation of model training and loss function configuration, corresponding to step S7 in the aforementioned overall scheme, and together with the intermediate data generated in steps S1 to S6, forms a training closed loop.

[0215] Specifically, in step S7, the water accumulation area segmentation probability map output in step S6 is used. and actual water accumulation label Calculate the segmentation loss. The segmentation loss can be composed of the Dice loss and the cross-entropy loss:

[0216]

[0217]

[0218] Enhanced amplitude spectrum after competitive selection output from step S5 Phase consistency edge features Calculate decoupling loss The timing consistency loss is calculated using the aligned amplitude spectrum, the original amplitude spectrum, and the water-accumulated frequency domain mask output from step S2. The physical constraint loss is calculated using the learnable bimodal spectral template parameters and their initialization parameters from step S3. ; Generate predicted edges using phase consistency edge features And use real water accumulation to annotate and generate realistic edges. Calculate the edge loss:

[0219]

[0220] Finally, the five-element composite loss function can be expressed as:

[0221] in, , , and These are the weighting coefficients for the corresponding loss terms. In one specific embodiment, It can be 0.1. It can be 0.1 or 0.2. It can be 0.05. It can be 0.1 or 0.2.

[0222] In terms of dataset construction, a UFVD dataset can be built, which includes four scene categories: sunny, rainy, nighttime, and strong light or shadow. Each scene category can include 500 video clips, each 5 seconds long, containing 60 frames of images at a frame rate of 12 FPS. The dataset can be divided into training, validation, and test sets in an 8:1:1 ratio. Annotation information includes pixel-level water accumulation masks, water accumulation boundary contours, and reference object calibration information. Training samples can be in the form of [length missing]. Sliding time window construction, for example Adjacent training samples overlap by 4 frames. Input images can be uniformly scaled to [size missing]. The mean and standard deviation of the data were normalized using the ImageNet standard.

[0223] Regarding training parameters, the AdamW optimizer can be used for model training, with the initial learning rate set to... The model employs a cosine annealing learning rate strategy; the batch size is set to 4; the maximum number of training epochs is set to 100; and the early stopping patience value is set to 10, meaning training stops when the validation set metric no longer improves for 10 consecutive epochs. During training, the model's mIoU gradually increases from an initial 0.42 to 0.787, triggering early stopping at the 87th epoch. This practical scenario embodiment demonstrates that the composite physics-temporal loss function disclosed herein can be stably optimized within a finite number of training epochs, enabling the model to converge simultaneously in terms of region segmentation, temporal stability, physical constraints, and boundary recovery. Tables 2 and 3 present quantitative results comparing the baseline method and different scenarios in this embodiment, illustrating the effectiveness of the model trained with the composite loss function in terms of segmentation accuracy and inter-frame stability.

[0224] Real-world scenario example 5: Dynamic water level estimation and early warning and engineering verification This embodiment provides an explanation of dynamic water level estimation, early warning, and engineering verification, corresponding to step S8 in the aforementioned overall scheme.

[0225] Specifically, in step S8, the trained water accumulation area segmentation model is used to process the surveillance video to be detected, obtaining the water accumulation area segmentation result. Then, combined with preset reference object calibration information, the pixel height difference corresponding to the water accumulation area is determined, and this pixel height difference is mapped to the initial water level estimation result. Let the actual height of the reference object be... The reference pixel height is The vertical coordinate of the top of the reference object is The vertical coordinate of the top of the waterlogged area is Then, the water level can be estimated based on the pixel ratio.

[0226] Subsequently, the water surface fluctuation correction factor is determined using the high-frequency ripple disturbance information contained in the learnable bimodal spectral template from step S3. The high-frequency ripple disturbance information may include at least one of the bandwidth, energy, or center frequency of the high-frequency ripple disturbance band. Dynamic water level estimation results. It can satisfy:

[0227] in, For water surface fluctuation correction factor, This refers to the bandwidth of the high-frequency ripple disturbance band. In one specific embodiment, if the initial water level estimate obtained solely based on the linear mapping from pixel to physical scale is 75cm, and the bandwidth of the high-frequency ripple disturbance band... If it converges to 18.7, then... At that time, the dynamic water level estimate after surface ripple correction was approximately 66.3 cm. Compared to the uncorrected linear mapping result, the corrected water level decreased by approximately 8.7 cm, corresponding to a 13.1% compensation for false water level height. This result indicates that when surface ripples cause the water accumulation boundary to float, flicker, or mislocate, correction based on high-frequency ripple disturbance information can reduce the water level estimation error.

[0228] Furthermore, tiered early warnings can be output based on the comparison between dynamic water level estimation results and preset water level thresholds. When A yellow alert will be issued at any time; when An orange alert will be issued at any time; when A red alert is issued in real time. Table 4 shows the real-time performance and stability indicators of the project in this embodiment, demonstrating that the method can support continuous video monitoring and alert output.

[0229] In engineering verification, a 72-hour continuous inference test can be conducted in a laboratory environment. The input source consists of four channels of pre-recorded 1080p video played in a loop. The videos are H.264 encoded and include 20 segments each of sunny, rainy, and nighttime conditions. The hardware environment is an Intel Xeon W-2235 @ 3.8GHz, NVIDIA RTX 3090, and 64GB DDR4. The software environment is PyTorch 1.12, TensorRT 8.4, and CUDA 11.6. The batch size is 1. Under this configuration, the average processing latency reaches 35.7ms / frame, the throughput reaches 28.0 FPS, the peak memory usage is approximately 6.8GB, and there are no crashes or memory leaks during 72 hours of continuous operation. In a simulated rainstorm video, the water level rises from 10cm to 45cm, the video duration is 8 minutes, and the frame rate is 12FPS. When the system detects that the water level has reached the orange warning threshold of 30cm, it can complete the audible and visual alarm within 3 seconds and push the notification to the preset duty terminal. This practical scenario embodiment illustrates that step S8 not only utilizes the water accumulation area segmentation results output in step S6, but also uses the high-frequency ripple disturbance information in step S3 to physically correct the water level estimation, thereby improving the reliability of the urban flooding early warning results.

[0230] In a second exemplary embodiment of this disclosure, a system for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling is provided. Please refer to... Figure 2 The system includes a video acquisition and preprocessing module, a time-series frequency domain consistency coding module, a water accumulation physical spectrum modeling module, a frequency band competition selection module, an adaptive gating decoupling interaction module, a multi-scale physical sensing decoding module, a model training module, and a water level estimation and early warning module.

[0231] The video acquisition and preprocessing module acquires surveillance videos of low-lying urban road sections, underpasses, or flood-prone areas, and outputs a normalized continuous video frame sequence as described in step S1. The temporal-frequency domain consistency coding module outputs complex spectrum, amplitude spectrum, phase spectrum, aligned amplitude spectrum, water accumulation frequency domain mask, and encoder corresponding layer features as described in step S2. The water accumulation physical spectrum modeling module outputs a learnable bimodal spectrum template and physically enhanced amplitude spectrum as described in step S3. The frequency band competition selection module outputs the enhanced amplitude spectrum after competition selection as described in step S4. The adaptive gating decoupling interaction module outputs cross-fusion features as described in step S5. The multi-scale physical sensing decoding module outputs the water accumulation area segmentation result as described in step S6. The model training module trains the water accumulation area segmentation model as described in step S7. The water level estimation and early warning module outputs dynamic water level estimation results and urban flooding early warning results as described in step S8.

[0232] Through the aforementioned modular system architecture, this disclosure can be deployed in urban flood monitoring platforms, edge computing nodes, smart water management platforms, or emergency command and dispatch platforms. Each module can be centrally deployed on the same computing device or distributed across video access gateways, edge computing devices, and a central server. The system can handle real-time video access, model training, online inference, and early warning output, facilitating integration with existing urban monitoring systems and flood control platforms.

[0233] In a third exemplary embodiment of this disclosure, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it can implement the steps of the above-described method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling. The computer device can be a server, edge computing device, industrial control computer, graphics processing workstation, cloud computing node, or other computing device with image processing and model reasoning capabilities.

[0234] In a fourth exemplary embodiment of this disclosure, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program that, when executed by a processor, can implement the steps of the above-described method for segmenting flooded areas in urban flooding videos based on time-series frequency domain physical modeling. The computer-readable storage medium may include a read-only memory, random access memory, a magnetic disk, an optical disk, flash memory, a solid-state drive, or other media capable of storing program code.

[0235] In summary, this disclosure establishes a complete technology chain from inputting flooding video to segmenting waterlogged areas, correcting water levels, and issuing early warnings. This chain utilizes continuous video frame sampling, temporal-frequency domain consistency coding, learnable bimodal physical spectrum templates, competitive selection of overlapping soft frequency bands, amplitude-phase gated decoupling interaction, multi-scale physical perception decoding, composite physical-temporal loss training, and dynamic water level physical estimation and early warning. This technology chain can improve the temporal stability, frequency domain interpretability, boundary accuracy, and reliability of engineering applications in complex urban road scenarios.

[0236] To further illustrate the overall differences between this disclosure and the prior art, the differences between this disclosure and common prior art in key technical features are summarized in the table below.

[0237] Table 5. Key Differences from Existing Technologies Input Dimensions Single frame image Temporal video sequence, frames Frequency domain modeling Fixed high and low frequency hard partitioning Learnable physical spectrum templates and soft band competition Timing constraints No time-series spectral constraints Temporal Spectral Alignment Loss physical constraints Lack of physical priors for water accumulation surfaces Fresnel reflection priors and spectral characteristics of water surface waves Branch Interaction Channel cascading or element-wise addition Adaptive gating selection and amplitude-phase cross-modulation Frequency band selection Fixed ideal filter or hard threshold Differentiable soft band contention selection Edge detection Spatial domain Sobel or ordinary edge operator Frequency Domain Phase Coherence and Timing Accumulation loss function Segmentation loss or simple decoupling loss Five-element composite loss, including temporal constraints and physical constraints. Water level estimation Pixel-to-physical-scale linear mapping Nonlinear mapping for water surface ripple correction As shown in Table 5, this disclosure is not merely a simple replacement of existing semantic segmentation networks, but rather represents a continuous improvement in multiple aspects, including input dimension, frequency domain modeling, temporal constraints, physical constraints, branch interactions, frequency band selection, boundary restoration, loss function, and water level estimation. These improvements collectively enable this disclosure to maintain good segmentation stability and engineering applicability under complex interference such as rain patterns, reflections, car headlights, shadows, and water ripples.

[0238] It should be noted that for certain implementations, if they are not key content of this disclosure and are well known to those skilled in the art, they are not described in detail in the accompanying drawings or text due to space limitations. In such cases, reference can be made to relevant prior art for understanding. Regarding the numerical values ​​and ranges mentioned in this disclosure, unless explicitly stated otherwise, the numerical parameters in the specification and claims of this disclosure may be approximate values ​​and can be changed according to the content of this disclosure. Specifically, all numbers in the specification and claims representing size, scale, frequency, threshold, weight, sampling rate, time window, resolution, or other parameters should be understood to be modified by the term "about" in all cases, which means that a specific quantity may vary by ±10% in some embodiments.

[0239] The ordinal numbers used in this disclosure, such as "first," "second," "third," "primary," "secondary," and Arabic numerals and letters, to modify corresponding elements, modules, features, or steps, are intended only to clearly distinguish one element, module, feature, or step with a certain name from another element, module, feature, or step with the same name. They do not imply that the element, module, feature, or step has any ordinal number, nor do they represent the order of one element, module, feature, or step with another. Regarding the steps in this disclosure, unless specifically described or steps must occur sequentially, the order of the steps is not limited to those listed above and can be varied or rearranged according to the desired design.

[0240] Those skilled in the art will understand that the modules or steps of this disclosure described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by the computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this disclosure is not limited to any particular combination of hardware and software.

[0241] This disclosure can also be implemented as a device or apparatus program for performing part or all of the methods described herein, such as a computer program and a computer program product. Such an implementation of the disclosure can be stored on a computer-readable medium or can take the form of one or more signals. Such signals can be downloaded from a network, provided on a carrier signal, or provided in any other form.

[0242] This disclosure can be implemented using hardware comprising several different elements and a suitably programmed computer. Various component embodiments of this disclosure can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. The physical implementation of the hardware structure includes, but is not limited to, physical devices, including, but not limited to, transistors, memristors, microcontrollers, microprocessors, or digital signal processors. Furthermore, this disclosure is not directed to any particular programming language. It should be understood that the contents of this disclosure can be implemented using various programming languages, and the description of specific languages ​​herein is for the purpose of disclosing the best mode of implementation of this disclosure.

[0243] Those skilled in the art will understand that in the claims and description of this disclosure, the word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element or step does not exclude the presence of a plurality of such elements or steps. Furthermore, the embodiments provided above are intended only to enable this disclosure to meet legal requirements, and this disclosure can be implemented in many different forms and should not be construed as limited to the embodiments set forth herein.

[0244] Similarly, it should be understood that, for the sake of brevity, in the foregoing description of exemplary embodiments of this disclosure, various features of this disclosure are sometimes grouped together in a single embodiment, figure, or description thereof. However, this approach to disclosure should not be construed as reflecting an intention that the claimed disclosure requires more features than are expressly recited in each claim. Rather, as reflected in the claims, each aspect of the disclosure comprises fewer than all the features of the preceding single embodiment. Furthermore, embodiments may be used in combination with each other or with other embodiments based on design and reliability considerations; that is, technical features from different embodiments can be freely combined to form more embodiments. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this disclosure.

[0245] The above specific embodiments have provided a detailed description of the purpose, technical means, and beneficial effects of this disclosure. It should be understood that the purpose of the detailed description is to enable those skilled in the art to better understand this disclosure, and it is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling, characterized in that, include: Step S1: Using surveillance videos from low-lying urban road sections, underpasses, or flood-prone areas, extract a continuous T-frame image sequence at a preset sampling rate, and perform size normalization and pixel value normalization to obtain a normalized continuous video frame sequence; determine the initial low-frequency peak and initial high-frequency band of the physical spectrum template based on the statistical prior of the water accumulation spectrum. Step S2: Using the normalized continuous video frame sequence, perform a Fast Fourier Transform on each frame to obtain the complex spectrum, amplitude spectrum, phase spectrum, and spatiotemporal frequency domain tensor; use the complex spectrum of adjacent frames to perform phase correlation calculation, estimate the displacement field, and use the displacement field to perform frequency domain warping alignment on the amplitude spectrum of the next frame to obtain the aligned amplitude spectrum; obtain the water accumulation frequency domain mask and the corresponding layer features of the encoder based on the output mask of the previous iteration step or the prediction mask of the previous frame. Step S3: Using the initial low-frequency peak, the initial high-frequency band, and the amplitude spectrum, construct a learnable bi-peak spectrum template characterizing the low-frequency specular reflection peak and the high-frequency ripple perturbation band, and match the learnable bi-peak spectrum template with the amplitude spectrum to obtain the physically enhanced amplitude spectrum; Step S4: Using the physically enhanced amplitude spectrum, the frequency domain is divided into multiple overlapping soft bands, and the selection weight of each soft band is calculated through a differentiable competitive selection mechanism to obtain the enhanced amplitude spectrum after competitive selection. Step S5: Use the phase spectrum and the enhanced amplitude spectrum after competitive selection to extract phase consistency edge features, and generate amplitude-gated signals and phase-gated signals; fuse amplitude branch features, phase branch features and amplitude-phase cross-modulation features according to the amplitude-gated signals and phase-gated signals to obtain cross-fusion features; Step S6: Use the cross-fusion features, the corresponding layer features of the encoder, and learnable bimodal spectral templates of different scales to perform multi-stage upsampling decoding to obtain the water accumulation region segmentation probability map and the water accumulation region segmentation result. Step S7: During model training, the water accumulation region segmentation results, real water accumulation labels, aligned amplitude spectra, learnable bimodal spectrum templates, enhanced amplitude spectra after competitive selection, and phase consistency edge features are used to construct a composite loss function including segmentation loss, decoupling loss, temporal consistency loss, physical constraint loss, and edge loss. Based on this, the model parameters are optimized to obtain the trained water accumulation region segmentation model. Step S8: Use the trained waterlogged area segmentation model to process the monitoring video to be detected, and obtain the waterlogged area segmentation result corresponding to the monitoring video to be detected; combine the preset reference calibration information and the high-frequency ripple disturbance information in the learnable bimodal spectrum template to correct the water level estimation result, obtain the dynamic water level estimation result, and output the urban flooding warning result.

2. The method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling as described in claim 1, characterized in that, Step S1 includes: Sub-step S1a: Using surveillance videos of low-lying urban road sections, underpasses, or flood-prone areas, extract T consecutive frames of images from the surveillance videos according to a preset sampling rate to obtain a sequence of T consecutive frames of images; where T is an integer greater than or equal to 2. Sub-step S1b involves using the continuous T-frame image sequence to perform size normalization and pixel value normalization on each frame, resulting in a normalized continuous video frame sequence; wherein the pixel value normalization process satisfies: , For the original image of frame t, For the normalized image of frame t, The mean of the dataset. The standard deviation of the dataset. ; In sub-step S1c, during the model training phase, training samples are constructed using the continuous T-frame image sequence and the pixel-level water accumulation annotations corresponding to the T-th frame to obtain the training dataset; wherein, the pixel-level water accumulation annotations are used to indicate whether each pixel in the T-th frame belongs to the water accumulation area; Sub-step S1d uses the statistical prior of the water accumulation spectrum and the reflection characteristics of the water surface to determine the initial low-frequency peak and the initial high-frequency band of the physical spectrum template; wherein, the initial low-frequency peak is used to initialize the low-frequency specular reflection peak, and the initial high-frequency band is used to initialize the high-frequency ripple disturbance band.

3. The method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling according to claim 1, characterized in that, Step S2 includes: Sub-step S2a involves using the normalized continuous video frame sequence to perform a Fast Fourier Transform on the normalized image of frame t, obtaining the complex spectrum of frame t, and decomposing the complex spectrum of frame t into the amplitude spectrum and phase spectrum of frame t; constructing a spatiotemporal frequency domain tensor from the complex spectra corresponding to the images of frames T; wherein... ; Sub-step S2b involves using the complex spectra corresponding to adjacent frames to calculate the cross-power spectrum between adjacent frames, and performing an inverse fast Fourier transform on the cross-power spectrum to obtain a phase-correlated surface; wherein the cross-power spectrum satisfies: Let the complex spectrum of frame t be in the frequency domain coordinates The value at that location, The complex spectrum of the (t+1)th frame in the frequency domain coordinates The complex conjugate at the location, To avoid the smoothing constant of division by zero; Sub-step S2c: Using the phase-correlated surface, determine the peak coordinates of the phase-correlated surface, and perform sub-pixel displacement estimation in the neighborhood of the peak coordinates to obtain displacement estimation results between adjacent frames, generating a dense displacement field; wherein, the displacement estimation results are used to characterize the pixel offset between adjacent frames; thus obtaining a frequency domain displacement field for frequency domain warping alignment of the amplitude spectrum of the next frame; Sub-step S2d involves using the displacement estimation result to perform bilinear interpolation sampling and frequency domain warping alignment on the amplitude spectrum of the next frame, resulting in an aligned amplitude spectrum; wherein the aligned amplitude spectrum satisfies: For the amplitude spectrum of the next frame after alignment, The amplitude spectrum of the (t+1)th frame. and The displacement field in the frequency domain coordinates are respectively The displacement components along two directions; In sub-step S2e, the segmentation mask output by the network forward propagation of the previous iteration or the segmentation mask predicted by the previous frame is used to perform thresholding and morphological dilation processing on the segmentation mask to obtain the water accumulation frequency domain mask; wherein, during the model training phase, gradient backpropagation of the corresponding branch of the water accumulation frequency domain mask is blocked by stopping gradient operations. Sub-step S2f: Using the water-filled frequency domain mask, the amplitude spectrum of the t-th frame, and the aligned amplitude spectrum of the next frame, a temporal spectrum alignment constraint is constructed to constrain the amplitude spectra of adjacent frames to remain consistent within the water-filled frequency domain mask. In sub-step S2g, the complex spectrum, the amplitude spectrum, the phase spectrum, the aligned amplitude spectrum, and the water-accumulated frequency domain mask are used to extract multi-layer frequency domain features through a time-series frequency domain consistency encoder to obtain the corresponding layer features of the encoder.

4. The method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling according to claim 3, characterized in that, The sub-step S2c further includes: In subdivision step S2c1, the amplitude spectrum corresponding to adjacent frames is used to divide the amplitude spectrum into multiple overlapping frequency domain blocks, resulting in M×N overlapping frequency domain blocks; where M and N are both positive integers. In sub-step S2c2, the complex spectrum corresponding to each overlapping frequency domain block is used to perform phase correlation calculations within each overlapping frequency domain block to obtain the local displacement estimation results corresponding to each overlapping frequency domain block. In sub-step S2c3, the local displacement estimation results corresponding to each overlapping frequency domain block are used to upsample to the original frequency domain resolution through bilinear interpolation to obtain a dense displacement field. Wherein, the dense displacement field satisfies: , The frequency domain coordinates between frame t and frame t+1 Dense displacement field at that location, and These are the displacement components of the dense displacement field along two directions, respectively; In sub-step S2c4, the dense displacement field is mapped to frequency domain coordinates to obtain a frequency domain displacement field for frequency domain warping alignment of the amplitude spectrum in the next frame.

5. The method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling according to claim 3, characterized in that, The sub-step S2e includes: In sub-step S2e1, during the model training phase, the network forward propagation output segmentation mask from the previous iteration step is used to perform thresholding and morphological dilation on the network forward propagation output segmentation mask from the previous iteration step, and gradient backpropagation is blocked by stopping gradient operations to obtain the water accumulation frequency domain mask for the training phase. In sub-step S2e2, during the model inference stage, the previous frame's prediction segmentation mask is used to perform morphological dilation processing on the previous frame's prediction segmentation mask to obtain the water accumulation frequency domain mask for the inference stage; and / or, The sub-step S2f includes: In sub-step S2f1, the temporal spectrum alignment loss is calculated using the water-accumulated frequency domain mask, the amplitude spectrum of the t-th frame, and the aligned amplitude spectrum of the next frame; wherein, the temporal spectrum alignment loss satisfies: For time-series spectral alignment loss, For the water accumulation frequency domain mask, The amplitude spectrum of the aligned next frame in the frequency domain coordinates The value at that location, Let the amplitude spectrum of frame t be in the frequency domain coordinates The value at this location, where T is the number of frames in the continuous video frame sequence; In the sub-step S2f2, the temporal spectrum alignment loss is calculated iteratively to construct the temporal spectrum alignment constraint, thereby ensuring that the amplitude spectrum of the water accumulation domain remains consistent.

6. The method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling according to claim 1, characterized in that, Step S3 includes: Sub-step S3a: Using the initial low-frequency peak and the initial high-frequency band, initialize the template parameters of the learnable bi-peak spectrum template to obtain the initialized template parameters; wherein, the template parameters include the center frequency of the low-frequency specular reflection peak, the center frequency of the high-frequency ripple disturbance band, the bandwidth of the low-frequency specular reflection peak, the bandwidth of the high-frequency ripple disturbance band, and the corresponding amplitude coefficient. Sub-step S3b involves constructing a learnable bimodal spectral template using the initialized template parameters; wherein the learnable bimodal spectral template satisfies: The learnable bimodal spectral template in frequency domain coordinates The response value at that location, The center frequency of the low-frequency specular reflection peak. The center frequency of the high-frequency ripple disturbance band. The bandwidth of the low-frequency specular reflection peak. This refers to the bandwidth of the high-frequency ripple disturbance band. and The amplitude coefficient; Sub-step S3c involves element-wise multiplication of the learnable bimodal spectral template and the amplitude spectrum to obtain the physically enhanced amplitude spectrum; wherein the physically enhanced amplitude spectrum satisfies: , The amplitude spectrum after physical enhancement in frame t. Let be the amplitude spectrum of the t-th frame. This indicates element-wise multiplication; In sub-step S3d, during model training, physical constraint loss is used to constrain the template parameters so that the template parameters are updated within a preset range relative to the initial low-frequency peak and the initial high-frequency band, and a learnable bi-peak spectrum template is obtained after training; wherein, the physical constraint loss is determined at least based on the center frequency offset of the low-frequency specular reflection peak, the center frequency offset of the high-frequency ripple perturbation band, the bandwidth constraint term of the low-frequency specular reflection peak, the bandwidth constraint term of the high-frequency ripple perturbation band, and the amplitude coefficient offset.

7. The method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling according to claim 1, characterized in that, Step S4 includes: Sub-step S4a: Using the physically enhanced amplitude spectrum and frequency domain coordinates, construct K overlapping soft frequency bands according to the preset number of frequency bands K, and obtain the frequency band response corresponding to each soft frequency band; where K is an integer greater than or equal to 3; Sub-step S4b: Using the center frequency and bandwidth corresponding to the k-th soft band, calculate the frequency domain coordinates of the k-th soft band. The frequency band response at a given location; wherein the frequency band response satisfies: The k-th soft band in frequency domain coordinates The response value at that location, For frequency domain coordinates The determined radial frequency value, Let k be the center frequency of the kth soft band. Let k be the bandwidth of the soft band. ; Sub-step S4c involves performing weighted pooling on the band response of the k-th soft band and the physically enhanced amplitude spectrum to obtain the band descriptor of the k-th soft band; wherein the band descriptor satisfies: Let H be the frequency band descriptor for the k-th soft band, and H and W be the height and width of the amplitude spectrum, respectively. Let the amplitude spectrum of the physically enhanced frame t be in the frequency domain coordinates. The value at; Sub-step S4d involves using the band descriptors of each soft band to calculate the selection weight of each soft band through a differentiable competitive selection mechanism; wherein the selection weight satisfies: The selection weight for the k-th soft band. For random noise that follows a Gumbel distribution, For temperature parameters; Sub-step S4e involves weighted summation of the selection weights of each soft band, the band response of each soft band, and the physically enhanced amplitude spectrum to obtain the enhanced amplitude spectrum after competitive selection; wherein the enhanced amplitude spectrum after competitive selection satisfies: The enhanced amplitude spectrum after competitive selection in frame t. This indicates element-wise multiplication.

8. The method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling according to claim 7, characterized in that, Step S4 further includes: Sub-step S4f: Set the number of frequency bands K to 8, and distribute the center frequencies of the 8 overlapping soft frequency bands in ascending order from low frequency to mid-high frequency, so as to cover the low frequency specular reflection frequency band and mid-high frequency ripple disturbance frequency band required for water accumulation identification. Sub-step S4g sets adjacent soft bands to have overlapping regions, so that soft bands can form a soft band transition, thereby reducing the spectrum cutoff caused by hard threshold band division. Sub-step S4h uses temperature parameters during model training. Adjust the sparsity of the selection weights for each soft band, and make the temperature parameter... The soft band selection is gradually reduced during the training process so that it converges from differentiable soft selection to sparse band selection. Sub-step S4i: Use the selection weights of each soft band to form a band selection probability distribution, and determine the target band based on the band selection probability distribution; wherein, the target band includes at least one of the low-frequency specular reflection correlation band and the mid-to-high frequency ripple disturbance correlation band.

9. The method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling according to claim 1, characterized in that, Step S5 includes: Sub-step S5a: Using the phase spectrum and multi-scale amplitude response, calculate the phase coherence response to obtain a phase coherence edge response map; wherein, the phase coherence response satisfies: Let x be the phase consistency response at pixel position x in frame t. Let x be the magnitude response at pixel position x for the nth scale. Let x be the phase deviation at pixel position x for the nth scale. The scale weights for the nth scale. Let n be the noise threshold for the nth scale. It is the numerical stability constant; Sub-step S5b involves fusing the phase-consistent edge response map and the phase spectrum to obtain phase-consistent edge features; wherein the phase-consistent edge features satisfy: , For the phase consistency edge features of the t-th frame, Let be the phase spectrum of the t-th frame. Indicates channel cascading. This indicates a 3x3 convolution process; Sub-step S5c involves performing global average pooling and gated mapping using the enhanced amplitude spectrum after competitive selection and the phase consistency edge features to obtain amplitude-gated signals and phase-gated signals; wherein the amplitude-gated signals and the phase-gated signals respectively satisfy: For amplitude-gated signals, For phase-gated signals, GAP represents global average pooling. and For learnable weights, and For bias, Represents the Sigmoid function; In sub-step S5d, using the amplitude-gated signal, the phase-gated signal, the enhanced amplitude spectrum after competitive selection, and the phase consistency edge feature, amplitude branch feature, phase branch feature, and amplitude-phase cross-modulation feature are generated respectively. The amplitude branch feature, phase branch feature, and amplitude-phase cross-modulation feature are then dynamically fused to obtain a cross-fused feature; wherein the cross-fused feature satisfies: For the cross-fusion features of frame t, This represents a one-to-one convolution process; In sub-step S5e, when both the amplitude-gated signal and the phase-gated signal are less than a preset gating threshold, the contribution of the amplitude-phase cross-modulation feature to the cross-fusion feature is enhanced, so that the cross-fusion feature retains nonlinear amplitude-phase interaction information when neither structural information nor edge information is dominant.

10. The method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling according to claim 9, characterized in that, The decoupling loss is calculated using the enhanced amplitude spectrum after the competitive selection and the phase consistency edge features; wherein the decoupling loss satisfies: For decoupling loss, The enhanced amplitude spectrum after competitive selection in frame t. For the phase consistency edge features of the t-th frame, This is a learnable low-pass filter constructed from low-frequency side soft band weights. It is a learnable high-pass filter constructed from the soft band weights on the high-frequency side.

11. The method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling according to claim 1, characterized in that, Step S6 includes: Sub-step S6a: Use the cross-fusion feature as the decoding input feature and use the corresponding layer feature of the encoder as the skip connection feature to obtain the upsampled fusion feature; Sub-step S6b involves using learnable bimodal spectral templates of different scales to perform physical template-guided multi-stage upsampling decoding on the features to be upsampled and fused, obtaining the decoded features at each stage; wherein the decoded features at the k-th stage satisfy: For the decoding features of the k-th stage, This indicates double upsampling. For the encoder corresponding layer features, For the k-th scale, it is a learnable bimodal spectral template. This indicates a 3x3 convolution process, where k is the upsampling stage number; Sub-step S6c involves fusing the final stage decoding features, temporally averaged edge features, and temporally averaged physical enhancement amplitude spectrum for prediction to obtain a water accumulation region segmentation probability map; wherein, the water accumulation region segmentation probability map satisfies: Probability map for segmenting waterlogged areas. For the final stage of decoding features, For the phase consistency edge features of the t-th frame, The amplitude spectrum after physical enhancement in frame t. Represents the Sigmoid function; In sub-step S6d, the water accumulation area segmentation probability map is used for thresholding to obtain the water accumulation area segmentation result.

12. The method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling according to claim 1, characterized in that, Step S7 includes: Sub-step S7a: Using the water accumulation region segmentation result and the real water accumulation label, calculate the segmentation loss to obtain the segmentation loss value used to constrain the pixel classification result of the water accumulation region; Sub-step S7b: Using the enhanced amplitude spectrum after the competitive selection and the phase consistency edge features, calculate the decoupling loss to obtain the decoupling loss value used to constrain the frequency band specificity of the amplitude branch and the phase branch; Sub-step S7c: Using the aligned amplitude spectrum, the amplitude spectrum, and the water accumulation frequency domain mask, calculate the temporal consistency loss to obtain the temporal consistency loss value used to constrain the spectral stability of the water accumulation region in adjacent frames; Sub-step S7d: Using the template parameters and corresponding initialization parameters of the learnable bimodal spectrum template, calculate the physical constraint loss to obtain the physical constraint loss value used to constrain the template parameters to be within a physically reasonable range; Sub-step S7e: Use the phase consistency edge features to generate predicted edges and use the real water accumulation annotations to generate real edges, calculate the edge loss, and obtain the edge loss value used to constrain the consistency of the water accumulation boundary; Sub-step S7f involves constructing a composite loss function using the segmentation loss value, the decoupling loss value, the temporal consistency loss value, the physical constraint loss value, and the edge loss value, and updating the model parameters based on the composite loss function to obtain the trained water accumulation region segmentation model; wherein, the composite loss function satisfies: For composite loss function, To divide the loss, For decoupling loss, For time-series consistency loss, For physical constraint loss, For edge loss, , , and These are the weighting coefficients for the corresponding loss terms.

13. The method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling according to claim 12, characterized in that, The segmentation loss includes Dice loss and cross-entropy loss, and the edge loss includes binary cross-entropy between predicted edges and ground truth edges; wherein, the predicted edges are generated by convolution and sigmoid activation of the phase-consistent edge features, and the ground truth edges are generated by edge detection of the ground truth water accumulation labels.

14. The method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling according to claim 1, characterized in that, Step S8 includes: Sub-step S8a: Use the trained water accumulation area segmentation model to process the surveillance video to be detected, and obtain the water accumulation area segmentation result corresponding to the surveillance video to be detected; Sub-step S8b: Using the water accumulation area segmentation result and the preset reference calibration information, determine the pixel height difference corresponding to the water accumulation area, and perform a mapping from pixel scale to physical scale based on the pixel height difference to obtain the initial water level estimation result; Sub-step S8c: Use the high-frequency ripple disturbance information in the learnable bimodal spectrum template to determine the surface wave correction factor; wherein, the high-frequency ripple disturbance information includes at least one of the bandwidth, energy or center frequency of the high-frequency ripple disturbance band; Sub-step S8d involves correcting the initial water level estimation result using the water surface fluctuation correction factor to obtain a dynamic water level estimation result; wherein the dynamic water level estimation result satisfies: h represents the dynamic water level estimation result. To pre-set the actual height of the reference object, To set the pixel height of the reference object, The top ordinate of the preset reference object. The vertical coordinate of the top of the waterlogged area is . For water surface fluctuation correction factor, The bandwidth of the high-frequency ripple disturbance band; Sub-step S8e: The dynamic water level estimation result is compared with the preset water level threshold, and the corresponding level of waterlogging warning result is output based on the comparison result.

15. The method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling according to claim 14, characterized in that, The water surface fluctuation correction factor is determined based on the bandwidth of the high-frequency ripple disturbance band and the preset normalization constant; when the dynamic water level estimation result is in the first threshold interval, the second threshold interval and the third threshold interval, a yellow warning, an orange warning and a red warning are output respectively.

16. A system for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling, characterized in that, include: The video acquisition and preprocessing module is used to acquire surveillance videos of low-lying urban road sections, underpasses, or flood-prone areas, extract a continuous T-frame image sequence according to a preset sampling rate, and perform size normalization and pixel value normalization to obtain a normalized continuous video frame sequence. The temporal frequency domain consistency coding module is used to perform a fast Fourier transform on each frame using the normalized continuous video frame sequence to obtain the complex spectrum, amplitude spectrum, phase spectrum and spatiotemporal frequency domain tensor, and obtain the aligned amplitude spectrum and encoder corresponding layer features based on phase correlation calculation and water accumulation frequency domain mask. The water accumulation physical spectrum modeling module is used to construct a learnable bi-peak spectrum template that characterizes the low-frequency specular reflection peak and the high-frequency ripple disturbance band, and to match the learnable bi-peak spectrum template with the amplitude spectrum to obtain the physically enhanced amplitude spectrum. The frequency band competition selection module is used to construct multiple overlapping soft frequency bands using the physically enhanced amplitude spectrum, and to calculate the selection weight of each soft frequency band through a differentiable competition selection mechanism to obtain the enhanced amplitude spectrum after competition selection. An adaptive gated decoupling interaction module is used to extract phase consistency edge features using the phase spectrum and the enhanced amplitude spectrum after competitive selection, and to obtain cross-fusion features through amplitude-gated signals and phase-gated signals; The multi-scale physical perception decoding module is used to perform multi-stage upsampling decoding using the cross-fusion features, the corresponding layer features of the encoder, and learnable bimodal spectral templates at different scales to obtain the water accumulation region segmentation result. The model training module is used to construct a composite loss function that includes segmentation loss, decoupling loss, temporal consistency loss, physical constraint loss and edge loss, and optimize the model parameters based on the composite loss function to obtain the trained water accumulation area segmentation model. The water level estimation and early warning module is used to process the monitoring video to be detected using the trained water accumulation area segmentation model, obtain the water accumulation area segmentation result, obtain the dynamic water level estimation result based on the preset reference calibration information and high frequency ripple disturbance information, and output the waterlogging early warning result.

17. A computer device, characterized in that, include: A memory on which computer programs are stored; A processor, when executing the computer program, implements the method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling as described in any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program; When the computer program is executed by the processor, it implements the method for segmenting waterlogged areas in urban flooding videos based on time-series frequency domain physical modeling as described in any one of claims 1 to 15.

Images