A flash flood disaster real-time monitoring method and system fusing radar and video

Abstract

The present application relates to mountain torrent monitoring technical field, especially to a kind of mountain torrent disaster real-time monitoring method and system fusing radar and video.It includes:through radar equipment and video equipment respectively obtain water flow velocity signal and mountain torrent area image sequence, and through time stamp calibration mechanism alignment acquisition data frequency difference, form preliminary alignment of time and space multi-source data set;From data set extract motion feature vector and boundary contour feature, and filter smooth processing is carried out to motion feature, remove noise;Correct time stamp offset, ensure data time uniformity, and through spatial mapping, motion feature and boundary contour feature are associated, and fusion feature map is generated;Through fusion feature map, the corresponding relationship of water flow velocity and boundary contour is calculated, dynamic hydrological parameter distribution map is generated, high-risk disaster range is marked, and comprehensive monitoring report is generated, and mountain torrent disaster state is determined in real time.The present application effectively improves the real-time monitoring precision and early warning capability of mountain torrent disaster.

Description

Technical Field

[0001] This invention relates to the field of flash flood monitoring technology, and in particular to a method and system for real-time monitoring of flash flood disasters that integrates radar and video. Background Technology

[0002] With the intensification of extreme weather and climate change, the frequency and destructive power of flash floods are constantly increasing, posing a serious threat to people's lives and property. Flash floods usually occur rapidly and suddenly, and traditional monitoring methods are insufficient to capture real-time changes in water flow and the scope of the disaster, resulting in insufficient early warning and delayed emergency response. Currently, most flash flood monitoring systems rely on a single data source, such as radar or video surveillance, but each has obvious shortcomings: radar can provide water flow data over a wide area, but its temporal resolution is low, making it difficult to capture rapidly changing water flow dynamics; video surveillance can provide intuitive image information, but it is limited by the field of view and environmental interference, and lacks accurate water flow dynamic parameters.

[0003] Furthermore, existing technologies face challenges in multi-source data fusion and spatiotemporal alignment, particularly due to the difference in acquisition frequencies between radar and video equipment, leading to spatiotemporal inconsistencies in their data. This makes accurate synchronous analysis and early warning difficult at the moment a disaster occurs. To address these issues, an efficient and precise monitoring method is needed that can fully integrate multi-source data to improve the monitoring accuracy and response speed for flash flood disasters.

[0004] Therefore, how to achieve spatiotemporal data fusion from radar and video equipment to monitor and accurately warn of flash floods in real time has become a key technical problem that urgently needs to be solved in the field of disaster prevention and mitigation. This invention provides an innovative method for real-time monitoring of flash floods by fusing radar and video data. Through precise alignment and fusion of multi-source data, it significantly improves the timeliness and accuracy of disaster monitoring. Summary of the Invention

[0005] This invention provides a method and system for real-time monitoring of flash floods that integrates radar and video, for accurate monitoring and early warning of flash flood dynamics.

[0006] In a first aspect, the present invention provides a method for real-time monitoring of flash floods that integrates radar and video, comprising:

[0007] Step S1: The water flow velocity signal and the image sequence of the flash flood area are acquired by radar equipment and video equipment respectively as the dynamic data stream of the flash flood area. The acquisition frequency difference of the dynamic data stream is aligned by the timestamp calibration mechanism to obtain a multi-source dataset with preliminary spatiotemporal alignment.

[0008] Step S2: Extract motion feature vectors and boundary contour features from the multi-source dataset, and perform filtering and smoothing processing on the motion feature vectors to obtain a feature set after noise removal;

[0009] Step S3: Correct the timestamp offset of the motion feature vector and the boundary contour feature to obtain a time-unified feature set; perform spatial mapping transformation on the feature set, associate the motion feature vector with the boundary contour feature to obtain a cross-modal associated fusion feature map;

[0010] Step S4: Calculate the correspondence between water flow velocity and boundary contour using the fused feature map to obtain a dynamic hydrological parameter distribution map; mark the high-risk disaster area based on the dynamic hydrological parameter distribution map to obtain a marked disaster area map;

[0011] Step S5: Generate a comprehensive monitoring report based on the marked disaster range map to determine the real-time status of the flash flood disaster.

[0012] As a preferred embodiment of the present invention, step S1 involves obtaining a multi-source dataset with preliminary spatiotemporal alignment, including:

[0013] Water flow velocity signals are collected using radar equipment to generate a radar data stream containing timestamps; image sequences of flash flood areas are collected using video equipment to generate a video data stream containing timestamps; based on the timestamps of the radar data stream and the video data stream, a synchronization algorithm is used to align their acquisition frequencies; if the timestamp deviation between the radar data stream and the video data stream exceeds a preset threshold, the data alignment is adjusted using a timestamp interpolation method; based on the aligned radar data stream and video data stream, a multi-source dataset containing water flow velocity signals and image sequences with preliminary spatiotemporal alignment is generated; the timestamp consistency of the multi-source dataset is verified to obtain the final multi-source dataset with preliminary spatiotemporal alignment.

[0014] As a preferred embodiment of the present invention, step S2 includes:

[0015] The water flow velocity signal in the radar data stream is extracted from the multi-source dataset to generate a motion feature vector; image sequences in the video data stream are extracted from the multi-source dataset, and boundary contour features are generated using an edge detection algorithm; noise interference is smoothed using a Kalman filtering algorithm on the motion feature vector to generate a smoothed motion feature vector; the spatiotemporal consistency between the smoothed motion feature vector and the boundary contour features is verified; a noise-removed feature set containing motion and contour information is generated based on the smoothed motion feature vector and the boundary contour features; the noise-removed feature set is stored for subsequent processing.

[0016] As a preferred embodiment of the present invention, in step S3, the timestamp offset of the motion feature vector and the boundary contour feature is corrected to obtain a time-uniform feature set, including:

[0017] The deviation of the noise-removed feature set on the time axis is detected; if the deviation is greater than a preset threshold, the timestamp of the motion feature vector is corrected using a linear interpolation method; if the deviation is greater than the preset threshold, the timestamp of the boundary contour feature is corrected using a nonlinear interpolation method; an intermediate feature set with timestamp alignment is generated based on the corrected motion feature vector and the boundary contour feature; the timestamp consistency of the intermediate feature set is verified; based on the verification result, the timestamp of the intermediate feature set is adjusted to obtain a time-accurate and unified feature set.

[0018] As a preferred embodiment of the present invention, step S3, obtaining the fusion feature map of cross-modal correlation, includes:

[0019] Obtain radar coordinate system data from the time-unified feature set; project the radar coordinate system data onto the video pixel coordinate system to generate a spatially mapped feature set; for the spatially mapped feature set, use a feature matching algorithm to associate the motion feature vector with the boundary contour feature; based on the association result, generate cross-modal feature pairs containing motion features and contour features; generate a fused feature map based on the cross-modal feature pairs; verify the spatial consistency of the fused feature map to obtain the cross-modal associated fused feature map.

[0020] As a preferred embodiment of the present invention, step S4, obtaining the dynamic hydrological parameter distribution map, includes:

[0021] Feature pairs of water flow velocity and boundary contour are extracted from the fused feature map; a convolutional neural network is used to perform pixel-level correlation processing on the feature pairs to generate a correlation feature matrix; the spatiotemporal correspondence between water flow velocity and boundary contour is calculated based on the correlation feature matrix; an intermediate parameter map containing the water flow velocity distribution is generated based on the spatiotemporal correspondence; a smoothing algorithm is used to optimize the distribution continuity of the intermediate parameter map; and a dynamic hydrological parameter distribution map is generated based on the optimized intermediate parameter map.

[0022] As a preferred embodiment of the present invention, step S4 involves marking high-risk disaster areas based on the dynamic hydrological parameter distribution map to obtain a marked disaster area map, including:

[0023] Water flow velocity values ​​are extracted from the dynamic hydrological parameter distribution map; a threshold judgment mechanism is used to determine areas exceeding the normal range based on the water flow velocity values; a high-risk area mask is generated based on the areas exceeding the normal range; the high-risk area mask is applied to the dynamic hydrological parameter distribution map to generate an intermediate distribution map containing high-risk markers; the continuity of high-risk areas in the intermediate distribution map is verified; based on the verification results, the intermediate distribution map is adjusted to obtain a marked disaster range map.

[0024] As a preferred embodiment of the present invention, step S5, determining the real-time status of the flash flood disaster, includes:

[0025] Feature data of high-risk disaster areas are extracted from the marked disaster area map; the feature data is integrated with the associated features in the fused feature map to generate a comprehensive feature set; the comprehensive feature set is processed using a data aggregation algorithm to generate intermediate monitoring data; a comprehensive monitoring report containing disaster area and hydrological parameters is generated based on the intermediate monitoring data; the completeness of the comprehensive monitoring report is verified; the comprehensive monitoring report is adjusted based on the verification results to determine the real-time status of flash flood disasters.

[0026] Secondly, the present invention provides a real-time monitoring system for flash flood disasters that integrates radar and video, for implementing the above-mentioned method, the system comprising:

[0027] The data alignment unit is used to acquire water flow velocity signals and image sequences of flash flood areas through radar equipment and video equipment, respectively, as dynamic data streams of flash flood areas. The acquisition frequency differences of the dynamic data streams are aligned through a timestamp calibration mechanism to obtain a multi-source dataset with preliminary spatiotemporal alignment.

[0028] The feature extraction unit is used to extract motion feature vectors and boundary contour features from the multi-source dataset, and to perform filtering and smoothing processing on the motion feature vectors to obtain a feature set after noise removal.

[0029] The correction mapping unit is used to correct the timestamp offset between the motion feature vector and the boundary contour feature to obtain a time-unified feature set; the feature set is then spatially mapped and transformed to associate the motion feature vector with the boundary contour feature to obtain a cross-modal associated fusion feature map;

[0030] The distribution map marking unit is used to calculate the correspondence between water flow velocity and boundary contour through the fused feature map to obtain a dynamic hydrological parameter distribution map; and to mark the high-risk disaster area according to the dynamic hydrological parameter distribution map to obtain a marked disaster area map.

[0031] The report generation unit is used to generate a comprehensive monitoring report based on the marked disaster range map to determine the real-time status of flash flood disasters.

[0032] Thirdly, the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the above-described method.

[0033] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0034] This invention solves the data inconsistency problem caused by differences in acquisition frequency by synchronously acquiring data from radar and video equipment and using a timestamp calibration mechanism to initially align spatiotemporal data. Motion feature vectors and boundary contour features are extracted from the aligned dataset, and a filtering algorithm is used to remove noise, optimizing data smoothing and eliminating errors caused by environmental interference, providing reliable basic data for subsequent spatiotemporal analysis. Based on the corrected feature dataset, motion features and boundary contour features are associated through spatial mapping transformation, and cross-modal fusion is achieved through feature matching algorithms, ensuring accurate alignment of radar data and video images. Further calculation of the spatiotemporal correspondence between water flow velocity and boundary contour generates a dynamic hydrological parameter distribution map, enabling accurate identification of high-risk areas for flash floods. After marking high-risk areas, a comprehensive monitoring report is generated to determine the status of flash floods in real time, providing timely decision-making basis for disaster early warning and prevention. Through the synergy of the above technical solutions, the advantages of multiple data sources are fully utilized, improving the accuracy and efficiency of flash flood monitoring. Attached Figure Description

[0035] Figure 1 This is a flowchart of a real-time monitoring method for flash floods that integrates radar and video, according to the present invention.

[0036] Figure 2 This is a structural diagram of a real-time monitoring system for flash floods that integrates radar and video, according to the present invention. Detailed Implementation

[0037] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0038] like Figure 1 This embodiment presents a real-time monitoring method for flash floods that integrates radar and video, which may specifically include:

[0039] Step S1: Acquire water flow velocity signals and image sequences of flash flood areas using radar and video equipment respectively, as dynamic data streams of the flash flood area. Align the acquisition frequency differences of the dynamic data streams using a timestamp calibration mechanism to obtain a multi-source dataset with preliminary spatiotemporal alignment; specifically including:

[0040] Water flow velocity signals are collected using radar equipment to generate a radar data stream containing timestamps; image sequences of flash flood areas are collected using video equipment to generate a video data stream containing timestamps; based on the timestamps of the radar data stream and the video data stream, a synchronization algorithm is used to align their acquisition frequencies; if the timestamp deviation between the radar data stream and the video data stream exceeds a preset threshold, the data alignment is adjusted using a timestamp interpolation method; based on the aligned radar data stream and video data stream, a multi-source dataset containing water flow velocity signals and image sequences with preliminary spatiotemporal alignment is generated; the timestamp consistency of the multi-source dataset is verified to obtain the final multi-source dataset with preliminary spatiotemporal alignment.

[0041] Specifically, in this embodiment, to achieve real-time monitoring of flash floods, water flow velocity signals and image sequences of the flash flood area are first acquired using radar and video equipment, respectively. This data serves as a dynamic data stream of the flash flood area. A timestamp calibration mechanism is used to align the acquisition frequency differences of the dynamic data stream, ultimately resulting in a multi-source dataset with preliminary spatiotemporal alignment. The radar equipment is deployed at key locations in the flash flood area, utilizing the Doppler effect to measure water flow velocity. A timestamp accurate to milliseconds is added to each radar signal point, forming a continuous radar data stream. This allows for the capture of dynamic changes in water flow, providing a basis for time alignment. Simultaneously, video equipment, such as high-definition cameras, is installed near the radar equipment to continuously capture images of the flash flood area. Image sequences of the flood area are generated to ensure that the images capture the water flow boundaries and flow patterns. Each frame of the image is also timestamped to form a video data stream. After acquiring the radar data stream and the video data stream, a synchronization algorithm is used to align their acquisition frequencies based on their timestamps. The synchronization algorithm uses the timestamp matching principle to match the sampling points of the radar data stream with the time points of the video data stream. If there is a difference in the frequencies of the radar data stream and the video data stream, such as the radar device using the first sampling frequency and the video device using the second sampling frequency, the nearest neighbor matching algorithm is used to align them to a unified time grid to ensure accurate alignment of spatiotemporal data. This helps to reduce spatiotemporal misalignment caused by differences in device frequencies, thereby improving the accuracy of subsequent data fusion.

[0042] If the timestamp deviation between the radar data stream and the video data stream exceeds a preset threshold, a timestamp interpolation method is used for data alignment. In this interpolation method, a linear interpolation algorithm is used to calculate the data values ​​for missing time points. For example, when the timestamp deviation of the radar signal is large, an intermediate value is inserted into the radar data stream to ensure continuous alignment of the two data streams on the time axis. Linear interpolation calculates a weighted average by taking data points from adjacent timestamps to achieve a smooth transition, thereby correcting the time offset. To handle more complex deviation situations, this invention further extends the interpolation method by employing a polynomial interpolation algorithm, particularly... In rapidly changing scenarios such as flash floods, quadratic polynomial interpolation is used to correct timestamps. A more accurate intermediate data point is generated by fitting a curve based on three adjacent points, avoiding errors in linear interpolation methods under non-uniform change conditions. After data alignment, a spatiotemporally pre-aligned multi-source dataset containing water flow velocity signals and image sequences is generated. The accuracy is ensured by verifying the consistency of timestamps in this multi-source dataset. The verification process involves checking the continuity and deviation of all timestamps. If the deviation is below a preset threshold, the data alignment is confirmed as successful, and the final spatiotemporally pre-aligned multi-source dataset is output. Otherwise, the process returns to the interpolation step until the consistency requirements are met.

[0043] The above technical solution achieves precise alignment of multi-source data streams. Through synchronization algorithms and interpolation methods, it effectively solves the problem of spatiotemporal misalignment between devices with different acquisition frequencies, ensuring that radar and video data streams can remain consistent in time and space. This provides a reliable foundation for subsequent data processing and disaster monitoring, and lays a solid foundation for further dynamic hydrological parameter calculation and disaster risk labeling, significantly improving the accuracy and response speed of real-time monitoring of flash floods.

[0044] Step S2: Extract motion feature vectors and boundary contour features from the multi-source dataset, and perform filtering and smoothing processing on the motion feature vectors to obtain a feature set after noise removal; specifically including:

[0045] The water flow velocity signal in the radar data stream is extracted from the multi-source dataset to generate a motion feature vector; the image sequence in the video data stream is extracted from the multi-source dataset, and an edge detection algorithm is used to generate boundary contour features; the motion feature vector is filtered and smoothed to generate a smoothed motion feature vector; the spatiotemporal consistency between the smoothed motion feature vector and the boundary contour features is verified; based on the smoothed motion feature vector and the boundary contour features, a noise-removed feature set containing motion information and contour information is generated and stored.

[0046] Specifically, in this embodiment, the water flow velocity signal is first extracted from the radar data stream from the multi-source dataset. This process generates a motion feature vector. The radar device acquires the water flow velocity signal through the Doppler effect principle and uses timestamps to mark each data point, ensuring that the temporal and spatial information of the radar signal can accurately reflect the dynamic changes of the water flow. Image sequences are extracted from the video data stream from the multi-source dataset, and edge detection algorithms, such as the Canny edge detection algorithm, are used to generate boundary contour features. Noise is suppressed by Gaussian filtering, and then the gradient intensity and direction of each point in the image are calculated. Non-maximum suppression and double thresholding are used to detect edges, thereby accurately extracting the boundary contour features of the water flow and generating the corresponding feature vector. The above motion feature vector is then filtered and smoothed to generate a smoothed motion feature vector. To achieve noise removal, a Kalman filtering algorithm is optimized. This is a recursive estimation method that minimizes noise errors in the system through prediction and update stages. At each time step, the Kalman filter corrects the state estimate of the motion feature vector to generate accurate smooth values. For example, in turbulent water flow scenarios, the water velocity of the original radar signal may be affected by wind interference, resulting in large fluctuations. Filtering can reduce the bias, thereby significantly improving the robustness of the data and avoiding the impact of noise on subsequent data processing. For complex environments, such as flash flood monitoring under heavy rain conditions, the process noise covariance of the Kalman filter can be adjusted to adapt to greater uncertainty, ensuring that the smoothed motion feature vector can accurately capture the real water flow trend and have a positive impact on real-time disaster assessment.

[0047] After smoothing the motion feature vector, the timestamps of the smoothed motion feature vector and the boundary contour feature are compared to ensure that the deviation is less than a preset threshold. The spatial overlap is calculated, and the matching rate between the vector point and the contour point is measured using Euclidean distance. If the consistency score is higher than the preset value, the verification is considered successful; otherwise, a realignment operation is triggered. This ensures the consistency between radar and video data in the spatiotemporal dimension, which can effectively prevent misjudgment of disasters caused by data misalignment, especially in the case of rapidly changing flash floods. For example, during the flood peak, when the motion feature vector shows the peak water flow velocity, the boundary contour feature should simultaneously show the expansion of the water flow boundary. The accuracy of the data can be ensured through verification. For complex terrain such as canyon flash floods, a time window function can be introduced to adjust the verification threshold and improve the adaptability of the verification to meet the monitoring needs in different environments.

[0048] The above technical solution generates a noise-removed feature set containing both motion and contour information based on smoothed motion feature vectors and boundary contour features. The motion feature vectors and boundary contour features are then fused to ensure that motion information, such as velocity direction, and contour information, such as boundary shape, are complementary, forming a complete, noise-removed feature set. This provides reliable data support for subsequent spatial mapping transformation and disaster prediction. The denoised feature set is then stored to ensure the accuracy, robustness, and real-time performance of data during flash flood disaster monitoring, thereby enabling efficient and reliable disaster early warning in complex environments.

[0049] Step S3: Correct the timestamp offset of the motion feature vector and the boundary contour feature to obtain a time-unified feature set; perform spatial mapping transformation on the feature set, associate the motion feature vector with the boundary contour feature to obtain a cross-modal associated fusion feature map;

[0050] In step S3, the timestamp offset of the motion feature vector and the boundary contour feature is corrected to obtain a time-uniform feature set, including:

[0051] The deviation of the noise-removed feature set on the time axis is detected; if the deviation is greater than a preset threshold, the timestamp of the motion feature vector is corrected using a linear interpolation method; if the deviation is greater than the preset threshold, the timestamp of the boundary contour feature is corrected using a nonlinear interpolation method; an intermediate feature set with timestamp alignment is generated based on the corrected motion feature vector and the boundary contour feature; the timestamp consistency of the intermediate feature set is verified; based on the verification result, the timestamp of the intermediate feature set is adjusted to obtain a time-uniform feature set.

[0052] Specifically, in the embodiments of the present invention, due to the difference in sampling frequencies between the radar and video equipment, i.e., the communication delay, the aforementioned feature set will deviate on the time axis. Therefore, the deviation of the feature set after noise removal on the time axis is detected by comparing the timestamp sequences of the motion feature vector and the boundary contour feature, and calculating their average offset on the time axis. When the offset is greater than a preset threshold, since the sampling frequency of the radar is usually smaller than that of the video equipment, the timestamps of the motion feature vector are corrected by linear interpolation. That is, by calculating the linear function value between adjacent timestamps, missing or offset time points are inserted to ensure the accuracy of time alignment. Among them, for the water flow velocity signal in the motion feature vector, if its timestamp deviation exceeds the set threshold, the interpolation point is calculated and the corrected timestamp is inserted to ensure the continuity of the motion feature vector and eliminate the time deviation. For each pair of offsets... The time stamp is calculated, linear coefficients are used, and new timestamps are inserted based on these linear coefficients to update the time series of the motion feature vector. If the deviation is still greater than a preset threshold, a nonlinear interpolation method is used to correct the timestamps of the boundary contour features. The nonlinear interpolation method fits the boundary contour changes in the image sequence through spline functions or polynomial functions to ensure smooth transition of features. For example, cubic spline interpolation is used to insert missing timestamps by minimizing curvature. Specifically, the nonlinear change points of the boundary contour features are first extracted, then the parameters of the spline function are calculated, and the spline function is applied to the timestamp region with large deviation to generate the corrected interpolated timestamps. The continuity of the interpolated contour features is also verified by iteratively adjusting the parameters until the error is less than a preset standard, such as 0.1 seconds. This effectively handles the nonlinear offset caused by complex terrain such as steep terrain or rock obstruction in flash floods, and can more accurately align the water flow boundaries in video images.

[0053] After timestamp correction, the corrected motion feature vectors and boundary contour features are merged into a unified, timestamp-aligned intermediate feature set. First, points with the same timestamps in the motion feature vectors and boundary contour features are matched and grouped into an intermediate set. The timestamp consistency of this intermediate feature set is verified by checking if the synchronization error between all timestamps is less than a preset threshold, such as 0.05 seconds. If the synchronization error meets the requirement, the intermediate feature set is considered successfully aligned. If the synchronization error exceeds the threshold, further adjustments are made; for the remaining offset, fine-tuning interpolation, such as quadratic linear interpolation, is applied to ensure the accuracy of the timestamp-aligned data. The final output is a time-accurate and unified feature set. This technical solution ensures the temporal consistency of motion feature vectors and boundary contour features, providing high-quality data support for subsequent spatial mapping transformations and disaster risk analysis. It also overcomes the challenge of time synchronization between different devices and sensors, especially in complex environments such as the rainy season or mountainous rapids, effectively improving data accuracy and stability, and providing a reliable foundation for subsequent data processing and disaster early warning.

[0054] Further, in step S3, the fused feature map of cross-modal association is obtained, including:

[0055] Motion feature vectors and boundary contour features are extracted from the time-accurate and unified feature set; the similarity between the motion feature vectors and the boundary contour features is calculated using a feature matching algorithm; a feature association matrix is ​​generated based on the similarity; the correspondence between the motion feature vectors and the boundary contour features is determined based on the feature association matrix; a fused feature map containing cross-modal features is generated based on the correspondence; the association accuracy of the fused feature map is verified to obtain a cross-modal associated fused feature map.

[0056] Specifically, in this embodiment, the process of extracting motion feature vectors and boundary contour features from a time-precisely unified feature set to generate a cross-modal associated fusion feature map includes extracting motion feature vectors and boundary contour features from the time-precisely unified feature set. The motion feature vectors represent the speed and direction of the water flow, while the boundary contour features represent the water flow boundary using edge pixel coordinates and shape descriptors extracted from the video image, thus ensuring the integrity of the basic data required for subsequent matching. The similarity between the motion feature vectors and boundary contour features is calculated using a feature matching algorithm. To accurately match the dynamic changes in water flow speed and boundary contour, the distance between each pair of feature points is calculated to quantify the difference between the motion feature vectors and boundary contour features. For each motion feature vector, its Euclidean distance to the corresponding vectors of all boundary contour features is calculated to obtain a similarity score. A lower similarity score indicates a higher matching degree. If the similarity score is below a preset threshold, it is marked as a potential matching pair. By adjusting the weighting factor, the matching accuracy can be optimized according to scene changes with different water flow speeds, especially in dynamically changing scenarios such as flash floods or torrential rain, enabling real-time capture of the relationship between water flow and the boundary.

[0057] Based on the aforementioned similarity, a feature association matrix is ​​generated, where each row corresponds to a motion feature vector and each column corresponds to a boundary contour feature. Each element of the matrix is ​​filled with the corresponding similarity score. This matrix format facilitates subsequent analysis and processing. Based on the feature association matrix, the correspondence between motion feature vectors and boundary contour features is determined. To this end, the boundary contour feature with the lowest similarity between each motion feature vector and its corresponding feature is selected as its correspondence, ensuring that each motion feature vector matches a unique boundary contour feature, thereby achieving a precise one-to-one correspondence.

[0058] Based on the above correspondence, a fused feature map containing cross-modal features is generated. By applying the correspondence to spatial mapping, each motion feature vector is superimposed on the pixel position of the boundary contour feature to form the fused feature map. Each pixel contains water flow velocity and water flow boundary information. In addition, to ensure the smooth transition and feature continuity of the fused map, a gridding method is used to smooth the edges of the fused map to eliminate errors caused by boundary irregularities or resolution changes. To ensure that the generated fused feature map has high correlation accuracy, the map needs to be verified. By calculating the average similarity of all feature pairs in the fused feature map, if the average similarity is higher than the preset accuracy threshold, the fused map is considered to have passed verification. If the similarity is lower than the preset value, iterative adjustment of the correspondence is required to further improve the accuracy. To remove potential residual noise, Kalman filtering is used to smooth the verified map. During the filtering process, the state is corrected through prediction and update steps to ensure the accuracy of the fused map. In complex environments, such as video images under nighttime flash floods or low light conditions, the threshold is adjusted to adapt to specific scenarios, thereby ensuring the accuracy of the fused feature map and supporting accurate assessment of real-time flash flood disaster status.

[0059] The above technical solution, through the generation of cross-modal correlation fusion feature maps, can provide important basis for subsequent disaster early warning and analysis, ensuring the accuracy and robustness of data in the disaster monitoring process. By combining information from radar and video images, it achieves accurate correspondence between motion features and boundary features, optimizes the performance of the flash flood disaster monitoring system, and improves the response speed and accuracy of disaster early warning.

[0060] Step S4: Calculate the correspondence between water flow velocity and boundary contour using the fused feature map to obtain a dynamic hydrological parameter distribution map; mark the high-risk disaster area based on the dynamic hydrological parameter distribution map to obtain a marked disaster area map;

[0061] In step S4, the dynamic hydrological parameter distribution map is obtained, including:

[0062] Feature pairs of water flow velocity and boundary contour are extracted from the fused feature map; a convolutional neural network is used to perform pixel-level correlation processing on the feature pairs to generate a correlation feature matrix; the spatiotemporal correspondence between water flow velocity and boundary contour is calculated based on the correlation feature matrix; an intermediate parameter map containing the water flow velocity distribution is generated based on the spatiotemporal correspondence; a smoothing algorithm is used to optimize the distribution continuity of the intermediate parameter map; and a dynamic hydrological parameter distribution map is generated based on the optimized intermediate parameter map.

[0063] Specifically, in this embodiment, feature pairs of water flow velocity and boundary contour are extracted from the fused feature map, and pixel-level correlation processing is performed on these feature pairs to ultimately obtain a dynamic hydrological parameter distribution map. First, motion feature vectors and boundary contour features are extracted from the fused feature map. The motion feature vectors represent the velocity information of the water flow, while the boundary contour features reflect the boundary shape of the water flow. To ensure subsequent correlation processing, the water flow velocity vector from the radar signal portion and the boundary contour point set from the video image portion are paired to form feature pairs. By analyzing each pixel position, the corresponding water flow velocity value and boundary contour descriptor are identified, thereby forming an initial set of feature pairs.

[0064] The feature pairs are processed at the pixel level using a convolutional neural network (CNN) to generate an associated feature matrix. A CNN is a deep learning model that extracts image features through multiple convolutional and pooling operations. Here, it is used to calculate the matching degree between the velocity and contour of each pixel. The feature pairs are input into the first layer of the CNN, and local association patterns are extracted through convolutional operations. The convolutional operations calculate a weighted sum using a sliding window, thereby capturing the spatial relationship between velocity information and contour features. The convolutional results are then processed with an activation function such as ReLU to introduce non-linearity and enhance the model's expressive power. After multiple convolutions, an associated feature matrix is ​​generated, where each element represents the association strength between a feature pair, with values ​​ranging from 0 to 1. The kernel size of the CNN is adjusted for different water flow intensities. For example, a 3x3 convolutional kernel is used for low-speed water flow, while a 5x5 convolutional kernel is used in high-speed turbulent flow scenarios to adapt to different water flow velocities, thereby enhancing the model's robustness, reducing noise interference caused by differences in water flow rates, and improving the accuracy and stability of the processing. Based on the obtained associated feature matrix, the spatiotemporal correspondence between water flow velocity and boundary contour is calculated. The spatiotemporal correspondence refers to the mapping relationship between velocity vectors and contour features in time and space. Specifically, temporal aggregation is first performed to calculate the corresponding vector between velocity and contour at each timestamp. The least squares method is then applied to fit these corresponding vectors. The least squares method is an optimization method that solves for the best fitting parameters by minimizing the sum of squared errors, thereby quantifying the dynamic mapping between velocity and contour, generating a spatiotemporal correspondence map, ensuring the spatiotemporal consistency of velocity information and boundary contour, and thus improving the accuracy of disaster early warning.

[0065] By mapping the spatiotemporal correspondence onto a grid structure, an intermediate parameter map containing the water flow velocity distribution is generated. This intermediate parameter map represents the spatial distribution of velocity in pixel form and provides the basis for subsequent hydrological parameter calculations. To ensure the continuity and accuracy of the intermediate parameter map, a smoothing algorithm is used to optimize it. Specifically, a Gaussian smoothing algorithm is used to perform weighted averaging on discontinuous regions of the velocity distribution, thereby reducing abrupt changes and ensuring a smooth transition in the velocity distribution. The Gaussian smoothing algorithm smooths each pixel by calculating the weighted average of neighboring pixels, iterating until the continuity index of the image reaches a preset standard. Finally, based on the optimized intermediate parameter map, a dynamic hydrological parameter distribution map is generated. This dynamic distribution map is integrated with other hydrological data, such as water level information, to provide accurate parameter support for the real-time assessment of flash flood disaster status. The above technical solution ensures the accurate generation from the fused feature map to the final hydrological parameter distribution map, enabling real-time and efficient identification of dynamic changes in water flow and the expansion of water flow boundaries in complex flash flood disaster monitoring, greatly improving the response speed and accuracy of disaster early warning.

[0066] Further, in step S4, the high-risk disaster area is marked according to the dynamic hydrological parameter distribution map to obtain the marked disaster area map, including:

[0067] Water flow velocity values ​​are extracted from the dynamic hydrological parameter distribution map; a threshold judgment mechanism is used to determine areas exceeding the normal range based on the water flow velocity values; a high-risk area mask is generated based on the areas exceeding the normal range; the high-risk area mask is applied to the dynamic hydrological parameter distribution map to generate an intermediate distribution map containing high-risk markers; the continuity of high-risk areas in the intermediate distribution map is verified; based on the verification results, the intermediate distribution map is adjusted to obtain a marked disaster range map.

[0068] Specifically, the process of marking high-risk disaster areas based on the aforementioned dynamic hydrological parameter distribution map to obtain the marked disaster area map includes: firstly, extracting water flow velocity values ​​from the dynamic hydrological parameter distribution map; secondly, identifying the water flow velocity data corresponding to each pixel by traversing each pixel in the distribution map and storing it as a velocity value array, which retains the spatial location information of each data point; and thirdly, using a preset threshold judgment mechanism to determine which areas have water flow velocities exceeding the normal range for the aforementioned water flow velocity values; for example, assuming the normal water flow velocity range is X1 to X2 meters per second, if the water flow velocity in some areas reaches or exceeds X2 meters per second, these areas are considered abnormal areas. This judgment mechanism helps to quickly identify potential high-risk disaster areas. Based on the areas exceeding the normal range, a high-risk area mask is generated. First, connected component analysis is performed on the areas exceeding the normal range to calculate the area and boundary of each connected region, generating a binary mask. A mask value of 1 indicates that the area is a high-risk point, and a value of 0 indicates that the area is a normal area. To further ensure the capture of potential risks of water flow spread, based on the terrain data of the flash flood area, a morphological dilation operation is performed on the generated mask to expand the boundary to cover possible water flow spread areas. For example, the dilation kernel size is set to NxN pixels, where N is a positive integer greater than 2 and less than or equal to 5, which can effectively handle local water flow changes and ensure the capture of areas that may lead to the spread of disasters. Especially in complex terrains such as mountain rapids and steep terrain, after processing pixel-level correlation through convolution operations, the mask for areas exceeding the normal range is further processed. For example, in scenarios such as the rainy season or heavy rain, if the area of ​​the abnormal region is large, the expansion radius of the dilated mask will increase to better reflect the risk of water flow acceleration, thereby improving the accuracy of high-risk area marking and reducing missed detections.

[0069] The aforementioned high-risk area mask is applied to the dynamic hydrological parameter distribution map to generate an intermediate distribution map containing high-risk markers. The mask and the original distribution map are merged through a pixel-by-pixel overlay operation. Areas exceeding the normal range are marked as high-risk areas, typically highlighted in red for visualization and monitoring, thus clearly identifying high-risk areas requiring special attention during disaster monitoring. The continuity of high-risk areas in the generated intermediate distribution map is verified by applying a contour tracing algorithm to analyze the high-risk marked areas, calculating the boundary length and internal fill rate of each area, and determining whether its continuity score exceeds a set threshold based on the calculation results. The contour tracing algorithm is based on the Moore neighborhood boundary following method for detection. The system checks whether the measured area is a continuous closed shape. If gaps are found between areas, and these gaps are smaller than a set number of pixels, the areas can be merged to ensure the continuity of the disaster area. If the overlap rate between areas is lower than a preset standard, interpolation is used to fill adjacent areas to connect the breakpoints, further optimizing the stability and accuracy of real-time monitoring of flash floods. Based on the verification results, the intermediate distribution map is adjusted to obtain the final marked disaster area map. If the verification shows discontinuous areas, these areas are filled using interpolation methods to generate smooth boundaries, ensuring the integrity of the disaster area map. This ensures that all high-risk areas are uniformly marked, forming a comprehensive and accurate disaster area map suitable for subsequent flash flood disaster assessment and emergency response.

[0070] The above-mentioned technical solution can accurately identify and mark high-risk areas in flash flood disasters in complex environments and dynamically changing flash flood scenarios, generate high-quality disaster range maps, and help effectively improve the accuracy and response speed of disaster early warning.

[0071] Step S5: Generate a comprehensive monitoring report based on the marked disaster area map to determine the real-time status of the flash flood disaster; specifically including:

[0072] Feature data of high-risk disaster areas are extracted from the marked disaster area map; the feature data is integrated with the associated features in the fused feature map to generate a comprehensive feature set; the comprehensive feature set is processed using a data aggregation algorithm to generate intermediate monitoring data; a comprehensive monitoring report containing disaster area and hydrological parameters is generated based on the intermediate monitoring data; the completeness of the comprehensive monitoring report is verified; the comprehensive monitoring report is adjusted based on the verification results to determine the real-time status of flash flood disasters.

[0073] Specifically, the process of generating a comprehensive monitoring report based on the marked disaster range map and determining the real-time status of flash flood disasters includes: firstly, extracting feature data of high-risk disaster ranges from the marked disaster range map, i.e., identifying areas where water flow velocity exceeds the normal range by scanning the disaster range map at the pixel level, and extracting the boundary coordinates of these areas and the statistical values ​​of their water flow velocity as feature data; the above feature data provides an important basis for subsequent disaster assessment; then, integrating the above feature data with the associated features in the fused feature map to generate a comprehensive feature set, specifically matching the extracted feature data with the motion feature vectors and boundary contour features already associated in the fused feature map through feature matching algorithms, ensuring the continuity of the spatiotemporally aligned multi-source dataset, and forming a multi-dimensional comprehensive feature set through the above integration, enabling features from different data sources to work synergistically and avoiding the emergence of information silos.

[0074] The comprehensive feature set is processed by data clustering to generate intermediate monitoring data. Specifically, the comprehensive feature set is grouped using the k-means clustering algorithm, and similar features are aggregated by calculating the Euclidean distance from data points to cluster centers to generate a disaster intensity index. The k-means clustering algorithm process includes initializing cluster centers, assigning data points to the nearest cluster center, updating cluster centers until convergence. In flash flood monitoring, the above clustering method can effectively reduce data redundancy and ensure more accurate assessment of disaster intensity and identification of high-risk areas. For example, if the comprehensive feature set contains feature vectors of multiple water flow velocities and boundary expansion values, clustering can yield aggregated results reflecting different disaster scenarios, further improving monitoring efficiency.

[0075] Based on the aforementioned intermediate monitoring data, a comprehensive monitoring report is generated, including the disaster area and hydrological parameters. This report combines coordinate descriptions of high-risk areas with hydrological parameters such as average flow velocity by compiling aggregated indicators from the intermediate monitoring data to facilitate disaster assessment. It also verifies the completeness of the comprehensive monitoring report, ensuring that the disaster area and hydrological parameters described in the report cover all high-risk areas and that these data are consistent with the data in the fused feature map. If any missing or inconsistent data is found, the corresponding aggregated values ​​from the intermediate monitoring data are supplemented to ensure the accuracy and completeness of the report. The comprehensive monitoring report is adjusted based on the verification results to determine the real-time status of the flash flood disaster. If the verification reveals any missing or erroneous data in the report, the corresponding parameters are adjusted to ultimately output a report confirming the real-time status of the flash flood disaster. For example, if the report omits boundary extension values ​​for certain areas, adjustments can improve the report's accuracy. The final comprehensive monitoring report ensures that it provides timely and accurate data support for disaster response and provides decision-making basis for emergency response personnel.

[0076] The above technical solution, by extracting feature data from the marked disaster range map, integrating and fusing data from the feature map, applying clustering algorithms to generate monitoring data, verifying the integrity of the report, and adjusting the report based on the verification results, can accurately assess the real-time status of flash flood disasters, provide high-quality disaster early warning information, and ensure the response speed and accuracy of the disaster early warning system.

[0077] This invention also provides a real-time monitoring system for flash floods that integrates radar and video, used to implement the above-mentioned methods, such as... Figure 2 As shown, the system includes:

[0078] The data alignment unit is used to acquire water flow velocity signals and image sequences of flash flood areas through radar equipment and video equipment, respectively, as dynamic data streams of flash flood areas. The acquisition frequency differences of the dynamic data streams are aligned through a timestamp calibration mechanism to obtain a multi-source dataset with preliminary spatiotemporal alignment.

[0079] The feature extraction unit is used to extract motion feature vectors and boundary contour features from the multi-source dataset, and to perform filtering and smoothing processing on the motion feature vectors to obtain a feature set after noise removal.

[0080] The correction mapping unit is used to correct the timestamp offset between the motion feature vector and the boundary contour feature to obtain a time-unified feature set; the feature set is then spatially mapped and transformed to associate the motion feature vector with the boundary contour feature to obtain a cross-modal associated fusion feature map;

[0081] The distribution map marking unit is used to calculate the correspondence between water flow velocity and boundary contour through the fused feature map to obtain a dynamic hydrological parameter distribution map; and to mark the high-risk disaster area according to the dynamic hydrological parameter distribution map to obtain a marked disaster area map.

[0082] The report generation unit is used to generate a comprehensive monitoring report based on the marked disaster range map to determine the real-time status of flash flood disasters.

[0083] The present invention also provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the above-described method.

[0084] In summary, this invention solves the data inconsistency problem caused by differences in acquisition frequency by synchronously acquiring data from radar and video equipment and using a timestamp calibration mechanism to perform preliminary alignment of spatiotemporal data. Motion feature vectors and boundary contour features are extracted from the aligned dataset, and noise is removed using a filtering algorithm, optimizing data smoothing and eliminating errors caused by environmental interference and other factors, providing reliable basic data for subsequent spatiotemporal analysis. Based on the corrected feature dataset, motion features and boundary contour features are associated through spatial mapping transformation, and cross-modal fusion is achieved through feature matching algorithms, ensuring accurate alignment of radar data and video images. Further calculation of the spatiotemporal correspondence between water flow velocity and boundary contour generates a dynamic hydrological parameter distribution map, enabling accurate identification of high-risk areas for flash floods. After marking high-risk areas, a comprehensive monitoring report is generated, determining the status of flash floods in real time and providing timely decision-making basis for disaster early warning and prevention. Through the synergy of the above technical solutions, the advantages of multiple data sources are fully utilized, improving the accuracy and efficiency of flash flood monitoring.

[0085] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. A method for real-time monitoring of flash floods integrating radar and video, characterized in that, include: Step S1: The water flow velocity signal and the image sequence of the flash flood area are acquired by radar equipment and video equipment respectively as the dynamic data stream of the flash flood area. The acquisition frequency difference of the dynamic data stream is aligned by the timestamp calibration mechanism to obtain a multi-source dataset with preliminary spatiotemporal alignment. Step S2: Extract motion feature vectors and boundary contour features from the multi-source dataset, and perform filtering and smoothing processing on the motion feature vectors to obtain a feature set after noise removal; Step S3: Correct the timestamp offset of the motion feature vector and the boundary contour feature to obtain a time-unified feature set; perform spatial mapping transformation on the feature set, associate the motion feature vector with the boundary contour feature to obtain a cross-modal associated fusion feature map; Step S4: Calculate the correspondence between water flow velocity and boundary contour using the fused feature map to obtain a dynamic hydrological parameter distribution map; mark the high-risk disaster area based on the dynamic hydrological parameter distribution map to obtain a marked disaster area map; Step S5: Generate a comprehensive monitoring report based on the marked disaster range map to determine the real-time status of the flash flood disaster.

2. The method as described in claim 1, characterized in that, In step S1, a multi-source dataset with preliminary spatiotemporal alignment is obtained, including: Water flow velocity signals are collected using radar equipment to generate a radar data stream containing timestamps; image sequences of flash flood areas are collected using video equipment to generate a video data stream containing timestamps; based on the timestamps of the radar data stream and the video data stream, a synchronization algorithm is used to align their acquisition frequencies; if the timestamp deviation between the radar data stream and the video data stream exceeds a preset threshold, the data alignment is adjusted using a timestamp interpolation method; based on the aligned radar data stream and video data stream, a multi-source dataset containing water flow velocity signals and image sequences with preliminary spatiotemporal alignment is generated; the timestamp consistency of the multi-source dataset is verified to obtain the final multi-source dataset with preliminary spatiotemporal alignment.

3. The method as described in claim 1, characterized in that, Step S2 includes: The water flow velocity signal in the radar data stream is extracted from the multi-source dataset to generate a motion feature vector; image sequences in the video data stream are extracted from the multi-source dataset, and boundary contour features are generated using an edge detection algorithm; noise interference is smoothed using a Kalman filtering algorithm on the motion feature vector to generate a smoothed motion feature vector; the spatiotemporal consistency between the smoothed motion feature vector and the boundary contour features is verified; a noise-removed feature set containing motion and contour information is generated based on the smoothed motion feature vector and the boundary contour features; the noise-removed feature set is stored for subsequent processing.

4. The method as described in claim 1, characterized in that, In step S3, the timestamp offset of the motion feature vector and the boundary contour feature is corrected to obtain a time-uniform feature set, including: The deviation of the noise-removed feature set on the time axis is detected; if the deviation is greater than a preset threshold, the timestamp of the motion feature vector is corrected using a linear interpolation method; if the deviation is greater than the preset threshold, the timestamp of the boundary contour feature is corrected using a nonlinear interpolation method; an intermediate feature set with timestamp alignment is generated based on the corrected motion feature vector and the boundary contour feature; the timestamp consistency of the intermediate feature set is verified; based on the verification result, the timestamp of the intermediate feature set is adjusted to obtain a time-accurate and unified feature set.

5. The method as described in claim 4, characterized in that, In step S3, the fused feature map of cross-modal association is obtained, including: Obtain radar coordinate system data from the time-unified feature set; project the radar coordinate system data onto the video pixel coordinate system to generate a spatially mapped feature set; for the spatially mapped feature set, use a feature matching algorithm to associate the motion feature vector with the boundary contour feature; based on the association result, generate cross-modal feature pairs containing motion features and contour features; generate a fused feature map based on the cross-modal feature pairs; verify the spatial consistency of the fused feature map to obtain the cross-modal associated fused feature map.

6. The method as described in claim 1, characterized in that, In step S4, a dynamic hydrological parameter distribution map is obtained, including: Feature pairs of water flow velocity and boundary contour are extracted from the fused feature map; a convolutional neural network is used to perform pixel-level correlation processing on the feature pairs to generate a correlation feature matrix; the spatiotemporal correspondence between water flow velocity and boundary contour is calculated based on the correlation feature matrix; an intermediate parameter map containing the water flow velocity distribution is generated based on the spatiotemporal correspondence; a smoothing algorithm is used to optimize the distribution continuity of the intermediate parameter map; and a dynamic hydrological parameter distribution map is generated based on the optimized intermediate parameter map.

7. The method as described in claim 6, characterized in that, In step S4, the high-risk disaster area is marked according to the dynamic hydrological parameter distribution map to obtain the marked disaster area map, including: Water flow velocity values ​​are extracted from the dynamic hydrological parameter distribution map; a threshold judgment mechanism is used to determine areas exceeding the normal range based on the water flow velocity values; a high-risk area mask is generated based on the areas exceeding the normal range; the high-risk area mask is applied to the dynamic hydrological parameter distribution map to generate an intermediate distribution map containing high-risk markers; the continuity of high-risk areas in the intermediate distribution map is verified; based on the verification results, the intermediate distribution map is adjusted to obtain a marked disaster range map.

8. The method as described in claim 1, characterized in that, In step S5, the real-time status of the flash flood disaster is determined, including: Feature data of high-risk disaster areas are extracted from the marked disaster area map; the feature data is integrated with the associated features in the fused feature map to generate a comprehensive feature set; the comprehensive feature set is processed using a data aggregation algorithm to generate intermediate monitoring data; a comprehensive monitoring report containing disaster areas and hydrological parameters is generated based on the intermediate monitoring data; the completeness of the comprehensive monitoring report is verified; based on the verification results, the comprehensive monitoring report is adjusted to determine the real-time status of flash flood disasters.

9. A real-time monitoring system for flash floods integrating radar and video, used to implement the method as described in any one of claims 1-8, characterized in that, The system includes: The data alignment unit is used to acquire water flow velocity signals and image sequences of flash flood areas through radar equipment and video equipment, respectively, as dynamic data streams of flash flood areas. The acquisition frequency differences of the dynamic data streams are aligned through a timestamp calibration mechanism to obtain a multi-source dataset with preliminary spatiotemporal alignment. The feature extraction unit is used to extract motion feature vectors and boundary contour features from the multi-source dataset, and to perform filtering and smoothing processing on the motion feature vectors to obtain a feature set after noise removal. A correction mapping unit is used to correct the timestamp offset between the motion feature vector and the boundary contour feature to obtain a time-unified feature set; the feature set is then spatially mapped and transformed to associate the motion feature vector with the boundary contour feature to obtain a cross-modal associated fusion feature map; The distribution map marking unit is used to calculate the correspondence between water flow velocity and boundary contour through the fused feature map to obtain a dynamic hydrological parameter distribution map; and to mark the high-risk disaster area according to the dynamic hydrological parameter distribution map to obtain a marked disaster area map. The report generation unit is used to generate a comprehensive monitoring report based on the marked disaster range map to determine the real-time status of flash flood disasters.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the method as described in any one of claims 1-9.

Images