A method for real-time monitoring of urban flood water depth based on intelligent 3D vision

Abstract

This invention belongs to the interdisciplinary field of water conservancy and computer vision, proposing a real-time monitoring method for urban flood water depth based on intelligent 3D vision. The method first uses drones to collect multi-view images of urban scenes and performs refined 3D reconstruction. Then, it uses a camera to capture 2D images of the scenes and matches them with the drone images, selecting the most similar images. Next, it performs dense feature comparison between the 2D images and 3D information, calculating the camera pose based on the correspondence of feature points. Finally, through SAM2 segmentation and autoregressive optimization, it extracts time-stable water surface areas and, combined with the known camera pose, backprojects the water surface edges into 3D space to obtain the water depth around the scenes, achieving real-time monitoring of urban flood water depth data. This method provides important technical support for refined real-time early warning of urban flooding and has practical engineering application value.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of water conservancy and computer vision, and in particular relates to a method for real-time monitoring of urban flood water depth based on intelligent three-dimensional vision. Background Technology

[0002] Floods severely threaten the lives and property of residents and hinder sustainable urban development, becoming an urgent urban governance challenge. The core of flood prevention and control lies in improving disaster response and early warning capabilities, and building a refined forecasting and early warning system that includes "pre-event prediction, pre-disaster warning, and short-term early warning" to provide scientific and technological support for urban disaster prevention and mitigation.

[0003] Currently, the key to accurate flood early warning lies in acquiring high spatiotemporal resolution hydrodynamic data, especially water depth data. Existing research typically uses hydrological and hydrodynamic models to simulate flood evolution. Hydrological models primarily describe rainfall, infiltration, and runoff processes; hydrodynamic models simulate the flow and changes of surface water, commonly including 1D, 2D, and coupled models. In practical applications, hydrological and hydrodynamic processes are usually integrated to construct a unified simulation system, with typical models including SWMM and MIKE Urban. These models are widely used in urban flood control engineering design, drainage planning, and risk assessment, achieving significant results in both basic theoretical research and engineering practice.

[0004] However, due to the inherent complexity of urban flooding processes, existing models still face challenges in practical applications, including long construction cycles, insufficient adaptability, and high computational costs. On one hand, these models rely on a large number of parameter inputs, resulting in high update and maintenance costs and difficulty adapting to dynamic changes in urban surfaces and drainage systems. On the other hand, high-precision simulations typically require meter-level spatial grids, leading to enormous computational resource consumption and failing to meet the real-time early warning requirements with minute-level responses. Furthermore, input errors such as rainfall intensity and surface conditions tend to accumulate during simulations, reducing the reliability and practicality of the model in complex scenarios. Therefore, limited by these factors, traditional modeling methods struggle to meet the demands for dynamic perception and rapid inversion of flood evolution processes.

[0005] In recent years, computer vision methods have been gradually introduced into urban flood monitoring, such as rainfall intensity recognition and waterlogging level recognition, demonstrating good application potential. Some studies utilize video images captured by cameras to extract water surface boundaries, offering advantages such as flexible deployment and rapid response. However, existing visual methods mainly estimate hydrodynamic parameters of local areas under fixed environments (such as fixed camera poses) or use remote sensing images to invert large-scale, low-resolution flood processes, thus making it difficult to provide high spatiotemporal resolution hydrodynamic data for urban flood processes. Summary of the Invention

[0006] To effectively address the shortcomings of existing technologies in terms of spatiotemporal resolution, monitoring accuracy, and computational efficiency, this invention proposes a real-time monitoring method for urban flood hydrodynamics based on intelligent 3D vision. This method utilizes widely distributed cameras throughout the city, combined with computer vision technology, to directly acquire information on the depth of urban floodwaters, thereby achieving high spatiotemporal resolution hydrodynamic data acquisition at low cost.

[0007] According to a first aspect of the embodiments of this application, a method for real-time monitoring of urban flood water depth based on intelligent three-dimensional vision is provided, including:

[0008] S1: Using multi-angle UAV images, calculate the camera intrinsic and extrinsic parameters corresponding to the images, construct an initial 3D model of the target area, and use feature trajectory and geometric structure iterative refinement strategy to obtain the 3D model of the target area;

[0009] S2: Acquire two-dimensional images captured by cameras within the target area and match them with the drone images to obtain several camera-drone image pairs with the closest captured images;

[0010] S3: Perform dense feature matching on the camera-drone image pair to estimate the spatial pose of the camera;

[0011] S4: Acquire the video of the urban flood scene captured by the camera, extract the pixel position of the water surface edge, combine the camera spatial pose with the three-dimensional model of the target area, calculate its spatial position through back projection, and thus invert the water depth of the area.

[0012] Further, step S1 includes:

[0013] S11: Utilizing multi-angle UAV images, a feature matching method with a fusion grid quantization strategy is used to generate the correspondence between feature points in the multi-view images and construct the initial feature trajectory; through motion structure recovery, camera parameters are calculated and 3D point coordinates are reconstructed to generate the initial 3D model of the target area and establish the mapping relationship between 3D points and image pixels.

[0014] S12: Based on the initial feature trajectory and the initial three-dimensional model of the target region, the feature trajectory and geometric structure are alternately optimized through an iterative refinement process to reconstruct a high-precision three-dimensional point cloud of the target region;

[0015] Specifically, the feature trajectory and geometric structure are alternately optimized through an iterative refinement process:

[0016] Based on the initial feature trajectory, a multi-view matching method based on the Transformer architecture is used to optimize the trajectory. By constructing the feature correspondence between the reference image and other views, the matching confidence and consistency under each view are calculated, the trajectory point positions are updated, and the optimized multi-view feature trajectory set is obtained.

[0017] Based on the optimized multi-view feature trajectory set, geometric optimization is performed. Bundle adjustment is used to jointly optimize camera extrinsic and intrinsic parameters as well as 3D point coordinates. The optimization objective is to minimize the reprojection error of all 3D points in the image. The error function is defined as follows:

[0018]

[0019] in, This represents the theoretical position of a 3D point projected onto the image plane. This corresponds to the actual observation location in the image. The number of feature points participating in the optimization.

[0020] Further, step S2 includes:

[0021] S21: Extracting local visual features from 2D camera images and drone images using convolutional neural networks. This aggregates local features into a global feature descriptor for the image. ;

[0022] S22: Based on the global feature descriptors of drone images and camera images, match the camera images and drone images to obtain several camera-drone image pairs with the closest captured images.

[0023] Further, step S3 includes:

[0024] S31: Multi-scale feature extraction is performed on the camera-drone image pair using a convolutional neural network to obtain coarse-grained feature maps and high-resolution feature maps, and feature enhancement is performed using self-attention mechanism and cross-attention mechanism;

[0025] S32: Based on the enhanced features, a two-stage feature matching method is used to match image pairs to obtain dense feature correspondences;

[0026] S33: Based on the dense feature correspondence, establish the projection equation of a three-dimensional spatial point onto a two-dimensional plane to obtain the camera pose;

[0027] The projection equation of the three-dimensional point onto the two-dimensional plane is as follows:

[0028]

[0029] in, This represents the homogeneous coordinates of a point in three-dimensional space in the world coordinate system. This represents the homogeneous coordinates of the point's projection position on the image plane. This is the scaling factor, corresponding to the depth value of the point in the camera coordinate system. This is the intrinsic parameter matrix of the camera. and These are its rotation matrix and translation vector, respectively.

[0030] Further, in step S32, the two-stage feature matching method includes:

[0031] (1) Coarse-grained matching stage

[0032] Using the coarse-grained feature map as input, similarity is calculated based on feature vectors to construct a score matrix of matching confidence between images. An nearest neighbor matching strategy is adopted to select initial matching point pairs from the score matrix.

[0033] (2) Refined matching stage

[0034] Based on the initial matching point pairs, the high-resolution feature map is used to calculate the dot product similarity of feature vectors in the local region of each pair of matching points, construct a local score map, and use the nearest neighbor strategy to filter and obtain pixel-level matching point pairs.

[0035] Based on the pixel-level matching point pairs, a local region of a predetermined size is constructed around each pair of matching points. The feature similarity score at each position is calculated using the feature vector dot product method, and the matching probability distribution is normalized. The weighted average coordinates are calculated based on the matching probability distribution to obtain dense feature pairs.

[0036] Further, step S4 includes:

[0037] S41: Based on the SAM2 image segmentation model, the water surface region is extracted from the input video image sequence to obtain the initial water surface mask in each frame image. The initial water surface mask is then optimized across frames using an autoregressive model to obtain the water surface region segmentation result.

[0038] S42: Using the camera parameters obtained during the camera pose recovery process, project the water surface edge line based on the water surface region segmentation result onto three-dimensional space, and calculate the coordinates of the water surface edge points in the camera coordinate system:

[0039]

[0040] in, These are the position coordinates of a point at the edge of the water surface in the camera coordinate system.

[0041] S43: Using the rotation matrix obtained in step S3 and translation vector Convert camera coordinates to world coordinates The formula is as follows:

[0042]

[0043] S44: Based on the images acquired before the flood, the world coordinates of the building's base are calculated using the process in steps S41-S43. The submerged water depth of the building is then determined based on the world coordinates of the water surface edge and the building's base.

[0044] According to a second aspect of the embodiments of this application, a real-time urban flood water depth monitoring device based on intelligent three-dimensional vision is provided, comprising:

[0045] The 3D reconstruction module is used to calculate the camera intrinsic and extrinsic parameters corresponding to the images using UAV images from multiple angles, construct an initial 3D model of the target area, and obtain the 3D model of the target area by using feature trajectory and geometric structure iterative refinement strategy.

[0046] The matching module is used to acquire two-dimensional images captured by cameras within the target area and match them with the drone images to obtain several camera-drone image pairs with the closest captured images.

[0047] The pose estimation module is used to perform dense feature matching on the camera-UAV image pairs to estimate the spatial pose of the camera.

[0048] The water depth inversion module is used to acquire urban flood scene videos captured by the camera, extract the pixel positions of the water surface edges, combine the camera's spatial pose with the 3D model of the target area, calculate its spatial position through back projection, and thus invert the water depth of the area.

[0049] According to a third aspect of the embodiments of this application, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the method described in the first aspect.

[0050] According to a fourth aspect of the embodiments of this application, an electronic device is provided, comprising:

[0051] One or more processors;

[0052] Memory, used to store one or more programs;

[0053] When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described in the first aspect.

[0054] According to a fifth aspect of the embodiments of this application, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0055] The technical solutions provided by the embodiments of this application may include the following beneficial effects:

[0056] (1) The present invention proposes a real-time monitoring method for urban flood water depth based on intelligent three-dimensional vision, which uses cameras widely distributed in the city to acquire image data, greatly reducing the cost of data acquisition. This method does not rely on expensive and complex traditional monitoring equipment, and has the ability to carry out efficient monitoring in areas that are difficult to cover by conventional means, thereby achieving efficient acquisition of flood data at low cost.

[0057] (2) The method of the present invention has high spatiotemporal resolution and can capture the dynamic changes of water depth in urban flooding in real time with a response frequency of minutes. Compared with the shortcomings of traditional monitoring methods such as low data update frequency and limited spatial coverage, this method can obtain gridded water depth data in real time, providing support for the refined analysis of urban flood disasters and significantly improving the ability to judge the development trend of floods.

[0058] (3) The method of the present invention does not rely on high-precision terrain data or complex modeling processes. It achieves automatic extraction of water depth in flood scenarios based on computer vision technology, and has good environmental adaptability and automated processing capabilities. The system can adapt to different camera deployment conditions and complex urban environments, supports stable operation in multiple application scenarios, and has good scalability and engineering deployment value.

[0059] In summary, this invention proposes a real-time urban flood water depth monitoring method based on intelligent 3D vision. Utilizing existing cameras in the city, a high-precision 3D scene model is constructed using unmanned aerial vehicles (UAVs). Image feature matching is used to calculate the spatial pose of the cameras, and further, water surface segmentation and geometric inversion methods are combined to obtain the water depth in the flood scene. This method features simple deployment, rapid response, and strong robustness, effectively solving the problems of high cost, low spatiotemporal resolution, strong data dependence, and poor adaptability commonly found in traditional flood water depth monitoring methods. It significantly improves the accuracy and efficiency of urban flood monitoring. This method provides technical support for the scientific management and emergency decision-making of urban flood disasters, and is of great significance to the improvement of urban disaster prevention and mitigation systems, possessing extremely high promotional value and practical engineering application potential. It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit this application. Attached Figure Description

[0060] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0061] Figure 1 This is a flowchart illustrating a real-time monitoring method for urban flood water depth based on intelligent 3D vision, according to an exemplary embodiment.

[0062] Figure 2 This is a schematic diagram of the 3D reconstruction of the scene in the embodiment.

[0063] Figure 3 This is a schematic diagram of feature point matching in the embodiment.

[0064] Figure 4 This is a schematic diagram of camera pose inversion in the embodiment.

[0065] Figure 5 This is a schematic diagram of water surface segmentation and water depth calculation in the embodiment, where (a) is the actual image captured by the camera under test conditions, (b) is the water surface area segmented by this method, (c) is the water depth monitored by this method and the water depth measured by the water level gauge, and (d) is the error curve between the water depth monitored by this method and the water depth measured by the water level gauge.

[0066] Figure 6 This is a block diagram illustrating a real-time urban flood water depth monitoring device based on intelligent 3D vision, according to an exemplary embodiment.

[0067] Figure 7 This is a schematic diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0068] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0069] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0070] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0071] like Figure 1 This invention provides a method for real-time monitoring of urban flood water depth based on intelligent three-dimensional vision. The steps of the monitoring method are as follows:

[0072] S1: Using multi-angle UAV images, calculate the camera intrinsic and extrinsic parameters corresponding to the images, construct an initial 3D model of the target area, and use feature trajectory and geometric structure iterative refinement strategy to obtain the 3D model of the target area;

[0073] Specifically, this step addresses the challenge of extracting features with weak texture and strong reflectivity in urban areas by proposing a robust 3D visual modeling method. Multi-view image acquisition of the target area using drones yields image data containing elements such as buildings, roads, and greenery. An initial model is constructed using dense feature matching and mesh quantization strategies based on the Transformer architecture. An iterative optimization process combining feature trajectory refinement and geometric refinement generates a high-precision 3D point cloud model for subsequent water level monitoring. This step may include the following sub-steps:

[0074] S11: Utilizing multi-angle UAV images, a feature matching method based on a fusion grid quantization strategy is used to generate the correspondence between feature points in the multi-view images and construct the initial feature trajectory. Furthermore, through motion structure recovery, camera parameters are calculated and 3D point coordinates are reconstructed to generate the initial 3D model of the target area and establish the mapping relationship between 3D points and image pixels.

[0075] Specifically, firstly, multi-view images of the monitoring area are acquired, and then a dense feature matching method based on the Transformer architecture is used to match image pairs to extract the initial correspondence between feature points in the images. This matching method does not rely on traditional keypoint detection and has higher robustness to low-texture urban areas (such as flat walls, roads, etc.) and repetitive structures, making it suitable for multi-view 3D reconstruction tasks in urban scenes.

[0076] To improve the consistency of multi-view matching results, a fixed grid quantization strategy is introduced to merge feature points with similar spatial locations into a unified grid node, constructing a coherent initial feature trajectory, thereby effectively reducing the matching error between multiple views.

[0077] Based on this, the SfM (Structure-from-Motion) method is used to calculate the approximate camera intrinsic parameters (including focal length and principal point position) and extrinsic parameters (i.e., the camera rotation matrix in the world coordinate system) for each image. Translation vector ), and through multi-view Figure 3 Angle measurement method to recover the three-dimensional coordinates of feature points in the world coordinate system Simultaneously, the projection relationship between the 3D points and the corresponding pixels in the original image is established, thus completing the construction of the initial 3D model of the target region.

[0078] S12: Based on the initial feature trajectory and the initial three-dimensional model of the target region, the feature trajectory and geometric structure are alternately optimized through an iterative refinement process to reconstruct a high-precision three-dimensional point cloud of the target region;

[0079] First, based on the initial feature trajectories generated in the coarse reconstruction stage, a multi-view matching method based on the Transformer architecture is used to optimize the trajectories. By constructing feature correspondences between the reference image and other views, the matching confidence and consistency under each view are calculated, and the trajectory point positions are updated to obtain the optimized multi-view feature trajectory set.

[0080] Based on this, geometric optimization is performed, employing bundle adjustment to jointly optimize camera extrinsic and intrinsic parameters as well as 3D point coordinates. The optimization objective is to minimize the reprojection error of all 3D points in the image, defined as follows:

[0081] (1-1)

[0082] in, This represents the theoretical position of a 3D point projected onto the image plane. This corresponds to the actual observation location in the image. The number of feature points involved in the optimization is determined. By minimizing this error function, the 3D structure and camera pose parameters are optimized, making the reconstructed model closer to real image observations. Furthermore, to further improve trajectory quality, topology adjustment operations are included, mainly comprising: trajectory merging, missing observation point completion, and outlier removal. These processes can be executed iteratively, ultimately outputting a high-precision 3D point cloud model and camera parameter results, providing an accurate spatial basis for subsequent analyses such as water depth estimation.

[0083] S2: Acquire two-dimensional camera images captured by cameras within the target area and match them with the drone images to obtain several camera-drone image pairs with the closest captured images;

[0084] Specifically, to achieve accurate estimation of the camera's spatial pose, this step first acquires a two-dimensional image and compares it with the UAV image used in the three-dimensional reconstruction in step S1 to find the image that most closely resembles the captured image, which is then used for subsequent camera pose estimation. This step may include the following sub-steps:

[0085] S21: Extracting local visual features from 2D camera images and drone images using convolutional neural networks. This aggregates local features into a global feature descriptor for the image. This is used to measure the overall information of an image. The formula is as follows:

[0086] (2-1)

[0087] in, , These are drone images used for 3D reconstruction and camera images used for pose estimation, respectively. VLAD is a feature aggregation encoding method.

[0088] S22: Based on the global feature descriptors of drone images and camera images, compare the camera images and drone images to obtain several camera-drone image pairs with the closest captured images;

[0089] In one embodiment, the matching method is to calculate the dot product of the global feature descriptors of the drone image and the camera image. :

[0090] (2-2)

[0091] For each camera, for all calculated Sort the values ​​and select the k largest ones. The value corresponds to the camera image and drone image pair, where the value of k can be determined according to the task accuracy requirements;

[0092] S3: Perform dense feature matching on the camera-drone image pair to estimate the spatial pose of the camera;

[0093] S31: Multi-scale feature extraction is performed on the camera-drone image pair using a convolutional neural network to obtain coarse-grained feature maps and high-resolution feature maps, and feature enhancement is performed using self-attention mechanism and cross-attention mechanism;

[0094] A lightweight convolutional neural network is input to the camera-drone image pairs to perform multi-scale feature extraction, resulting in coarse-grained and high-resolution feature maps. The coarse-grained feature map is extracted at 1 / 8 scale of the original image and used in the subsequent coarse matching stage; the high-resolution feature map is extracted at 1 / 2 scale and used in the subsequent fine matching stage. During feature extraction, the neural network encodes each location in the image into a multi-dimensional feature vector to describe the texture, structure, and other semantic information of that location. For scene features such as water reflection and sparse texture, self-attention and cross-attention mechanisms are introduced to fuse contextual information between images, generating feature representations with enhanced robustness.

[0095] S32: Based on the enhanced features, a two-stage feature matching method is used to match image pairs to obtain dense feature correspondences;

[0096] Based on the enhanced feature representation, coarse-grained feature matching is performed on camera image and drone image pairs to calculate similarity score matrices, and preliminary screening is conducted based on nearest neighbor matching and confidence thresholds. Subsequently, local cropping and two-stage fine matching are performed on candidate regions, and spatial expectation coordinates and similarity constraints are used to improve matching accuracy, ultimately obtaining high-quality, dense feature pairs at the sub-pixel level. ,in , These represent the spatial position of a point in the world coordinate system and its projection onto the image, respectively.

[0097] In urban flood scenes, water surface textures are weak and viewing angles vary greatly, making traditional keypoint matching methods difficult to apply. This method eliminates the need for keypoint extraction and can perform dense feature matching across the entire image. The specific process includes the following two stages:

[0098] (1) Coarse-grained matching stage

[0099] Using the 1 / 8 resolution feature map extracted in step S31 as input, a score matrix is ​​constructed based on the similarity calculated from the feature vectors to describe the matching confidence between all pixel locations. Based on this, a nearest neighbor (MNN) matching strategy is employed to select initial matching point pairs from the score matrix as a candidate set for subsequent fine-tuning. This method does not require explicit keypoint detection and is suitable for scenarios with weak textures and large viewpoint differences, such as urban flooding.

[0100] (2) Refined matching stage

[0101] Pixel-level fine-grained matching: Based on the initial point pairs obtained in the coarse-grained matching stage, the 1 / 2 scale feature map from step S31 is called to calculate the dot product similarity of feature vectors in the local region of each pair of matching points, and a local score map is constructed. Subsequently, the nearest neighbor (MNN) strategy is used for filtering to obtain pixel-level matching point pairs with more precise spatial location;

[0102] Subpixel-level fine-tuning: Based on pixel-level matching results, a 3×3 local region is constructed around each pair of matching points. The feature similarity score at each location is calculated using the feature vector dot product method, and a matching probability distribution is generated through softmax normalization. A weighted average coordinate is calculated based on this probability map to achieve subpixel-level fine-tuning of the matching point positions, further improving matching accuracy and distribution continuity.

[0103] S33: Based on the dense feature correspondence, establish the projection equation of a three-dimensional spatial point onto a two-dimensional plane to obtain the camera pose;

[0104] The projection equation of the three-dimensional point onto the two-dimensional plane is as follows:

[0105] (2-3)

[0106] in, This represents the homogeneous coordinates of a point in three-dimensional space in the world coordinate system. This represents the homogeneous coordinates of the point's projection position on the image plane. The scaling factor corresponds to the depth value of the point in the camera coordinate system. This depth value is calculated by the geometric relationship between the spatial ray formed by the back projection of the image points and the three-dimensional model constructed in step S1. This is the intrinsic parameter matrix of the camera. and These are its rotation matrix and translation vector, respectively.

[0107] Expand the equation into a system of linear equations and obtain the projection matrix. ,right Decompose the matrix and calculate the rotation matrix. Translation vector This gives the camera's pose.

[0108] S4: Acquire urban flood scene video captured by the camera, extract the pixel positions of the water surface edges, combine the camera pose with the 3D model of the target area, and calculate its spatial position through back projection to invert the water depth in the area; specifically, during urban flooding, combine the water surface edges in the camera image with the camera pose, and invert the flood inundation depth of the area surrounding buildings based on the image projection relationship; wherein, the water surface edges are extracted by the SAM2 segmentation model, and an autoregressive mechanism is introduced to optimize its time series stability. This step may include the following sub-steps:

[0109] S41: Based on the SAM2 segmentation model, water surface regions are extracted from the input video image sequence to obtain the initial water surface mask in each frame. Considering that buildings are prone to reflections, this invention introduces an autoregressive model to perform cross-frame temporal optimization of the segmentation results. This model uses the water surface edge positions in historical frames as a reference to predict the stable boundary trajectory of the current frame, ultimately outputting a continuous, smooth water surface edge line without obvious jumps, providing reliable input for subsequent projection inversion.

[0110] S42: Using the camera parameters obtained during camera pose recovery, project the water surface edge line into three-dimensional space and calculate the coordinates of the water surface edge points in the camera coordinate system:

[0111] Having obtained the camera's pose and intrinsic parameter matrix in the 3D model through the aforementioned steps, the pixel coordinates of the water surface edge points in the captured image can be back-projected to their 3D spatial positions in the camera coordinate system.

[0112] (3-1)

[0113] in, These are the position coordinates of a point at the edge of the water surface in the camera coordinate system.

[0114] S43: Using the rotation matrix obtained in step S3 and translation vector Convert camera coordinates to world coordinates The formula is as follows:

[0115] (3-2)

[0116] S44: Based on the images acquired before the flood, the world coordinates of the building's base are calculated using the process in steps S41-S43. The submerged water depth of the building is then determined based on the world coordinates of the water surface edge and the building's base.

[0117] The following simulated embodiments illustrate the practical application of the method of the present invention in engineering. The examples demonstrate that the present invention can be used in engineering practice and can achieve technical effects.

[0118] Taking a flood simulation at the test site as an example, the test site was equipped with 10 sets of building models. A water level gauge was placed at the bottom of each building model, and nine cameras were positioned on top of the building models, inside the grass, and along both sides of the road to simulate the widespread distribution of cameras in a real urban environment. First, the intrinsic and extrinsic parameters of the cameras were calculated using multi-angle UAV images to construct an initial 3D model of the test area. Then, using a feature trajectory and geometric structure iterative refinement strategy, the 3D model of the test area was obtained, as shown below. Figure 2 As shown. Next, acquire two-dimensional images captured by cameras within the target area and match them with the drone images to obtain several camera-drone image pairs with the closest captured images; perform dense feature matching on the camera-drone image pairs, and the feature point matching is as follows. Figure 3 As shown, the spatial pose of the camera is estimated, and the result is as follows. Figure 4 As shown. Finally, urban flooding was simulated, and the test conditions were as follows. Figure 5 As shown in (a), the video of the urban flood scene captured by the camera is obtained, and the water surface edge is extracted based on the water surface segmentation result. Figure 5 In (b) of this invention, the spatial location of the urban flood inundation depth is calculated by back projection, combining the camera pose and the 3D model. The method of this invention and a water level gauge are used to monitor the water depth during flooding. The water depth results monitored by this invention and the actual water depth measured by the water level gauge are as follows: Figure 5 As shown in (c), comparing the two, the monitoring error of one of the cameras in this method is as follows: Figure 5 As shown in (d) above, the errors of the nine cameras in the flood simulation experiment are shown in Table 1:

[0119] Table 1. Error of water depth monitoring using this method

[0120]

[0121] It can be seen that the water depth monitoring results using the method of this invention are in good agreement with the measured data. The relative error of water depth identified by each camera is less than 10%, and the average error is less than 5%, indicating that the water depth identification effect of this invention is good. Therefore, the urban flood water depth real-time monitoring method based on intelligent three-dimensional vision proposed in this invention realizes the real-time acquisition of urban flood water depth data, solving the problems of low computational efficiency, large basic data requirements, and long model construction cycle of traditional methods. It provides important technical support for solving the problem of refined real-time early warning of urban floods and has practical engineering application value.

[0122] Corresponding to the aforementioned embodiment of a method for real-time monitoring of urban flood water depth based on intelligent 3D vision, this application also provides an embodiment of a device for real-time monitoring of urban flood water depth based on intelligent 3D vision.

[0123] Figure 6 This is a block diagram illustrating a real-time urban flood water depth monitoring device based on intelligent 3D vision, according to an exemplary embodiment. (Refer to...) Figure 6 The device may include:

[0124] The 3D reconstruction module 21 is used to calculate the camera intrinsic and extrinsic parameters corresponding to the images using multi-angle UAV images, construct an initial 3D model of the target area, and optimize the 3D model of the target area using a feature trajectory and geometric structure iterative refinement strategy.

[0125] Matching module 22 is used to acquire two-dimensional images captured by cameras in the target area and match them with the drone images to obtain several camera-drone image pairs with the closest captured images;

[0126] The pose estimation module 23 is used to perform dense feature matching on the camera-UAV image pair in order to estimate the spatial pose of the camera.

[0127] The water depth inversion module 24 is used to acquire urban flood scene videos captured by the camera, extract the pixel positions of the water surface edges, combine the camera's spatial pose with the 3D model of the target area, and calculate its spatial position through back projection, thereby inverting the water depth in the area. Regarding the apparatus in the above embodiments, the specific methods by which each module performs its operation have been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0128] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0129] Accordingly, this application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the above-described method for real-time monitoring of urban flood water depth based on intelligent 3D vision.

[0130] Accordingly, this application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the above-described method for real-time monitoring of urban flood water depth based on intelligent three-dimensional vision. Figure 7 The diagram shown is a hardware structure diagram of any device with data processing capabilities, including an urban flood water depth real-time monitoring device based on intelligent 3D vision provided by an embodiment of the present invention. (Except for...) Figure 7 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0131] Accordingly, this application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the aforementioned method for real-time monitoring of urban flood water depth based on intelligent 3D vision. The computer-readable storage medium can be an internal storage unit of any data processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0132] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

Claims

1. A method for real-time monitoring of urban flood water depth based on intelligent three-dimensional vision, characterized in that, include: S1: Using multi-angle UAV images, calculate the camera intrinsic and extrinsic parameters corresponding to the images, construct an initial 3D model of the target area, and use feature trajectory and geometric structure iterative refinement strategy to obtain the 3D model of the target area; S2: Acquire two-dimensional images captured by cameras within the target area and match them with the drone images to obtain several camera-drone image pairs with the closest captured images; S3: Perform dense feature matching on the camera-drone image pair to estimate the spatial pose of the camera; S4: Acquire the video of the urban flood scene captured by the camera, extract the pixel position of the water surface edge, combine the camera spatial pose with the three-dimensional model of the target area, calculate its spatial position through back projection, and thus invert the water depth of the area. Step S1 includes: S11: Utilizing multi-angle UAV images, a feature matching method with a fusion grid quantization strategy is used to generate the correspondence between feature points in the multi-view images and construct the initial feature trajectory; through the motion structure recovery method, camera parameters are calculated and 3D point coordinates are reconstructed to generate the initial 3D model of the target area and establish the mapping relationship between 3D points and image pixels. S12: Based on the initial feature trajectory and the initial three-dimensional model of the target region, the feature trajectory and geometric structure are alternately optimized through an iterative refinement process to reconstruct a high-precision three-dimensional point cloud of the target region; Specifically, the feature trajectory and geometric structure are alternately optimized through an iterative refinement process: Based on the initial feature trajectory, a multi-view matching method based on the Transformer architecture is used to optimize the trajectory. By constructing the feature correspondence between the reference image and other views, the matching confidence and consistency under each view are calculated, the trajectory point positions are updated, and the optimized multi-view feature trajectory set is obtained. Based on the optimized multi-view feature trajectory set, geometric optimization is performed. Bundle adjustment is used to jointly optimize camera extrinsic and intrinsic parameters as well as 3D point coordinates. The optimization objective is to minimize the reprojection error of all 3D points in the image. The error function is defined as follows: ， in, This represents the theoretical position of a 3D point projected onto the image plane. This corresponds to the actual observation location in the image. This represents the number of feature points participating in the optimization.

2. The method for real-time monitoring of urban flood water depth based on intelligent three-dimensional vision according to claim 1, characterized in that, Step S2 includes: S21: Extracting local visual features from 2D camera images and drone images using convolutional neural networks. It aggregates local visual features into a global feature descriptor for the image. ; S22: Based on the global feature descriptors of drone images and camera images, match the camera images and drone images to obtain several camera-drone image pairs with the closest captured images.

3. The method for real-time monitoring of urban flood water depth based on intelligent three-dimensional vision according to claim 1, characterized in that, Step S3 includes: S31: Multi-scale feature extraction is performed on the camera-drone image pair using a convolutional neural network to obtain coarse-grained feature maps and high-resolution feature maps, and feature enhancement is performed using self-attention mechanism and cross-attention mechanism; S32: Based on the enhanced features, a two-stage feature matching method is used to match image pairs to obtain dense feature correspondences; S33: Based on the dense feature correspondence, establish the projection equation of a three-dimensional spatial point onto a two-dimensional plane to obtain the camera pose; The projection equation of the three-dimensional point onto the two-dimensional plane is as follows: ， in, This represents the homogeneous coordinates of a point in three-dimensional space in the world coordinate system. This represents the homogeneous coordinates of the point's projection position on the image plane. This is the scaling factor, corresponding to the depth value of the point in the camera coordinate system. This is the intrinsic parameter matrix of the camera. and These are its rotation matrix and translation vector, respectively.

4. The method for real-time monitoring of urban flood water depth based on intelligent three-dimensional vision according to claim 3, characterized in that, In step S32, the two-stage feature matching method includes: (1) Coarse-grained matching stage Using the coarse-grained feature map as input, similarity is calculated based on feature vectors to construct a score matrix of matching confidence between images. An nearest neighbor matching strategy is adopted to select initial matching point pairs from the score matrix. (2) Refined matching stage Based on the initial matching point pairs, the high-resolution feature map is used to calculate the dot product similarity of feature vectors in the local region of each pair of matching points, construct a local score map, and use the nearest neighbor strategy to filter and obtain pixel-level matching point pairs. Based on the pixel-level matching point pairs, a local region of a predetermined size is constructed around each pair of matching points. The feature similarity score at each position is calculated using the feature vector dot product method, and the matching probability distribution is normalized. The weighted average coordinates are calculated based on the matching probability distribution to obtain dense feature pairs.

5. The method for real-time monitoring of urban flood water depth based on intelligent three-dimensional vision according to claim 3, characterized in that, Step S4 includes: S41: Based on the SAM2 image segmentation model, the water surface region is extracted from the input video image sequence to obtain the initial water surface mask in each frame image. The initial water surface mask is then optimized across frames using an autoregressive model to obtain the water surface region segmentation result. S42: Using the camera parameters obtained during the camera pose recovery process, project the water surface edge line based on the water surface region segmentation result onto three-dimensional space, and calculate the coordinates of the water surface edge points in the camera coordinate system: ， in, These are the position coordinates of a point at the edge of the water surface in the camera coordinate system. S43: Using the rotation matrix obtained in step S3 and translation vector Convert camera coordinates to world coordinates The formula is as follows: ， S44: Based on the images acquired before the flood, the world coordinates of the building's base are calculated using the process in steps S41-S43. The submerged water depth of the building is then determined based on the world coordinates of the water surface edge and the building's base.

6. A real-time urban flood water depth monitoring device based on intelligent three-dimensional vision, used to implement the method described in claim 1, characterized in that, include: The 3D reconstruction module is used to calculate the camera intrinsic and extrinsic parameters corresponding to the images using UAV images from multiple angles, construct an initial 3D model of the target area, and obtain the 3D model of the target area by using feature trajectory and geometric structure iterative refinement strategy. The matching module is used to acquire two-dimensional images captured by cameras within the target area and match them with the drone images to obtain several camera-drone image pairs with the closest captured images. The pose estimation module is used to perform dense feature matching on the camera-UAV image pairs to estimate the spatial pose of the camera. The water depth inversion module is used to acquire urban flood scene videos captured by the camera, extract the pixel positions of the water surface edges, combine the camera's spatial pose with the 3D model of the target area, calculate its spatial position through back projection, and thus invert the water depth of the area.

7. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the method as described in any one of claims 1-5.

8. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

9. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the steps of the method as described in any one of claims 1-5.

Images