Real-time monitoring method for urban flood surface flow velocity based on binocular vision and point tracking

Abstract

The application discloses a kind of urban flood surface flow rate real-time monitoring methods based on binocular vision and point tracking, comprising: obtaining the calibration parameters of binocular camera in urban flood monitoring area and the synchronous flood surface video of binocular camera, and correcting to obtain left visual angle video frame sequence and right visual angle video frame sequence;Based on point tracking model, the water surface effectively continuous tracking motion feature in the left visual angle video frame sequence is cross-frame tracked, and the two-dimensional trajectory of each water surface floating object feature point in each video frame is obtained;Right visual angle feature point matching is carried out to each water surface floating object feature point in left visual angle video frame sequence, and based on the matching result, binocular triangulation is carried out, and the three-dimensional space coordinates of water surface floating object feature point at different time are restored;According to the time interval between frames and the three-dimensional space coordinate difference of each water surface floating object feature in the time interval, the representative flow rate of urban flood surface is calculated.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of water conservancy and computer vision, and in particular relates to a method for real-time monitoring of urban flood surface flow velocity based on binocular vision and point tracking. Background Technology

[0002] In recent years, influenced by factors such as frequent extreme rainfall, increased impermeable urban areas, and localized overload of drainage systems, urban flooding has become characterized by its sudden onset, significant spatial variations, rapid localized evolution, and high safety risks. Low-lying areas along roads, underpasses, subway entrances, pump station forebays, areas around drainage outlets, and entrances to underground spaces are prone to rapid confluence and high-velocity surface runoff, which can easily cause pedestrians to lose their footing while wading, vehicles to stall and drift, manhole covers to become lodged, infrastructure damage, and obstruction of emergency passage. Therefore, real-time acquisition of urban flood surface flow velocity is of great significance for flood risk assessment, traffic control, personnel evacuation, and emergency dispatch.

[0003] Existing methods for monitoring urban flood flow velocity mainly include contact current meters, radar velocity measurement, hydrodynamic model simulation, and conventional video velocity measurement. Contact current meters need to be deployed inside the water body, resulting in high installation and maintenance costs. They are also susceptible to impact or blockage in urban environments with water accumulation on roads, numerous floating objects, and heavy rainfall. While radar velocity measurement equipment can achieve non-contact measurement, the cost of a single point device is high, making large-scale, dense deployment difficult. Conventional video velocity measurement typically obtains two-dimensional pixel displacement by tracking floating objects, texture patches, or water surface ripples in images and relies on preset water surface elevation, planar assumptions, or scale mapping relationships to calculate the actual flow velocity. When the water surface elevation or road slope changes, the original mapping relationship is prone to deviation, leading to instability in the recovery of the actual scale. Hydrodynamic model simulation relies on a large amount of input data such as rainfall, pipe networks, topography, and roughness. The cost of model construction and parameter maintenance is high, making it difficult to meet the needs of rapid, real-time, and dynamic correction in localized flood scenarios. Summary of the Invention

[0004] To effectively address the challenges of deploying and maintaining contact-based equipment, unstable scale recovery in monocular vision, insufficient accuracy of ordinary video velocity measurement, and limited usability in complex flood scenarios in existing urban flood flow velocity monitoring, this invention proposes a real-time monitoring method for urban flood surface flow velocity based on binocular vision and point tracking.

[0005] According to a first aspect of the embodiments of this application, a method for real-time monitoring of urban flood surface flow velocity based on binocular vision and point tracking is provided, comprising: S1: Obtain the calibration parameters of the binocular camera and the synchronous flood surface video of the binocular camera within the urban flood monitoring area, and correct to obtain the left-view video frame sequence and the right-view video frame sequence; S2: Based on the point tracking model, the motion features of the water surface in the left-view video frame sequence can be effectively and continuously tracked across frames to obtain the two-dimensional trajectory of each floating object feature point in each video frame. S3: Match right-view feature points for each floating object feature point in the left-view video frame sequence, perform binocular triangulation based on the matching results, and recover the three-dimensional spatial coordinates of the floating object feature points at different times. S4: Calculate the representative surface flow velocity of urban flooding based on the inter-frame time interval and the three-dimensional spatial coordinate difference of each floating object within that time interval.

[0006] Further, step S1 includes: S11: Calibrate the binocular cameras and obtain the intrinsic parameter matrix of the left camera, the intrinsic parameter matrix of the right camera, the distortion parameters, the relative rotation matrix and translation vector between the left and right cameras, and the video frame rate; S12: Determine the binocular baseline length based on the translation vector between the left and right cameras, and determine the time interval between adjacent video frames based on the video frame rate; S13: Acquire flood surface video with synchronized left and right perspectives, and perform distortion correction and epipolar correction based on the distortion parameters, the relative rotation matrix and translation vector between the left and right cameras to obtain the left perspective video frame sequence and the right perspective video frame sequence.

[0007] Further, step S2 includes: S21: Determine the water surface area in the initial query frame of the left-view video frame sequence; S22: In the water surface area, several initial query points are automatically deployed according to a preset grid size; S23: Based on the left-view video frame sequence and the initial query point, the two-dimensional trajectory of the initial query point in the continuous video frames is obtained using a point tracking model, and effective floating object feature points and their two-dimensional trajectories are selected according to the cross-frame motion state of each query point.

[0008] Further, in step S23, based on the visibility information output by the point tracking model, it is determined whether all two-dimensional trajectory points corresponding to each initial query point are valid visible points. If so, that is, the initial query point can be effectively and continuously tracked, then it is determined to be the feature point of the floating object on the water surface, and its two-dimensional trajectory is the two-dimensional trajectory of the feature point of the floating object on the water surface.

[0009] Further, step S3 includes: S31: Based on the left-view video frame sequence and the right-view video frame sequence, feature points corresponding to the feature points of floating objects on the water surface in each right-view video frame and the feature points of each left-view video frame at the same time are obtained by feature matching, and left-right view feature point pairs with matching confidence are constructed. S32: Based on a predetermined matching confidence threshold, filter out valid left and right view feature point pairs that meet the matching confidence requirements; S33: Based on the effective left and right view feature point pairs, and combined with the calibration parameters of the binocular camera, perform binocular triangulation to restore the three-dimensional spatial coordinates of each floating object feature point at the corresponding time.

[0010] Further, step S4 includes: Based on the three-dimensional spatial coordinate difference and time interval of the water surface motion characteristics at two adjacent moments, the effective instantaneous flood surface velocity of each floating object characteristic is calculated. Quantile statistics were performed on the effective instantaneous flood surface velocity of each water surface movement characteristic within the same time interval, so as to select the representative urban flood surface velocity within that time interval.

[0011] Furthermore, if no effective instantaneous flood surface velocity exists for any of the water surface movement characteristics within a time interval, then the effective representative velocity of the previous moment or the preset default velocity is used as the compensation velocity for the current moment.

[0012] According to a second aspect of the embodiments of this application, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the method described in the first aspect.

[0013] According to a third aspect of the embodiments of this application, an electronic device is provided, comprising: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method as described in the first aspect.

[0014] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0015] The technical solution provided in this application utilizes a point tracking model to obtain the cross-frame trajectory of trackable motion features of the water surface in a left-view video; it uses feature point matching and binocular triangulation in the right-view video frames to recover the three-dimensional spatial coordinates of each trajectory point; and then calculates the surface flow velocity of the water surface based on the three-dimensional spatial displacement of adjacent frames and the time interval between frames, thereby obtaining the time series of surface flow velocity in urban flooding, which has the following beneficial effects: (1) The present invention uses a binocular camera to recover the three-dimensional spatial coordinates of the water surface motion characteristics, avoids the strong dependence of monocular video velocity measurement on the assumption of an ideal horizontal plane, and improves the ability to recover the true scale in urban flood flow velocity measurement.

[0016] (2) The present invention uses the CoTracker visual AI point tracking model to track the trackable motion features of the water surface across frames, which can obtain continuous two-dimensional trajectory and visibility information in the video sequence, thereby improving the automation of water surface motion feature extraction and tracking.

[0017] (3) This invention restores the three-dimensional coordinates of the trajectory points by matching feature points from the left and right perspectives and binocular triangulation, and calculates the flow velocity based on the three-dimensional displacement of adjacent frames. It can transform image motion into real space motion and is applicable to urban flooding scenarios such as road water accumulation, underpasses, subway entrances and exits and drainage outlets.

[0018] (4) The present invention uses visibility information, matching confidence, physical velocity threshold and 80th percentile statistics to screen and robustly estimate the flow velocity results, which can reduce the impact of false tracking, reflection, occlusion and local abnormal movement, and improve the stability of the flow velocity time series.

[0019] In summary, this invention proposes a real-time monitoring algorithm for urban flood surface velocity based on binocular vision and point tracking. Utilizing deployable binocular cameras in cities, it directly acquires surface velocity in flood scenarios through visual AI point tracking, binocular feature point matching, triangulation, and 3D displacement velocimetry. This algorithm features non-contact operation, rapid response, high automation, and applicability to complex urban flood scenarios. It effectively solves the problems commonly found in traditional velocity monitoring methods, such as high deployment costs, insufficient local real-time performance, and unstable scale recovery, providing technical support for the scientific management and emergency decision-making of urban flood disasters.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0022] Figure 1 This is a flowchart illustrating a method for real-time monitoring of urban flood surface flow velocity based on binocular vision and point tracking, according to an exemplary embodiment.

[0023] Figure 2 This is a schematic diagram illustrating the tracking results of floating objects and trackable motion features on the water surface based on CoTracker, according to an exemplary embodiment.

[0024] Figure 3 This is a schematic diagram illustrating the feature point matching results of left and right view images from a binocular camera according to an exemplary embodiment.

[0025] Figure 4 This is a schematic diagram illustrating the three-dimensional positioning and three-dimensional trajectory visualization results of water surface motion characteristics according to an exemplary embodiment.

[0026] Figure 5 This is a time series diagram of surface flow velocity in urban flooding, illustrated according to an exemplary embodiment.

[0027] Figure 6 This is a block diagram illustrating a real-time monitoring device for urban flood surface flow velocity based on binocular vision and point tracking, according to an exemplary embodiment.

[0028] Figure 7 This is a schematic diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0030] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0031] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0032] Figure 1 This is a flowchart illustrating a real-time monitoring method for urban flood surface flow velocity based on binocular vision and point tracking, according to an exemplary embodiment. Figure 1 As shown, the method may include the following steps: S1: Acquire the calibration parameters of the binocular cameras and the synchronous flood surface video of the binocular cameras within the urban flood monitoring area; Specifically, this step addresses the difficulty of restoring the true scale in urban flood flow velocity monitoring. It utilizes fixedly deployed binocular cameras to acquire left and right-view videos with overlapping fields of view, and obtains the geometric relationship between the left and right cameras through binocular calibration. This step may include the following sub-steps: S11: Calibrate the binocular cameras and obtain the intrinsic parameter matrix of the left camera, the intrinsic parameter matrix of the right camera, the distortion parameters, the relative rotation matrix and translation vector between the left and right cameras, and the video frame rate.

[0033] Let the intrinsic parameter matrices of the left and right cameras be respectively: in, and Let these represent the intrinsic parameter matrices of the left and right cameras, respectively. , This represents the equivalent focal length of the left camera in both pixel coordinate directions. , Indicates the coordinates of the principal point of the left camera; , This represents the equivalent focal length of the right camera in both pixel coordinate directions. , This indicates the coordinates of the principal point of the right camera.

[0034] The translation vector is represented as: S12: Determine the binocular baseline length based on the translation vector between the left and right cameras, and determine the time interval between adjacent video frames based on the video frame rate. Binocular baseline length B Translation vector Euclidean norm: If the video frame rate is fps, then the time interval between adjacent video frames is: S13: Acquire flood surface videos with synchronized left and right perspectives, perform distortion correction and epipolar correction, and obtain left-view video frame sequences and right-view video frame sequences for water surface motion feature tracking and binocular matching.

[0035] Left and right view videos can be synchronously acquired through hardware synchronization triggering, or they can be synchronously corrected through software using timestamps.

[0036] Among them, distortion correction corrects the radial and tangential distortions in the left and right view video frames based on the distortion parameters of the left and right cameras obtained by dual-camera calibration; epipolar correction makes the corresponding points of the same spatial point in the left and right images lie on the same epipolar line based on the relative rotation matrix and translation vector between the left and right cameras.

[0037] S2: Based on the point tracking model, the motion features of the water surface in the left-view flood video frame sequence can be effectively and continuously tracked across frames to obtain the two-dimensional trajectory of each floating object feature point on the water surface. Specifically, this step is used to obtain the two-dimensional trajectory of water surface motion features in consecutive frames from a left-view flood video frame sequence. This step may include the following sub-steps: S21: Determine the water surface region in the initial query frame of the left-view flood video frame sequence. The water surface region can be determined by manual specification, fixed monitoring area configuration, water surface segmentation model, foreground motion detection model, or a combination thereof.

[0038] S22: In the water surface area, several initial query points are automatically deployed according to a preset grid size. The preset grid size is used to control the deployment density of the initial query points. The number of initial query points contained in each area of ​​a floating object or other trackable motion feature is determined by the area range and the grid deployment density. In one embodiment, the grid size parameter is set to 30, that is, initial query points are generated according to a 30×30 regular grid, and query points located in the water surface area are retained; then, valid query points are selected according to the motion state of each initial query point in consecutive video frames.

[0039] S23: Based on the left-view flood video frame sequence and the initial query point, the two-dimensional trajectory of the initial query point in the continuous video frames is obtained using a point tracking model, and effective floating object feature points and their two-dimensional trajectories are selected according to the cross-frame motion state of each query point. The floating object feature points refer to effective query points that are located in the floating object area that moves with the water flow and can be continuously tracked.

[0040] Specifically, a point tracking model refers to a model capable of tracking the position of a given query point within consecutive video frames based on a video sequence and outputting a corresponding two-dimensional trajectory. The point tracking model can employ CoTracker, TAPIR, PIPs, or other models with similar functionality; this application uses CoTracker as an example. The query point refers to the initial pixel position selected within the water surface region of the video image and input into the point tracking model to obtain the two-dimensional trajectory of the corresponding water surface motion features within consecutive video frames.

[0041] like Figure 2As shown, dots of different colors represent the tracking positions of different floating object motion features in the left-view flood video frame sequence. The homogeneous pixel coordinates of the i-th water surface motion feature in the t-th frame of the left-view image are represented as: in and They represent the first i The feature points of the floating objects on the water surface at the first t The horizontal and vertical pixel coordinates in the left-view image frame are given, with the superscript L indicating the left-view image. These coordinates represent the coordinates of a single feature point of a floating object on the water surface, not the overall coordinates of the region containing that object.

[0042] The i-th water surface motion feature is T The two-dimensional trajectory sequence in a frame video is represented as: Based on the visibility information output by the point tracking model (i.e., information output by the point tracking model that characterizes whether the query point is effectively visible, occluded, or has failed tracking in the corresponding video frame), it is determined whether each two-dimensional trajectory point is a valid visible point. The visibility information is represented as follows: in, Indicates the first i The water surface motion characteristics in the first t Frame validly visible. This indicates that the water surface movement characteristic is in the first... t The frame is not visible, is occluded, or tracking has failed.

[0043] If all two-dimensional trajectory points corresponding to an initial query point are valid visible points, that is, the initial query point can be effectively and continuously tracked, then it is determined that it can be used as the feature point of the floating object on the water surface.

[0044] S3: Match right-view feature points for each floating object feature point in the left-view video frame sequence, perform binocular triangulation based on the matching results, and recover the three-dimensional spatial coordinates of the floating object feature points at different times. Specifically, this step converts the two-dimensional trajectory points from the left-view perspective into points in real three-dimensional space, thereby providing spatial coordinates with a real scale for subsequent flow velocity calculations. This step may include the following sub-steps: S31: Based on the left-view video frame sequence and the right-view video frame sequence, feature points corresponding to the feature points of floating objects on the water surface in each right-view video frame and the feature points of each left-view video frame at the same time are obtained by feature matching, and left-right view feature point pairs with matching confidence are constructed. like Figure 3As shown, the lines represent the matching relationship between the tracking points in the left view and the feature points in the right view. The matching process is expressed as follows: in, For the left-view image of frame t, This is the right-view image of frame t. This represents a feature point matching model. Indicates the first i The feature points of the floating objects on the water surface at the first t The matching confidence of a frame is used to characterize the reliability of left-view feature points and right-view feature points belonging to the same water surface motion feature. This represents its homogeneous pixel coordinates in the right-view image: S32: Based on a predetermined matching confidence threshold, filter out valid left and right view feature point pairs that meet the matching confidence requirements; The matching confidence level filtering criteria are: in, A preset matching confidence threshold is set. In one embodiment, a threshold value can be set to... =0.5.

[0045] S33: Based on the effective left and right view feature point pairs, and combined with the calibration parameters of the binocular camera, perform binocular triangulation to restore the three-dimensional spatial coordinates of each floating object feature point at the corresponding time.

[0046] like Figure 4 As shown, the three-dimensional points and trajectory lines represent the spatial motion process of water surface motion features in continuous video frames.

[0047] The projection matrices of the left and right cameras are expressed as follows: in, It is an identity matrix. in, and These are the projection matrices for the left and right cameras, respectively.

[0048] The homogeneous coordinates of spatial points corresponding to the water surface motion characteristics are represented as follows: in, Indicates according to the first i The feature points of the floating objects on the water surface at the first tThe matching pixel coordinates in the left and right view images of the frame, along with the projection matrices of the left and right cameras, are used to obtain the three-dimensional homogeneous coordinates through binocular triangulation. These coordinates are referenced to the left camera coordinate system. , and These represent the horizontal coordinates, vertical coordinates, and depth along the optical axis of the feature point in the left camera coordinate system, respectively. For homogeneous coordinate scale components.

[0049] The left and right perspective projection relationship is as follows: in, and The first i The feature points of the floating objects on the water surface at the first t The scale factor in the left and right view projection relationship of the frame corresponds to its depth along the optical axis in the left and right camera coordinate systems, respectively, under homogeneous coordinate normalization.

[0050] Expanding the above projection relationships, we can construct a system of linear equations: in: in, , , These represent the projection matrices of the left camera. The first, second, and third lines, , , These represent the projection matrix of the right camera. The first, second, and third rows. The homogeneous coordinates of the spatial points are obtained by solving the following optimization problem: Normalizing the homogeneous coordinates yields the non-homogeneous three-dimensional space coordinates: in, That is, the three-dimensional spatial coordinates of the i-th water surface motion feature in the t-th frame.

[0051] S4: Calculate the representative surface flow velocity of urban flooding based on the inter-frame time interval and the three-dimensional spatial coordinate difference of each floating object in the time interval. Specifically, this step is used to calculate the instantaneous flood surface velocity based on the three-dimensional displacement of the same water surface motion feature in consecutive frames, and to perform robust statistics on the velocities of multiple effective water surface motion features. This step may include the following sub-steps: S41: Calculate the effective instantaneous flood surface velocity of each floating object feature based on the three-dimensional spatial coordinate difference and time interval between two adjacent moments of water surface motion characteristics; Specifically, it is determined whether the same water surface motion feature satisfies both the visibility condition and the matching confidence condition in two adjacent frames. The effective set of water surface motion features used to calculate the velocity from frame t to frame t+1 can be represented as: For water surface motion features that meet the conditions, the instantaneous flood surface velocity is calculated based on the difference in their three-dimensional spatial coordinates between two adjacent frames and the inter-frame time interval. The i-th water surface motion feature is in the... t Frame to the t The 3D displacement between +1 frames is: The instantaneous velocity vector is: In this embodiment, the main output is the magnitude of the velocity. The instantaneous velocity magnitude is calculated as follows: The Euclidean distance can be expanded as follows: Combining the frame rate (fps), the instantaneous velocity can also be written as: S43: Perform quantile statistics on the effective instantaneous flood surface velocity of each water surface movement characteristic within the same time interval, and then select the representative urban flood surface velocity within that time interval.

[0052] No. t Frame to the t The effective velocity set between +1 frames is: Preferably, the 80th percentile value is taken as the representative flow velocity at that moment: When there is no effective instantaneous flood surface velocity within a certain frame interval, the effective representative velocity of the previous moment or the preset default velocity is used as the compensation velocity for the current moment. like Figure 5 As shown, by continuously calculating the representative flow velocity at each frame interval, the time series of surface flow velocity in urban flooding can be obtained. In embodiments where flow direction needs to be obtained, the three-dimensional coordinates can be transformed from the binocular camera coordinate system to the on-site monitoring coordinate system, and the main flow direction can be calculated based on the horizontal velocity components: The following simulated examples illustrate the practical application of the method of the present invention in engineering. These examples demonstrate that the present invention can be used in engineering practice and achieve the desired technical effects. Taking the monitoring of surface flow velocity in urban road flooding at a simulated test site as an example, a set of binocular cameras is fixedly installed above the road model. The left and right cameras respectively acquire left and right view flood video frame sequences of the same area. First, the binocular cameras are calibrated to obtain the intrinsic parameters, distortion parameters, relative pose of the left and right cameras, baseline length, and video frame rate. Then, the area covering the flooded road in the left-view video is set as the water surface area, and multiple initial query points are selected within this area.

[0053] During the flood, the CoTracker visual AI point tracking model performed cross-frame tracking of query points in the left-view video, obtaining the two-dimensional trajectory and visibility information of each query point in consecutive frames, such as... Figure 2 As shown. For each left-view trajectory point in a frame, the system searches for feature points in the corresponding right-view image using a depth feature matching model, and filters valid feature point pairs based on matching confidence, such as... Figure 3 As shown. Subsequently, binocular triangulation was performed on the effective feature point pairs using the parameters of the left and right cameras to recover the three-dimensional spatial coordinates of each water surface motion feature point in each frame, and the three-dimensional motion trajectory was plotted, as shown. Figure 4 As shown.

[0054] Between adjacent frames, the system calculates the 3D coordinate difference of the same water surface motion feature point and divides it by the inter-frame time interval to obtain multiple instantaneous velocities. For multiple valid instantaneous velocities within the same frame interval, the system takes the 80th percentile value as the representative flow velocity at that moment. Finally, the system outputs a time series of representative flow velocities on flooded surfaces, such as... Figure 5 As shown.

[0055] In this embodiment, the three-dimensional coordinates of the water surface motion characteristics are obtained by binocular triangulation, and the velocity is calculated from the three-dimensional displacement of consecutive frames and the time interval between frames. Therefore, the algorithm does not rely on manual readings and does not need to directly approximate the two-dimensional pixel displacement as the actual displacement, which can improve the automation level and the ability to restore the true scale of urban flood surface flow velocity monitoring.

[0056] Corresponding to the aforementioned embodiments of the real-time monitoring method for urban flood surface flow velocity based on binocular vision and point tracking, this application also provides embodiments of a real-time monitoring device for urban flood surface flow velocity based on binocular vision and point tracking.

[0057] Figure 6 This is a block diagram illustrating a real-time monitoring device for urban flood surface flow velocity based on binocular vision and point tracking, according to an exemplary embodiment. (Refer to...) Figure 6 The device may include: The binocular video acquisition and correction module M1 is used to acquire the calibration parameters of the binocular camera and the synchronous flood surface video of the binocular camera within the urban flood monitoring area, and correct the left-view video frame sequence and the right-view video frame sequence. The feature point tracking module M2 is used to perform cross-frame tracking of the motion features of the water surface in the left-view video frame sequence based on the point tracking model, so as to obtain the two-dimensional trajectory of each floating object feature point in each video frame. The feature point 3D recovery module M3 is used to match the feature points of each floating object on the water surface in the left-view video frame sequence with the right-view feature points, and perform binocular triangulation based on the matching results to recover the 3D spatial coordinates of the floating object feature points at different times. The flow velocity calculation module M4 is used to calculate the representative flow velocity of urban flooded surfaces based on the inter-frame time interval and the three-dimensional spatial coordinate difference of each floating object within that time interval.

[0058] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0059] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0060] Accordingly, this application also provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the above-described method for real-time monitoring of urban flood surface flow velocity based on binocular vision and point tracking.

[0061] Accordingly, this application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the above-described method for real-time monitoring of urban flood surface flow velocity based on binocular vision and point tracking. Figure 7 The diagram shown is a hardware structure diagram of any device with data processing capabilities, including a real-time urban flood surface flow velocity monitoring device based on binocular vision and point tracking, provided by an embodiment of the present invention. (Except for...) Figure 7In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0062] Accordingly, this application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the aforementioned method for real-time monitoring of urban flood surface flow velocity based on binocular vision and point tracking. The computer-readable storage medium can be an internal storage unit of any data processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0063] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

Claims

1. A method for real-time monitoring of urban flood surface flow velocity based on binocular vision and point tracking, characterized in that, include: S1: Obtain the calibration parameters of the binocular camera and the synchronous flood surface video of the binocular camera within the urban flood monitoring area, and correct to obtain the left-view video frame sequence and the right-view video frame sequence; S2: Based on the point tracking model, the motion features of the water surface in the left-view video frame sequence can be effectively and continuously tracked across frames to obtain the two-dimensional trajectory of each floating object feature point in each video frame. S3: Match right-view feature points for each floating object feature point in the left-view video frame sequence, perform binocular triangulation based on the matching results, and recover the three-dimensional spatial coordinates of the floating object feature points at different times. S4: Calculate the representative surface flow velocity of urban flooding based on the inter-frame time interval and the three-dimensional spatial coordinate difference of each floating object within that time interval.

2. The method according to claim 1, characterized in that, Step S1 includes: S11: Calibrate the binocular cameras and obtain the intrinsic parameter matrix of the left camera, the intrinsic parameter matrix of the right camera, the distortion parameters, the relative rotation matrix and translation vector between the left and right cameras, and the video frame rate; S12: Determine the binocular baseline length based on the translation vector between the left and right cameras, and determine the time interval between adjacent video frames based on the video frame rate; S13: Acquire flood surface video with synchronized left and right perspectives, and perform distortion correction and epipolar correction based on the distortion parameters, the relative rotation matrix and translation vector between the left and right cameras to obtain the left perspective video frame sequence and the right perspective video frame sequence.

3. The method according to claim 1, characterized in that, Step S2 includes: S21: Determine the water surface area in the initial query frame of the left-view video frame sequence; S22: In the water surface area, several initial query points are automatically deployed according to a preset grid size; S23: Based on the left-view video frame sequence and the initial query point, the two-dimensional trajectory of the initial query point in the continuous video frames is obtained using a point tracking model, and effective floating object feature points and their two-dimensional trajectories are selected according to the cross-frame motion state of each query point.

4. The method according to claim 3, characterized in that, In step S23, based on the visibility information output by the point tracking model, it is determined whether all two-dimensional trajectory points corresponding to each initial query point are valid visible points. If so, that is, the initial query point can be effectively and continuously tracked, then it is determined to be the feature point of the floating object on the water surface, and its two-dimensional trajectory is the two-dimensional trajectory of the feature point of the floating object on the water surface.

5. The method according to claim 1, characterized in that, Step S3 includes: S31: Based on the left-view video frame sequence and the right-view video frame sequence, feature points corresponding to the feature points of floating objects on the water surface in each right-view video frame and the feature points of each left-view video frame at the same time are obtained by feature matching, and left-right view feature point pairs with matching confidence are constructed. S32: Based on a predetermined matching confidence threshold, filter out valid left and right view feature point pairs that meet the matching confidence requirements; S33: Based on the effective left and right view feature point pairs, and combined with the calibration parameters of the binocular camera, perform binocular triangulation to restore the three-dimensional spatial coordinates of each floating object feature point at the corresponding time.

6. The method according to claim 1, characterized in that, Step S4 includes: Based on the three-dimensional spatial coordinate difference and time interval of the water surface motion characteristics at two adjacent moments, the effective instantaneous flood surface velocity of each floating object characteristic is calculated. Quantile statistics were performed on the effective instantaneous flood surface velocity of each water surface movement characteristic within the same time interval, so as to select the representative urban flood surface velocity within that time interval.

7. The method according to claim 6, characterized in that, If no effective instantaneous flood surface velocity exists for any of the water surface movement characteristics within a time interval, then the effective representative velocity of the previous moment or the preset default velocity is used as the compensation velocity for the current moment.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the method as described in any one of claims 1-7.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the steps of the method as described in any one of claims 1-7.

Images