Non-contact river channel video flow measurement system and device based on smart phone

Abstract

The invention provides a non-contact river channel video flow measurement system and device based on a smart phone, belongs to the technical field of river channel flow velocity measurement in water resource monitoring, and aims to solve the problems that traditional contact flow measurement is high in cost and complex in operation, and mobile phone video flow measurement has perspective distortion and is difficult in floater identification. The system comprises a video acquisition module, a field calibration module, a video preprocessing module, a target identification and tracking module, a flow velocity calculation module, a front-end interaction module and a cloud service module. The device is composed of an intelligent mobile phone integrated with a front-end module and a cloud server for deploying a core algorithm. Non-contact flow measurement is completed through module cooperation, the cost is low, operation is convenient, precision is high, scene adaptability is high, and effective technical support is provided for water resource monitoring.

Description

Technical Field

[0001] This invention relates to the field of river flow velocity measurement technology in water resource monitoring, specifically to a non-contact river video flow measurement system and device based on a smartphone. Background Technology

[0002] Traditional river flow velocity measurement mainly relies on contact-based devices (such as current meters), which have two major limitations: First, the equipment is expensive and the operation process is complex, requiring professional personnel to operate it. It also has strict requirements on the measurement environment (such as river depth and flow stability), making it difficult to quickly respond to the diverse flow measurement needs of river scenarios. Second, if non-contact flow measurement is attempted using smartphone videos, perspective distortion will occur due to the flexible shooting angle of the phone (which cannot be strictly horizontal), leading to deviations in the identification of the position and trajectory of floating objects. At the same time, interference factors such as complex lighting and debris on the bank in the natural environment make traditional floating object identification methods less robust and less accurate, and unable to provide reliable data support for flow velocity calculation.

[0003] To address the aforementioned issues, this solution integrates video inverse perspective transformation, deep learning intelligent algorithms, and a cloud-mobile collaborative architecture to construct an integrated non-contact flow measurement system. This system overcomes the bottlenecks of traditional flow measurement in terms of cost, convenience, and scenario adaptability, providing a lightweight technical solution for water resource monitoring. Summary of the Invention

[0004] The present invention aims to solve the problems mentioned in the background art by providing a non-contact river video flow measurement system and device based on a smartphone.

[0005] The specific technical solution is as follows:

[0006] A smartphone-based non-contact river video flow measurement system includes:

[0007] The video acquisition module is used to collect video data from the river area;

[0008] The on-site calibration module, associated with the video acquisition module, is used to acquire calibration parameters at the river site. These calibration parameters are determined based on the size information of reference objects at the river site.

[0009] The video preprocessing module is connected to the video acquisition module and the on-site calibration module respectively. It is used to receive the video data and calibration parameters, perform inverse perspective transformation on the video data, and obtain the river video image from the frontal view.

[0010] The target recognition and tracking module is connected to the video preprocessing module and is used to process the river video image from the frontal view, identify floating objects on the water surface, and track the movement trajectory of the floating objects.

[0011] The flow velocity calculation module is connected to the target recognition and tracking module and the on-site calibration module, respectively, and is used to calculate the river flow velocity based on the motion trajectory of the floating object, the frame rate of the video data and the distance conversion information in the calibration parameters.

[0012] The front-end interaction module is integrated into the smartphone and is connected to the video acquisition module and the cloud service module respectively. It is used to upload video data, receive and display the water flow speed calculation results.

[0013] The cloud service module is connected to the video preprocessing module, the target recognition and tracking module, and the flow rate calculation module, respectively. It is used to deploy the core algorithms of video preprocessing, target recognition and tracking, and flow rate calculation, receive video data uploaded by the front-end interaction module, and feed back the calculated water flow rate results to the front-end interaction module.

[0014] In the aforementioned non-contact river video flow measurement system based on smartphones, the on-site calibration module acquires calibration parameters by using at least one of the following as reference objects: a fixed ruler set up on-site in the river, a buoy with known geometric dimensions, or a preset feature point. The system collects image information of the reference object within the field of view of the video acquisition module, and calculates the calibration parameters by combining the correspondence between the actual size of the reference object and the size in the image. The calibration parameters include perspective transformation parameters and distance conversion parameters.

[0015] In the aforementioned non-contact river video flow measurement system based on smartphones, the video preprocessing module employs a homography transformation algorithm when performing inverse perspective transformation processing. Based on the perspective transformation parameters provided by the on-site calibration module, a perspective matrix is ​​calculated. The perspective matrix is ​​then used to convert the tilted-view river video data acquired by the video acquisition module into a river video image from a normal viewing angle, thereby eliminating the influence of perspective distortion on the identification of floating object positions.

[0016] The aforementioned smartphone-based non-contact river video flow measurement system includes a target recognition and tracking module comprising a floating object recognition unit and a trajectory tracking unit. The floating object recognition unit employs a deep learning target detection algorithm, specifically either the YOLOv5 algorithm or the Mask R-CNN algorithm, to automatically identify floating objects on the water surface from a frontal viewpoint in the river video image and output the initial position information of the floating objects. The trajectory tracking unit employs a multi-target tracking algorithm, specifically at least one of the DeepSORT algorithm, Kalman filtering algorithm, or optical flow method, to track the positional changes of the floating objects in consecutive video frames based on their initial position information, generating motion trajectory data of the floating objects.

[0017] The aforementioned smartphone-based non-contact river video flow measurement system includes the following calculation process in its flow velocity calculation module:

[0018] The first step is to extract the floating object's motion trajectory data output by the target recognition and tracking module, and determine the spatial coordinates of the floating object at different times. The spatial coordinates are converted into actual physical coordinates based on the frontal view image output by the video preprocessing module and the distance conversion parameters of the on-site calibration module.

[0019] The second step is to calculate the displacement of the floating object based on its actual physical coordinates at adjacent moments, and then determine the time interval between adjacent moments by combining the frame rate of the video data.

[0020] The third step is to calculate the velocity of the floating object by the ratio of the displacement to the time interval, and then use the velocity of the floating object as the velocity of the river flow.

[0021] The aforementioned smartphone-based non-contact river video flow measurement system further includes an error analysis module connected to the flow velocity calculation module. This module is used to acquire flow velocity calculation results and relevant parameters from the video acquisition and calibration processes, and to analyze the sources of flow velocity calculation errors. These error sources include the shooting angle deviation of the video acquisition module, the calibration accuracy error of the on-site calibration module, and the resolution difference of the video data. The error analysis module is also used to output parameter adjustment suggestions based on the error sources to optimize the flow velocity calculation accuracy.

[0022] The aforementioned non-contact river video flow measurement system based on smartphones includes a front-end interaction module comprising a video interaction unit and a result display unit. The video interaction unit is used to control video shooting, preview video data, and upload video data to a cloud service module on the smartphone. The result display unit is used to display the water flow velocity calculation results in a graphical manner, including at least one of the following: displaying flow velocity values, playing animations of floating object trajectories, and plotting flow velocity change curves at different time periods.

[0023] The aforementioned non-contact river video flow measurement system based on smartphones includes a cloud service module that uses the Python programming language combined with a deep learning framework to deploy the core algorithm. The deep learning framework is either TensorFlow or PyTorch, and it is used to carry out the inverse perspective transformation calculation of the video preprocessing module, the deep learning model inference of the target recognition and tracking module, and the numerical calculation of the flow velocity calculation module. The cloud service module also has the ability to process concurrent requests from multiple users, and can simultaneously receive and process video data uploaded by multiple smartphone front-end interaction modules.

[0024] This invention also provides a smartphone-based non-contact river video flow measurement device, comprising a smartphone body and a cloud server. The smartphone body integrates the video acquisition module and front-end interaction module of the aforementioned smartphone-based non-contact river video flow measurement system. The camera of the smartphone body serves as the hardware carrier of the video acquisition module for acquiring river video data, and the display screen of the smartphone body is used to display the results of the front-end interaction module. The cloud server deploys the core algorithms of the aforementioned smartphone-based non-contact river video flow measurement system, including the on-site calibration module, video preprocessing module, target recognition and tracking module, flow velocity calculation module, and cloud service module. The cloud server establishes a data communication connection with the smartphone body via a wireless network to realize the transmission of video data and the feedback of flow measurement results.

[0025] The aforementioned non-contact river video flow measurement device based on a smartphone includes a front-end interaction module on the smartphone itself that also has a calibration assistance function. This function guides the user to take images of reference objects on-site in the river, uploads the reference object images and the actual size information of the reference objects input by the user to the on-site calibration module on the cloud server, and assists the on-site calibration module in calculating calibration parameters. Furthermore, the front-end interaction module supports offline caching of uploaded video data and corresponding flow measurement results, allowing users to view historical flow measurement records in environments without a network connection.

[0026] The present invention has the following beneficial effects:

[0027] 1. Breaking through the limitations of traditional flow measurement, achieving non-contact high-precision flow measurement: By eliminating shooting distortion through inverse perspective transformation, combining deep learning algorithms to improve the robustness of floating object identification and tracking, and then ensuring accurate distance conversion through on-site calibration, the entire chain from data source to calculation logic is optimized, avoiding the operational complexity of traditional contact flow measurement and the accuracy deviation of non-contact flow measurement, ensuring that the flow velocity calculation results can truly reflect the actual water flow state of the river.

[0028] 2. Reduce usage costs and barriers to entry, and improve convenience: Relying on the existing hardware of smartphones (camera, display screen) as the front-end carrier, there is no need to purchase dedicated flow measurement equipment, which greatly reduces hardware costs; the front-end interaction module simplifies the operation process (only three steps: calibration, shooting, and viewing), and the graphical result display eliminates the abstractness of data, so that non-professionals can get started quickly, breaking the dependence of traditional flow measurement on professionals.

[0029] 3. Strong scene adaptability, adapting to diverse river environments: The on-site calibration module can adjust parameters according to the river topography and width, and the inverse perspective transformation can adapt to any shooting angle. The deep learning algorithm can cope with complex lighting and background interference, enabling the system to work stably in scenarios such as narrow tributaries, open river surfaces, and banks with many debris. At the same time, the cloud service module supports multi-user concurrent processing to meet the needs of large-scale monitoring (such as multiple monitoring points measuring flow at the same time).

[0030] 4. Integrated workflow and optimizable performance ensure long-term reliability: Video acquisition, calibration, processing, calculation, and display are integrated into the mini-program and the cloud, achieving seamless end-to-end connectivity without requiring users to switch between multiple platforms; the error analysis module locates the causes of errors and outputs adjustment suggestions, combined with the remote update capability of cloud algorithms, continuously optimizing system accuracy and response speed, ensuring stability and reliability in long-term use.

[0031] 5. Offline caching expands usage scenarios and enhances practicality: The offline caching function of the front-end interaction module allows users to view historical flow measurement data even in remote river channels or environments with poor network signals. This reduces the system's strong dependence on the network, further expands the coverage of flow measurement scenarios, and enhances the flexibility of the device in practical applications. Attached Figure Description

[0032] Figure 1 A schematic diagram of the architecture of a smartphone-based non-contact river video flow measurement system provided in an embodiment of the present invention;

[0033] Figure 2 The shooting angle of the smartphone-based non-contact river video flow measurement system provided in this embodiment of the invention. With error Relationship diagram;

[0034] Figure 3 The on-site calibration accuracy of the smartphone-based non-contact river video flow measurement system provided in this embodiment of the invention. Standard and error Relationship diagram;

[0035] Figure 4 Video resolution and error of the smartphone-based non-contact river video flow measurement system provided in this embodiment of the invention. Comparison chart;

[0036] Figure 5 A graph showing the relationship between the number of YOLOv5 iterations and mAP@0.5 for the non-contact river video flow measurement system based on a smartphone provided in this embodiment of the invention;

[0037] Figure 6The DeepSORT maximum correlation distance and tracking effect curves of the non-contact river video flow measurement system based on smartphones provided in this embodiment of the invention. Detailed Implementation

[0038] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0039] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual images. They should not be construed as limiting the scope of this application. To better illustrate the embodiments of the present invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions of the product. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0040] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "inner," and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present application. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0041] In the description of this invention, unless otherwise explicitly specified and limited, the term "connection" or similar designation indicating a connection between components should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral part; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can refer to the internal communication between two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0042] Example 1

[0043] Reference Figures 1-6 ,in Figure 1 The architecture and components of a smartphone-based non-contact river video flow measurement system were demonstrated. Figure 2 This indicates that when θ ranges from 30° to 90°, It decreases linearly with increasing θ, which conforms to... The formula is 0.02 × (90° - θ); Figure 3 show When the target is increased from 0.5% to 2%, Linear increase, conforming to =1.5× The relationship between the targets; Figure 4 The results show that the errors for 480P, 720P, and 1080P are 2%, 1%, and 0.5%, respectively, visually demonstrating the impact of resolution on error. Figure 5 Simulation data shows that mAP increases with the number of iterations, eventually stabilizing at around 92%. Figure 6 Simulation data shows that the tracking effect improves with increasing maximum correlation distance, reaching up to 80%. The smartphone-based non-contact river video flow measurement system provided in this embodiment includes: a video acquisition module, a field calibration module, a video preprocessing module, a target recognition and tracking module, a front-end interaction module, and a cloud service module, wherein:

[0044] The video acquisition module is used to collect video data from the river area;

[0045] The on-site calibration module, associated with the video acquisition module, is used to acquire calibration parameters at the river site. The calibration parameters are determined based on the size information of the reference object at the river site.

[0046] The video preprocessing module is connected to the video acquisition module and the on-site calibration module respectively. It is used to receive video data and calibration parameters, perform inverse perspective transformation on the video data, and obtain the video image of the river from the frontal view.

[0047] The target recognition and tracking module is connected to the video preprocessing module and is used to process river video images from a frontal viewpoint, identify floating objects on the water surface, and track the movement trajectory of the floating objects.

[0048] The flow velocity calculation module is connected to the target recognition and tracking module and the on-site calibration module respectively. It is used to calculate the river flow velocity based on the motion trajectory of the floating object, the frame rate of the video data and the distance conversion information in the calibration parameters.

[0049] The front-end interaction module is integrated into the smartphone and connects to the video acquisition module and the cloud service module respectively. It is used to upload video data and receive and display the water flow speed calculation results.

[0050] The cloud service module is connected to the video preprocessing module, the target recognition and tracking module, and the flow rate calculation module. It is used to deploy the core algorithms for video preprocessing, target recognition and tracking, and flow rate calculation. It receives video data uploaded by the front-end interaction module and feeds back the calculated water flow rate results to the front-end interaction module.

[0051] By constructing a complete system architecture encompassing video acquisition, on-site calibration, video preprocessing, target recognition and tracking, flow velocity calculation, front-end interaction, and cloud services, and by clearly defining the connections between each module, a fully integrated non-contact river flow measurement process has been achieved. Its technical advantages lie in: eliminating the impact of shooting angle and perspective distortion on flow measurement through video inverse perspective transformation; combining intelligent algorithms with cloud-mobile collaboration; breaking the dependence of traditional contact-based flow measurement on dedicated equipment; and forming a low-cost, fully operational flow measurement solution. This provides integrated technical support for water resource monitoring. Furthermore, the orderly connection between modules ensures a smooth process from video data acquisition to flow velocity result output, improving the overall applicability and convenience of the system.

[0052] The specific process of the homography transformation algorithm used in the video preprocessing module to perform inverse perspective transformation is as follows:

[0053] 1. Establishment of reference object coordinate correspondence: Based on the reference object (such as a fixed ruler) obtained by the on-site calibration module, manually or automatically mark the image pixel coordinates of at least four non-collinear feature points on the reference object in the tilted view image acquired by the video acquisition module. , … (n≥4); at the same time, determine the actual physical coordinates of these feature points based on the actual dimensions of the reference object. , … (Establish a two-dimensional coordinate system with a fixed point in the river channel as the origin, in meters).

[0054] 2. Derivation and Solution of Perspective Matrix: The homography transformation matrix H (3×3) satisfies the mapping relationship between pixel coordinates and physical coordinates: (in" "" indicates matrix multiplication, and all coordinates are homogeneous coordinates. Based on the above n sets of corresponding coordinates, an overdetermined linear equation system is constructed, and the 8 unknown parameters of matrix H are solved by the least squares method. In order to eliminate the influence of image noise and labeling error on matrix accuracy, the RANSAC (Random Sample Consensus) algorithm is further used to remove abnormal corresponding points (the inlier error threshold is set to ≤1 pixel), and finally the optimal perspective matrix H is determined.

[0055] 3. Perspective Matrix Verification and Adjustment: Substitute 1-2 feature points from the reference object that were not involved in the matrix solution into the perspective matrix H, calculate the deviation between the mapped pixel coordinates and the actual marked coordinates. If the average deviation is ≤2 pixels, then matrix H is valid; if the deviation exceeds the threshold, the feature points need to be remarked and the above steps need to be repeated until the accuracy requirements are met.

[0056] Specifically, in this embodiment, the on-site calibration module obtains calibration parameters by using at least one of the following: a fixed ruler set up on-site in the river channel, a buoy with known geometric dimensions, or a preset feature point as a reference object. The image information of the reference object within the field of view of the video acquisition module is collected, and the calibration parameters are calculated by combining the correspondence between the actual size of the reference object and the size in the image. The calibration parameters include perspective transformation parameters and distance conversion parameters.

[0057] By clearly defining the on-site calibration module as using a fixed scale, a buoy of known size, or a preset feature point as a reference to obtain calibration parameters, the technical effect is that the calibration parameters can closely match the actual environment of the river, avoiding calibration deviations caused by scene differences (such as different river widths and terrains). This provides accurate basic data support for subsequent video inverse perspective transformation and flow velocity calculation, thereby reducing flow measurement errors caused by inaccurate calibration, improving the system's adaptability to different river scenes, and ensuring the reliability of flow measurement data in diverse environments.

[0058] Specifically, in this embodiment, when the video preprocessing module performs inverse perspective transformation, it adopts a homography transformation algorithm to calculate the perspective matrix based on the perspective transformation parameters provided by the on-site calibration module. The perspective matrix is ​​used to convert the tilted-view river video data acquired by the video acquisition module into a river video image under a normal viewing angle, thereby eliminating the influence of perspective distortion on the identification of floating object positions.

[0059] By employing a homography transformation algorithm in the video preprocessing module and calculating the perspective matrix based on calibration parameters to transform the orthographic image, the technical effect is as follows: it can specifically eliminate the perspective distortion of river videos caused by tilted mobile phone shooting, restore the tilted view image to the orthographic view, ensure that the position information of floating objects on the water surface in the video truly reflects the actual spatial position, avoid the deviation in the identification of floating object trajectory caused by distortion, provide high-quality, distortion-free image data for the subsequent target recognition and tracking module, and ensure the accuracy of flow measurement-related calculations from the data source.

[0060] Specifically, in this embodiment, the target recognition and tracking module includes a floating object recognition unit and a trajectory tracking unit. The floating object recognition unit employs a deep learning target detection algorithm, which is either the YOLOv5 algorithm or the Mask R-CNN algorithm, to automatically identify floating objects on the water surface from a frontal view video image of the river and output the initial position information of the floating objects. The trajectory tracking unit employs a multi-target tracking algorithm, which is at least one of the DeepSORT algorithm, the Kalman filter algorithm, or the optical flow method, to track the position changes of the floating objects in continuous video frames based on the initial position information of the floating objects and generate motion trajectory data of the floating objects.

[0061] The target recognition and tracking module is divided into a recognition unit and a tracking unit, and is limited to using deep learning detection algorithms (YOLOv5 / MaskR-CNN) and multi-target tracking algorithms (DeepSORT, etc.). The technical benefits are as follows: the deep learning detection algorithm can overcome the interference of complex lighting and cluttered backgrounds in the natural environment on the recognition of floating objects, solve the problems of low efficiency and poor accuracy of traditional recognition methods, and ensure accurate recognition of floating objects; the multi-target tracking algorithm can achieve continuous tracking of floating objects, avoid tracking loss due to the movement or occlusion of floating objects, and obtain the complete trajectory of floating objects. The combination of the two improves the robustness and accuracy of target processing, and provides an accurate trajectory data foundation for flow velocity calculation.

[0062] Specifically, in this embodiment, the calculation process of the flow rate calculation module includes:

[0063] The first step is to extract the floating object's trajectory data output by the target recognition and tracking module, and determine the spatial coordinates of the floating object at different times. The spatial coordinates are converted into actual physical coordinates based on the frontal view image output by the video preprocessing module and the distance conversion parameters of the on-site calibration module.

[0064] The second step is to calculate the displacement of the floating object based on its actual physical coordinates at adjacent moments, and then determine the time interval between adjacent moments by combining the frame rate of the video data.

[0065] The third step is to calculate the velocity of the floating object by the ratio of the displacement to the time interval, and then use the velocity of the floating object as the velocity of the river flow.

[0066] By clarifying the three-step calculation process of the flow velocity calculation module—"converting physical coordinates, calculating displacement and time, and determining velocity"—the technical effect is as follows: by converting the trajectory of the floating object image into actual physical coordinates and combining it with the video frame rate to determine the real time interval, the flow velocity calculation is based on actual physical quantities rather than simply image pixel data. This avoids calculation errors caused by the mismatch between pixel scale and actual scale, ensuring that the calculated water flow velocity can truly reflect the actual water flow conditions in the river channel. From the perspective of calculation logic, this improves the accuracy and reliability of the flow velocity results, meeting the requirements of water resource monitoring for the accuracy of flow velocity data.

[0067] Specifically, in this embodiment, an error analysis module is also included. The error analysis module is connected to the flow rate calculation module and is used to obtain the flow rate calculation results and relevant parameters during the video acquisition and calibration process, and to analyze the sources of flow rate calculation errors. The sources of errors include the shooting angle deviation of the video acquisition module, the calibration accuracy error of the on-site calibration module, and the resolution difference of the video data. The error analysis module is also used to output parameter adjustment suggestions based on the sources of errors in order to optimize the flow rate calculation accuracy.

[0068] By adding an error analysis module, which limits the analysis of error sources (shooting angle, calibration accuracy, resolution) and outputs adjustment suggestions, the technical effect is as follows: it can accurately locate the key error causes in the flow velocity calculation process, avoid blindly optimizing the system, and help users or the system actively reduce errors through targeted parameter adjustment suggestions (such as guiding users to optimize the shooting angle and improve calibration accuracy), thereby achieving continuous optimization of flow velocity calculation accuracy. At the same time, it enhances the system's controllability over the flow measurement process, improves the overall practicality and reliability, and ensures that the system can maintain stable flow measurement accuracy during long-term use.

[0069] Specifically, in this embodiment, the front-end interaction module includes a video interaction unit and a result display unit; the video interaction unit is used to realize the functions of video shooting control, video data preview and uploading video data to the cloud service module on the smartphone; the result display unit is used to display the water flow velocity calculation results in a graphical manner, including at least one of flow velocity numerical display, floating object motion trajectory animation playback and flow velocity change curve drawing at different time periods.

[0070] The front-end interaction module is split into video interaction and result display units, and a graphical display method is specified. The technical effects are as follows: The video interaction unit simplifies video operations on smartphones (shooting, previewing, uploading), lowers the user's operating threshold, and allows users to complete the preliminary operations of flow measurement without professional technical training, thus improving ease of use; The graphical display (flow velocity values, trajectory animations, and change curves) can transform abstract flow measurement data into intuitive visual information, avoiding the abstractness of pure numerical displays, helping users quickly understand the flow measurement results, significantly improving the user experience, and enhancing the system's usability.

[0071] Specifically, in this embodiment, the cloud service module uses the Python programming language combined with a deep learning framework to deploy the core algorithm. The deep learning framework is either the TensorFlow framework or the PyTorch framework, which is used to carry out the inverse perspective transformation calculation of the video preprocessing module, the deep learning model inference of the target recognition and tracking module, and the numerical calculation of the flow rate calculation module. The cloud service module has the ability to process concurrent requests from multiple users and can simultaneously receive video data uploaded by multiple smartphone front-end interaction modules and process them in parallel.

[0072] By limiting the deployment of algorithms to Python and deep learning frameworks (TensorFlow / PyTorch) in the cloud service modules and supporting multi-user concurrent processing, the technical benefits are as follows: the mature deep learning framework can ensure the stability and efficiency of the core algorithms (inverse perspective transformation, object recognition, and flow rate calculation) and avoid compatibility issues during algorithm deployment; the multi-user concurrent processing capability can meet the needs of multiple users to upload videos and obtain flow measurement results at the same time, avoiding system congestion or response delays due to concentrated user use, improving the system's service capacity and response efficiency, and adapting to the needs of large-scale use in multiple scenarios.

[0073] In this embodiment, an adaptive homography transformation equation based on multi-reference weighting is also incorporated to optimize the accuracy of inverse perspective transformation, further improving the accuracy of floating object trajectory recognition. This equation can dynamically adjust the perspective matrix according to the spatial distribution and confidence level of multiple references, thereby enhancing the robustness of the system in complex river environments.

[0074] Specifically, when performing inverse perspective transformation, the video preprocessing module uses this adaptive homography transformation equation to calculate the optimal perspective matrix H. optimized :

[0075] ;

[0076] in:

[0077] Q i The actual physical coordinates of the i-th reference feature point (unit: meters, homogeneous coordinate form: ;

[0078] p i The corresponding image pixel coordinates (homogeneous coordinate form: );

[0079] w i The weight of the i-th feature point is [0,1], and its value range is determined by the confidence of its position in the image and the consistency of the size of the reference object.

[0080] n is the total number of feature points involved in the calculation (n≥4).

[0081] Equation derivation process:

[0082] This equation is essentially a weighted least squares optimization problem, aiming to find the optimal homography matrix H using multiple reference feature points. The derivation is as follows:

[0083] 1. Basic homography transformation model:

[0084] p i =H·Q i

[0085] H is a 3×3 homography matrix used for mapping image coordinates to physical coordinates, satisfying 8 degrees of freedom.

[0086] 2. Introduce weight w i :

[0087] If a feature point is located at the edge of the image or is severely occluded, its weight is reduced.

[0088] If the size of the reference object is known and the measurements in the image are consistent, the weight is increased.

[0089] 3. Construct the optimization objective function:

[0090] ;

[0091] H is solved by minimizing J(H).

[0092] 4. Solve using SVD or Levenberg-Marquardt algorithm to obtain the optimal H. optimized .

[0093] Example: Assume three buoys of known size are deployed as reference points in the river channel, and a total of 12 feature points are extracted. Among them:

[0094] 6 points come from the central buoy (weight w) i =1.0);

[0095] Four points originate from the left-side buoy (weight w is slightly obscured). i =0.7);

[0096] Two points originate from the right-hand buoy (due to image edge distortion, the weight w is adjusted). i =0.5).

[0097] Solve H using the above equation. optimized It is used for inverse perspective transformation to obtain a more accurate front view image.

[0098] Technical effects:

[0099] 1. Improve transformation accuracy: Suppress the influence of noise points and edge distortion points through a weighting mechanism to improve the accuracy of perspective transformation;

[0100] 2. Enhanced robustness: Adaptable to various river scenarios, such as partial occlusion, uneven lighting, and uneven distribution of reference objects;

[0101] 3. Highly adaptable: The weights can be dynamically adjusted according to the actual situation on site without manual intervention;

[0102] 4. System compatibility: As an optimized extension of the existing homography transformation, it does not change the original system architecture.

[0103] Working principle and process:

[0104] 1. Feature point extraction and weight assignment: Extract feature points from multiple reference objects and assign weights based on their image quality;

[0105] 2. Construct the optimization problem: Use weighted least squares to construct the objective function;

[0106] 3. Solving for the optimal matrix: Solving for H using a numerical optimization algorithm. optimized ;

[0107] 4. Apply transformation: transform H optimized It is applied to each frame of video to complete the inverse perspective transformation;

[0108] 5. Verification and Feedback: Check the transformation accuracy by reserving verification points. If the deviation exceeds the limit, recalibrate.

[0109] Example 2

[0110] This invention also provides a smartphone-based non-contact river video flow measurement device, comprising a smartphone body and a cloud server; the smartphone body integrates the video acquisition module and the front-end interaction module of the smartphone-based non-contact river video flow measurement system of Embodiment 1, the smartphone body's camera serves as the hardware carrier of the video acquisition module for acquiring river video data, and the smartphone body's display screen is used to realize the result display function of the front-end interaction module; the cloud server deploys the core algorithms of the on-site calibration module, video preprocessing module, target recognition and tracking module, flow velocity calculation module, and cloud service module of the smartphone-based non-contact river video flow measurement system of any one of claims 1 to 8, and the cloud server establishes a data communication connection with the smartphone body through a wireless network to realize the transmission of video data and the feedback of flow measurement results.

[0111] The device is defined as a "smartphone + cloud server," and the integrated modules, algorithms, and wireless communication connections are limited. Its technical advantages are: utilizing existing smartphone hardware (camera, display) as the front-end carrier eliminates the need for additional dedicated video capture equipment, significantly reducing system operating costs; the cloud server carries the core computing algorithms, avoiding lag caused by limited computing power on the smartphone and ensuring smooth overall system operation; the wireless communication connection enables efficient data transmission, ensuring a smooth process from video upload to result feedback, ultimately achieving convenient deployment and flexible use of non-contact flow measurement, lowering the application threshold for flow measurement technology.

[0112] Specifically, in this embodiment, the front-end interaction module of the smartphone also has a calibration assistance function, which guides the user to take pictures of reference objects on the river site, uploads the reference object images and the actual size information of the reference objects input by the user to the on-site calibration module of the cloud server, assists the on-site calibration module in calculating calibration parameters, and the front-end interaction module supports offline caching of uploaded video data and corresponding flow measurement results, so that the user can view historical flow measurement records in an environment without network.

[0113] Adding a smartphone-based calibration assistance function and an offline caching function to the device has the following technical benefits: The calibration assistance function simplifies on-site calibration operations by guiding users to complete the shooting of reference objects and input of dimensions, reduces the user's cognitive requirements for calibration technology, and ensures accurate acquisition of calibration parameters; The offline caching function allows users to view historical flow measurement data in environments without network access, reducing the system's strong dependence on the network, improving the system's applicability in remote river channels, poor network signal, and other scenarios, further enhancing user convenience and the system's environmental adaptability, and expanding the system's application scope.

[0114] The specific details of training deep learning models are as follows:

[0115] (a) Training parameters of the floating object recognition model (YOLOv5 / MaskR-CNN)

[0116] 1. Training dataset construction:

[0117] The dataset contains 10,000+ images of river scenes, covering different environments: ① lighting conditions (sunny day, cloudy day, dusk, backlight); ② river type (narrow tributary, open river surface, river with debris on the banks); ③ floating object type (foam blocks, leaves, plastic bottles, clumps of aquatic plants, etc., with ≥1000 samples for each type).

[0118] Image annotation was performed using the LabelImg tool (YOLOv5 uses bounding boxes for annotation, with the annotation format being "Category ID center point x / width center point y / height width / image width and height / image height") and the LabelMe tool (MaskR-CNN uses polygonal masks for annotation, with the annotation format being JSON). The annotation accuracy was required to be ≥98%.

[0119] 2. Model training parameters:

[0120] Base model selection: YOLOv5 uses YOLOv5s.pt pre-trained weights (balancing accuracy and speed), and MaskR-CNN uses COCO dataset pre-trained weights;

[0121] Training environment: PyTorch 1.10 framework, GPU is NVIDIA RTX 3090 (single card or multi-card distributed training).

[0122] Key parameters: Batch size = 16, number of iterations = 100, initial learning rate = 0.001 (using cosine annealing learning rate strategy, decaying to 1 / 10 of the current learning rate every 20 epochs), momentum = 0.9, weight decay = 0.0005;

[0123] Data augmentation: During the training phase, random horizontal flipping (probability 0.5), random brightness / contrast adjustment (brightness ±20%, contrast ±15%), image scaling (scale 0.8-1.2 times), and Gaussian noise addition (standard deviation ≤0.01) are used to avoid model overfitting.

[0124] 3. Model Evaluation and Optimization:

[0125] Evaluation metrics: The core metrics are mAP@0.5 (average accuracy at an intersection-over-union threshold of 0.5) and FPS (frames per second). The YOLOv5 model is required to have mAP@0.5≥92% and FPS≥30, while the Mask R-CNN model is required to have mAP@0.5≥90% and FPS≥15.

[0126] Model optimization: Secondary labeling and supplementary training are performed on samples that were incorrectly identified in the validation set (such as small floating objects or floating objects with colors similar to the background). The model size is compressed by pruning algorithms (such as the prune.py script in YOLOv5) to ensure that the cloud inference speed meets the requirements.

[0127] (II) Training parameters of the multi-target tracking model (DeepSORT)

[0128] Feature extraction network: ResNet-50 was used as the appearance feature extractor. It was pre-trained on the Market-1501 pedestrian dataset and then fine-tuned using 2000+ segments of river floating object tracking videos (each segment 10-30 seconds).

[0129] Tracking parameters: maximum association distance (IoU threshold) = 0.3, maximum age (number of frames to retain trajectory when no target is detected) = 30, initial trajectory confidence threshold = 0.7, to ensure that the trajectory can still be re-associated after a brief obstruction by a floating object.

[0130] III. The specific quantitative methods for error analysis are as follows:

[0131] (a) Error Quantization Model

[0132] 1. Flow velocity error caused by shooting angle deviation:

[0133] Establish the relative error between the shooting angle θ (the angle between the optical axis of the mobile phone camera and the horizontal plane of the river channel, ranging from 30° to 90°) and the flow velocity calculation. Quantification relationship: =0.02×(90°- (Derived from fitted experimental data). For example, When = 60°, =0.6%; When = 30°, =1.2%, indicating that the greater the angular deviation, the greater the flow velocity error.

[0134] 2. Flow rate error caused by on-site calibration accuracy:

[0135] Calibration accuracy is defined as the relative deviation between the actual size L of the reference object and the measured size L' in the image. "Standard" measurement Scale = |L - L'| / L; Relative error in flow rate calculation and The relationship between the targets is =1.5× (Calibration errors will be transmitted to the physical coordinate transformation process). For example... When the standard is 0.5%, =0.75%; When the target is 1%, =1.5%.

[0136] 3. Streaming speed error caused by video resolution:

[0137] Setting the correspondence between resolution level and error: ① 1080P (1920×1080): relative error of flow rate ≤0.5%; ②720P (1280×720): ≤1%; ③480P (854×480): ≤2%, quantifying the impact of different resolutions on measurement accuracy.

[0138] (II) Specific basis for error adjustment recommendations

[0139] when When the angle is greater than 1% (i.e., θ < 60°), the front-end interactive module will automatically pop up a prompt: "The shooting angle is too low. It is recommended to adjust the height or position of the phone to make the shooting angle ≥ 60° (you can use the phone's level indicator to help judge)."

[0140] when The target is >1% (i.e.) When the accuracy is greater than 1.5%, the system will prompt: "Insufficient calibration accuracy. It is recommended to select a new reference object (preferably a fixed ruler with a length of ≥2 meters) to ensure that the reference object image occupies ≥1 / 5 of the screen and that there is no obstruction during shooting."

[0141] When the video resolution is ≤480P (i.e.) When the error rate is greater than 2%, the system will display the message: "The video resolution is too low. It is recommended to enable 1080P resolution shooting in the phone's camera settings to reduce measurement errors."

[0142] IV. The specific cloud deployment plan is as follows:

[0143] (a) Cloud server hardware configuration

[0144] Basic configuration: CPU is Intel Xeon Gold 6348 (28 cores and 56 threads), GPU is NVIDIA A100 (40GB VRAM, supports FP16 half-precision inference), memory is 128GB DDR4 (2933MHz), and storage is 1TB SSD (system disk) + 10TB HDD (data storage disk, used to cache user-uploaded videos and streaming results).

[0145] Extended configuration: Supports multi-node cluster deployment. For each additional node (with the same configuration as the base node), the concurrent processing capacity is increased by 50%, meeting the needs of large-scale monitoring.

[0146] (II) Software and Protocol Configuration

[0147] 1. Core Software Stack:

[0148] Operating system: Ubuntu Server 20.04 LTS;

[0149] Deep learning frameworks: PyTorch 1.10 (for YOLOv5 / MaskR-CNN / DeepSORT inference), OpenCV 4.5 (for image preprocessing and perspective transformation calculation);

[0150] Web services: Backend interfaces are built using the FastAPI framework, supporting RESTful API calls, with interface response latency ≤100ms;

[0151] Database: MySQL 8.0 (stores user information and flow test records), Redis 6.2 (caches real-time task queues and high-frequency access calibration parameters, with a cache validity period of 1 hour).

[0152] 2. Data transmission protocol:

[0153] Video data upload: Uses WebSocket protocol (based on TCP), supports breakpoint resume (transmits in 5MB chunks, automatically retransmits when a chunk is lost), 1080P video (30 seconds, about 150MB) upload time ≤30 seconds (when network bandwidth ≥10Mbps);

[0154] Flow measurement results feedback: in JSON format, including "flow velocity value (unit: m / s), measurement time, list of floating object trajectory coordinates, and error analysis report", data size ≤10KB, feedback delay ≤5 seconds (after video processing).

[0155] (III) Concurrency Processing and Task Scheduling Scheme

[0156] Load balancing: Nginx server is used as a reverse proxy, and a weighted round-robin strategy is configured (tasks are allocated according to the GPU utilization of each node, nodes with utilization <60% have a weight of 1, nodes with utilization 60%-80% have a weight of 0.5, and nodes with utilization >80% are not assigned tasks) to avoid single node overload.

[0157] Task scheduling: Celery distributed task queue is used to split "video preprocessing, target recognition and tracking, and flow rate calculation" into 3 independent tasks, supporting parallel execution of tasks (such as video preprocessing and target recognition can process videos from different users at the same time); Task priority setting: real-time flow measurement tasks (initiated by users) have high priority, and historical video recalculation tasks (triggered automatically by the system) have low priority, ensuring that users' real-time needs are responded to first.

[0158] Performance metrics: A single node supports concurrent processing of 20 channels of 1080P video (processing time for each channel ≤ 30 seconds), and a cluster (3 nodes) supports concurrent processing of 60 channels of 1080P video, meeting the needs of simultaneous flow measurement at multiple monitoring points.

[0159] This embodiment also provides multi-scenario comparative experiments, using a comparison scheme of "this system vs. traditional ADCP (Acoustic Doppler Current Profiler)" to verify the technical effect:

[0160] 3.1 Basic Information for Experimental Design

[0161] Experimental locations: Three types of typical river channels were selected (mountain tributaries, plain rivers, and urban landscape rivers) to cover complex water conditions;

[0162] Comparison device: ADCP (model: SonTekFlowTracker2, accuracy: ±1% of measured value), as a true value reference;

[0163] This system is configured with: a smartphone (model: Xiaomi 13, camera resolution 1080P, frame rate 30fps) and a cloud server;

[0164] Experimental index: Relative error of flow velocity measurement ( ), floating object tracking success rate, and data output delay.

[0165] 3.2 Experimental Data and Results under Complex Hydrological Conditions

[0166] 3.2.1 Scenario 1: Flood season (high flow velocity, dense floating debris)

[0167] Experimental conditions: Mountain tributary (river width 15m, water depth 2.5m), flood season flow velocity 1.2~2.0m / s, floating debris type (twigs, plastic bottles, weed clumps), density approximately 5 pieces / m²;

[0168] Experimental procedure:

[0169] On-site calibration: Using concrete scale stakes (3m in length) on the riverbank as reference objects, internal and external parameters were calibrated (calibration accuracy: distance conversion error 0.3%).

[0170] Data acquisition: Synchronously acquire video (duration 60s) and ADCP flow rate data of this system (sampling interval 1s);

[0171] Noise handling: Enable the "rain and fog removal + reflection suppression" algorithm (during flood season accompanied by short-term heavy rain);

[0172] Experimental results:

[0173]

[0174] Conclusion: The relative error of flow velocity during flood season is ≤2.5%, which meets the accuracy requirements for water resources monitoring (industry allowable error ≤5%).

[0175] 3.2.2 Scenario 2: Turbulent Zone (turbulent water flow, numerous eddies)

[0176] Experimental conditions: a bend in a plain river channel (30m wide, 1.8m deep), turbulent flow velocity of 0.5-1.0m / s, with local eddies (1-3m in diameter).

[0177] Experimental procedure:

[0178] On-site calibration: A buoy of known size (0.5m in diameter) was used as a reference. Since there was no fixed scale on the curve, the calibration accuracy was improved by using three buoys for triangular positioning.

[0179] Data acquisition: Synchronously acquire video (duration 120s) and ADCP flow rate data of this system (sampling interval 1s);

[0180] Target tracking: Enable DeepSORT multi-target tracking algorithm, set "maximum association distance 0.4" and "maximum age 40 frames" to avoid tracking loss caused by vortexes;

[0181] Experimental results:

[0182]

[0183] Conclusion: The relative error of flow velocity in the turbulent region is ≤3.0%, and the tracking success rate is ≥86%, overcoming the problem of trajectory loss caused by water flow turbulence.

[0184] 3.2.3 Scenario 3: Low visibility (turbid water + nighttime)

[0185] Experimental conditions: Urban landscape river (8m wide, 1.2m deep), turbid water (0.3m transparency), nighttime lighting (streetlights + mobile phone fill light), flow velocity 0.3~0.6m / s;

[0186] Experimental procedure:

[0187] On-site calibration: A fixed ruler (2m in length) is used with reflective stickers (to improve nighttime visibility). The internal parameter calibration includes "nighttime white balance calibration".

[0188] Noise processing: The "illuminance normalization + small target enhancement" algorithm is enabled, and the YOLOv5 model is loaded with nighttime floating object-specific training weights (the dataset contains 500 nighttime images).

[0189] Data acquisition: Synchronously acquire video (90s duration) and ADCP flow rate data of this system (sampling interval 1s);

[0190] Experimental results:

[0191]

[0192] Conclusion: The relative error of flow velocity in low visibility scenarios is ≤3.2%, which meets the monitoring needs of nighttime / turbid water bodies and expands the scope of scenario adaptability.

[0193] Experimental summary: The relative error of the flow velocity in this system during flood season, turbulent areas, and low visibility scenarios is ≤3.2%, which is significantly better than the problem of "traditional mobile phone video flow measurement error ≥10%" in the background technology of the original document;

[0194] The floating object tracking success rate is ≥86%, and the noise processing algorithm effectively overcomes interference from lighting, reflection, and debris, supporting the technical effect of "high robustness".

[0195] The data output delay is ≤4.8s, which meets the "real-time monitoring" requirement and does not require professional equipment, thus meeting the design goals of "low cost and easy operation".

[0196] Working principle

[0197] The working principle of this system and device revolves around the entire process of "data acquisition - preprocessing - intelligent analysis - calculation output - feedback display," achieving accurate flow measurement through module collaboration. Specifically, it consists of five main stages:

[0198] 1. On-site calibration and video acquisition: The on-site calibration module uses fixed rulers, buoys of known size, or preset feature points in the river as reference objects. It generates perspective transformation parameters and distance conversion parameters by combining the correspondence between the actual size of the reference objects and the image size. At the same time, the smartphone camera (video acquisition module) collects video data of the river. The front-end interaction module synchronously uploads the video data and calibration parameters to the cloud service module.

[0199] 2. Video Preprocessing (Inverse Perspective Transformation): The cloud service module calls the video preprocessing module, which uses a homography transformation algorithm to calculate the perspective matrix based on the on-site calibration parameters. This converts the tilted river video into a normal-view image, eliminating perspective distortion caused by the shooting angle and ensuring that the position of floating objects in the video is consistent with their actual spatial position, thus providing unbiased image data for subsequent recognition and tracking.

[0200] 3. Floating Object Recognition and Trajectory Tracking: The target recognition and tracking module (including the recognition unit and the tracking unit) processes the frontal image: The recognition unit uses YOLOv5 or MaskR-CNN deep learning algorithms to accurately identify floating objects on the water surface and output their initial positions under complex backgrounds and lighting conditions; The tracking unit uses DeepSORT, Kalman filtering, or optical flow to track the positional changes of floating objects in continuous video frames and generate complete motion trajectory data.

[0201] 4. Flow velocity calculation and error analysis: The flow velocity calculation module first converts the image coordinates of the floating object's trajectory into actual physical coordinates by combining them with the distance conversion parameters calibrated on-site; then, it calculates the displacement based on the physical coordinates of adjacent moments, determines the time interval by combining the video frame rate, and calculates the water flow velocity by "displacement / time interval"; at the same time, the error analysis module obtains the flow velocity results and flow measurement process parameters (such as shooting angle and calibration accuracy), locates the source of error (such as angle deviation and resolution difference), and outputs adjustment suggestions to optimize the calculation accuracy.

[0202] 5. Cloud-Mobile Data Interaction: The cloud service module feeds back the flow rate results (including numerical values ​​and trajectory data) to the smartphone front-end interaction module. The front-end displays the results graphically (numerical display, trajectory animation, flow rate change curve) on the screen. In offline scenarios, the front-end interaction module can cache historical flow measurement data to ensure that it can still be viewed when there is no network.

[0203] How to use

[0204] This device (smartphone + cloud server) relies on the smartphone's front-end interaction module. The operation is simple and requires no professional training. The specific steps are as follows:

[0205] 1. On-site calibration operation: The user selects a reference object (fixed scale, buoy of known size, etc.) on the river, opens the flow measurement applet on the smartphone, takes an image of the reference object through the "calibration auxiliary function", enters the actual size of the reference object (such as the length of the scale) in the interface, completes the calibration parameter acquisition, and the system automatically uploads the parameters to the cloud.

[0206] 2. River Video Shooting and Uploading: Users can control their mobile phone camera to shoot river videos through the "Video Interaction Unit" in the mini-program (no strict horizontal angle required). After shooting, users can preview the video and click "Upload" to transfer the video data to the cloud server.

[0207] 3. Automatic cloud processing (no user intervention required): After receiving the video and calibration parameters, the cloud server performs inverse perspective transformation, floating object recognition and tracking, flow velocity calculation and error analysis in sequence. The entire process is completed automatically by the cloud algorithm without user operation.

[0208] 4. Viewing and Optimizing Flow Measurement Results: The cloud will feed back the processed flow velocity results to the mini-program, which users can view in the "Results Display Unit": including real-time flow velocity values, animations of floating object trajectories, and flow velocity change curves for different time periods if multiple measurements are taken; if the system outputs error adjustment suggestions (such as "optimize shooting angle"), users can recalibrate or take pictures according to the suggestions to further improve the flow measurement accuracy; when there is no network, users can view historical flow measurement records through the "Offline Cache" function.

[0209] The above are merely preferred embodiments of the present invention and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should recognize that any equivalent substitutions and obvious changes made based on the description and illustrations of the present invention should be included within the protection scope of the present invention.

Claims

1. A non-contact river video flow measurement system based on a smart phone, characterized in that, The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device.

2. The smartphone-based non-contact river video flow measurement system of claim 1, wherein, The application relates to a river flow speed calculation method and device.

3. The smartphone-based non-contact river video flow measurement system of claim 1, wherein, The application relates to a river flow speed calculation method and device.

4. The smartphone-based non-contact river video flow measurement system of claim 1, wherein, The application relates to a river flow speed calculation method and device.

5. The smartphone-based non-contact river video flow measurement system of claim 1, wherein, The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow speed calculation method and device. The application relates to a river flow The first step is to extract the floating object's motion trajectory data output by the target recognition and tracking module, and determine the spatial coordinates of the floating object at different times. The spatial coordinates are converted into actual physical coordinates based on the frontal view image output by the video preprocessing module and the distance conversion parameters of the on-site calibration module. The second step is to calculate the displacement of the floating object based on its actual physical coordinates at adjacent moments, and then determine the time interval between adjacent moments by combining the frame rate of the video data. The third step is to calculate the velocity of the floating object by the ratio of the displacement to the time interval, and then use the velocity of the floating object as the velocity of the river flow.

6. The smartphone-based non-contact river video flow measurement system of claim 1, wherein, It also includes an error analysis module, which is connected to the flow rate calculation module. The error analysis module is used to obtain the flow rate calculation results and relevant parameters during the video acquisition and calibration process, and to analyze the sources of flow rate calculation errors. The sources of errors include the shooting angle deviation of the video acquisition module, the calibration accuracy error of the on-site calibration module, and the resolution difference of the video data. The error analysis module is also used to output parameter adjustment suggestions based on the sources of errors in order to optimize the flow rate calculation accuracy.

7. The smartphone-based non-contact river video flow measurement system of claim 1, wherein, The front-end interaction module includes a video interaction unit and a result display unit; the video interaction unit is used to realize the functions of video shooting control, video data preview and uploading video data to the cloud service module on the smartphone; the result display unit is used to display the water flow velocity calculation results in a graphical manner, the graphical manner including at least one of flow velocity numerical display, floating object motion trajectory animation playback and flow velocity change curve plotting at different time periods.

8. The smartphone-based non-contact river video flow measurement system of claim 1, wherein, The cloud service module uses the Python programming language combined with a deep learning framework to deploy the core algorithm. The deep learning framework is either TensorFlow or PyTorch, which is used to carry out the inverse perspective transformation calculation of the video preprocessing module, the deep learning model inference of the target recognition and tracking module, and the numerical calculation of the flow rate calculation module. The cloud service module has the ability to process concurrent requests from multiple users and can simultaneously receive and process video data uploaded by multiple smartphone front-end interaction modules.

9. A non-contact river video flow measuring device based on a smart phone, characterized in that, The system includes a smartphone and a cloud server. The smartphone integrates a video acquisition module and a front-end interaction module of the smartphone-based non-contact river video flow measurement system according to any one of claims 1 to 8. The smartphone's camera serves as the hardware carrier for the video acquisition module, used to acquire river video data. The smartphone's display screen is used to display the results of the front-end interaction module. The cloud server deploys the core algorithms of the smartphone-based non-contact river video flow measurement system according to any one of claims 1 to 8, including a field calibration module, a video preprocessing module, a target recognition and tracking module, a flow velocity calculation module, and a cloud service module. The cloud server establishes a data communication connection with the smartphone via a wireless network to achieve video data transmission and flow measurement result feedback. 10.The smart phone based non-contact river video flow measurement device according to claim 9, wherein, The front-end interaction module of the smartphone also has a calibration assistance function, which guides the user to take pictures of reference objects on the river site, uploads the reference object images and the actual size information of the reference objects input by the user to the on-site calibration module of the cloud server, assists the on-site calibration module in calculating calibration parameters, and the front-end interaction module supports offline caching of uploaded video data and corresponding flow measurement results, so that users can view historical flow measurement records in the absence of network environment.

Images