A flow measurement method and system for smart water conservancy monitoring and sensing equipment

Abstract

This invention provides a flow measurement method and system for intelligent water conservancy monitoring and sensing equipment, belonging to the field of water conservancy monitoring technology. The method includes: establishing a camera calibration service and using an improved Unet network to train and identify water flow velocity; calling the camera calibration service interface at the edge device to obtain camera calibration parameters; performing camera correction and flow direction correction on the coordinates of a point set in a specified observation area; performing fine-grained cropping of the observation area and calculating the coordinates of multiple ROIs to be cropped based on the corrected observation area; using the undistort function built into the OpenCV library to perform camera correction on each frame of image, and simultaneously using warpAffine to rotate the corrected image so that the flow direction is upward; for each ROI, extracting 32 consecutive frames of images and inputting them into the model for inference, obtaining the maximum flow velocity value and the flow field distribution array, and multiplying them to obtain the dense flow velocity distribution information for a single inference; integrating, mapping, and outputting the results. This achieves efficient, accurate, and real-time monitoring of water flow.

Description

Technical Field

[0001] This invention relates to the field of water conservancy monitoring technology, and in particular to a flow measurement method and system for intelligent water conservancy monitoring and sensing equipment. Background Technology

[0002] In the water conservancy industry, accurate acquisition of water flow data plays a crucial role in water resource management, flood control and drought relief, and hydropower dispatch. Traditional flow measurement methods mainly include manual observation, mechanical current meter measurement, and ADCP (Acoustic Doppler Current Profiler) measurement.

[0003] Manual observation is inefficient and highly susceptible to human factors, making it difficult to guarantee measurement accuracy. Mechanical current meter measurements require on-site operation by staff, which is not only labor-intensive but also poses safety hazards in some complex or dangerous aquatic environments. Although ADCP has high measurement accuracy, the equipment is expensive, maintenance is complex, and it has certain requirements for water quality and flow conditions, which limits its large-scale promotion and application.

[0004] With the rapid development of technologies such as the Internet of Things, sensor technology, and big data analytics, smart water conservancy monitoring and sensing equipment has emerged. However, current flow measurement systems and methods based on these devices still have shortcomings in terms of accuracy, stability, and real-time performance, failing to meet the high requirements of modern water conservancy management for flow monitoring. Therefore, developing an efficient, accurate, and reliable flow measurement system and method for smart water conservancy monitoring and sensing equipment is of significant practical importance. Summary of the Invention

[0005] This invention provides a flow measurement method and system for intelligent water conservancy monitoring and sensing equipment, which solves the problems of high cost, complex operation, insufficient measurement accuracy and real-time performance of existing flow measurement technologies, and realizes efficient, accurate and real-time monitoring of water flow.

[0006] This invention provides a flow measurement method for intelligent water conservancy monitoring and sensing equipment, comprising:

[0007] Step 1: Build a camera calibration service. Deploy the camera calibration service on a server accessible to edge sensing devices to provide camera calibration services for smart water conservancy monitoring and sensing devices.

[0008] Step 2: Model training, using an improved Unet network to train and identify water flow velocity;

[0009] Step 3: Call the camera calibration service. On the edge device, call the camera calibration service HTTP interface to obtain the camera calibration parameters, including the camera intrinsic matrix camera_matrix, distortion parameter dist_coeffs, and pixel mapping ratio ppm;

[0010] Step 4: Perform camera calibration on the observation area. Based on the camera calibration parameters obtained in Step 3, use the undistortPoints function provided by the OpenCV repository to perform camera calibration on the coordinates of the point set in the specified observation area, and perform extreme value correction on the calibrated coordinate values.

[0011] Step 5: Correct the flow direction of the observation area by manually specifying the water flow direction and rotating the corrected image of the camera so that the flow direction is upward.

[0012] Step 6: Perform fine-grained cropping of the observation area, and calculate the coordinates of multiple ROIs to be cropped based on the observation area after camera correction and flow direction correction;

[0013] Step 7: Perform camera correction and flow direction correction on the input image. Based on the camera calibration parameters obtained in Step 3, use the undistort function built into the OpenCV library to perform camera correction on each frame of the image. Simultaneously, use warpAffine to rotate the corrected image so that the flow direction is upward.

[0014] Step 8: Traverse the ROI set obtained in Step 6, and for each ROI, extract 32 consecutive frames of images to input into the model for inference. Obtain the maximum flow velocity value and the flow field distribution array, and multiply them to obtain the dense flow velocity distribution information for a single inference.

[0015] Step 9: Integrate the results of the reasoning and map them back to the original image for output.

[0016] Preferably, the Unet network receives input from 32 consecutive frame images, uses feature fusion to overlap and merge the channels of adjacent frames to learn the temporal information of the video frames in the time dimension, outputs the first supervision information, namely the maximum flow velocity value, through channel dimensionality reduction after downsampling, gradually restores the spatial dimension of the data through channel skip connections and upsampling, and outputs a 18080 flow field distribution array as the second supervision information after upsampling.

[0017] Preferably, the calculation of multiple ROI coordinates to be clipped includes:

[0018] Based on the observed region, the start and end points clips to be clipped are obtained, including clip_l, clip_r, and clip_y, which represent the horizontal start point, horizontal end point, and vertical center point coordinates of the region to be clipped, respectively. model_size represents the model input size, i.e., the clipping size. The coordinates of each ROI region are calculated as follows:

[0019]

[0020] in, Indicates the first There are ROI coordinates, where l, t, r, and b represent the left, top, right, and bottom coordinates of the ROI, respectively.

[0021] Preferably, the camera calibration service built in step 1 enables edge devices to directly call the service interface to obtain camera calibration parameters.

[0022] Preferably, when correcting the flow direction of the observation area in step 5, the water flow direction is upward by default.

[0023] Preferably, in step 6, multiple ROIs are cropped in the observation area to adapt to the size input requirements of the model.

[0024] Preferably, before performing fine-grained cropping of the observation area, the following steps are included:

[0025] Extract the edge lines of the camera-corrected image, cut the edge lines into several sub-lines based on the image inflection points, and fit each sub-line.

[0026] Obtain the set of connecting lines for each sub-line, and determine the auxiliary fitting function of each connecting line in the set and the main fitting function of the corresponding sub-line;

[0027] Based on the first intersection point of each connecting line in the connecting line set with the corresponding sub-line, the first deviation of the corresponding auxiliary fitting function based on the first intersection point, and the second deviation of the corresponding main fitting function based on the first intersection point, a deviation sequence of the corresponding sub-line is constructed.

[0028] The deviation sequence is input into the sequence analysis model to obtain the auxiliary complexity coefficients of the corresponding sub-lines. At the same time, the main complexity coefficients are determined based on the edge pixel distribution of the corresponding sub-lines. ,in, The number of pixels at the edge of the sub-line; The area of ​​the region containing the sub-line; These are the weighting coefficients; It is the gradient direction The probability of occurrence; The number of intervals quantized for direction;

[0029] The global flow complexity coefficient is obtained by analyzing all auxiliary complexity coefficients and all principal complexity coefficients in the camera-corrected image. ,in, This represents the total number of all sub-lines; The main complexity coefficient of the j-th sub-line; Let j be the distance from the j-th sub-line to the center point of the image; This is the average distance of all sub-lines from the center point of the image; This is the adjustment coefficient;

[0030] Determine the number of lines in each set of connecting lines, and lock the sub-line corresponding to the largest number of lines as the first line. At the same time, select the sub-line with the largest coefficient from all major complexity coefficients and regard it as the second line.

[0031] The search range is defined by all sub-lines in the image after camera correction, with the first line as the initial search point and the second line as the end search point. The first closed search boundary is obtained according to the shortest distance principle, and the first local water flow complexity coefficient is determined. At the same time, the second closed search boundary is obtained according to the longest distance principle, and the second local water flow complexity coefficient is determined.

[0032] Connect the center points of the first line and the second line to obtain a connecting line, and determine the number of other lines that pass through the connecting line to obtain the third local water flow complexity coefficient.

[0033] The final trimming coefficient is determined based on the global flow complexity coefficient, the first local flow complexity coefficient, the second local flow complexity coefficient, and the third local flow complexity coefficient.

[0034] Fine-grained cropping is performed on the camera-corrected image by matching the cropping granularity with the global flow complexity coefficient and the final cropping coefficient from the dual-coefficient-granularity lookup table.

[0035] This invention provides a flow measurement system for smart water conservancy monitoring and sensing equipment, comprising:

[0036] The service setup module is used to set up camera calibration services, deploy camera calibration services on servers accessible to edge sensing devices, and provide camera calibration services for smart water conservancy monitoring and sensing devices.

[0037] The training module is used for model training and employs an improved Unet network to train the recognition of water flow velocity.

[0038] The parameter acquisition module is used to call the camera calibration service. On the edge device, the camera calibration service HTTP interface is called to obtain camera calibration parameters, including the camera intrinsic matrix camera_matrix, distortion parameter dist_coeffs and pixel mapping ratio ppm;

[0039] The camera calibration module is used to perform camera calibration on the observation area. Based on the obtained camera calibration parameters, it uses the undistortPoints function from the OpenCV repository to perform camera calibration on the coordinates of the point set in the specified observation area, and performs extreme value correction on the calibrated coordinate values.

[0040] The flow direction correction module is used to correct the flow direction in the observation area. The water flow direction can be manually specified and the corrected image from the camera can be rotated so that the flow direction is upward.

[0041] The fine-grained cropping module is used to perform fine-grained cropping of the observation area, and calculates the coordinates of multiple ROIs to be cropped based on the observation area after camera correction and flow direction correction;

[0042] The synchronization module is used to perform camera correction and flow direction correction on the input image. Based on the obtained camera calibration parameters, it uses the undistort function built into the OpenCV library to perform camera correction on each frame of the image, and synchronously uses warpAffine to rotate the corrected image so that the flow direction is upward.

[0043] The distribution information inference module is used to traverse the obtained ROI set. For each ROI, 32 consecutive frames of images are extracted and input into the model for inference. The maximum flow velocity value is obtained and multiplied with the flow field distribution array to obtain the dense flow velocity distribution information of a single inference.

[0044] The integration module is used to integrate the results of the inference and map them back to the original graph for output.

[0045] Compared with the prior art, the beneficial effects of this application are as follows:

[0046] For practical flow measurement scenarios in straight channels, this paper proposes a flow measurement model that discards flow direction information and focuses solely on monitoring flow velocity information. Furthermore, the flow velocity information is decomposed into two types: maximum flow velocity and normalized flow field distribution, accelerating model convergence and increasing model accuracy. Camera calibration is integrated into a central server, which allows for unified management of camera calibration data and visualization of calibration results, ensuring accuracy. Conversely, if camera calibration is performed at the edge, recalibration is required when camera parameters change (e.g., lens replacement, camera position adjustment), further increasing maintenance costs and workload. For models based on fixed camera calibration and flow direction, an adapted camera calibration and flow direction correction solution is proposed to ensure the accuracy and effectiveness of the input image. The fixed input size issue is addressed through multi-ROI cropping and separate inference. The resulting flow measurement algorithm is more adaptable to solving real-world problems, and the inference results can be converted into actual flow velocity values.

[0047] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0048] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0049] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0050] Figure 1 This is a flowchart of a flow measurement method for a smart water conservancy monitoring and sensing device according to an embodiment of the present invention;

[0051] Figure 2 This is a network structure diagram of the flow velocity identification model based on the improved Unet in an embodiment of the present invention;

[0052] Figure 3 This is a structural diagram of a flow measurement system for a smart water conservancy monitoring and sensing device according to an embodiment of the present invention;

[0053] Figure 4 The results are visualized in this embodiment of the invention. Detailed Implementation

[0054] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0055] This invention provides a flow measurement method for intelligent water conservancy monitoring and sensing equipment, such as... Figure 1 As shown, it includes:

[0056] Step 1: Build a camera calibration service. Deploy the camera calibration service on a server accessible to edge sensing devices to provide camera calibration services for smart water conservancy monitoring and sensing devices.

[0057] Step 2: Model training, using an improved Unet network to train and identify water flow velocity;

[0058] Step 3: Call the camera calibration service. On the edge device, call the camera calibration service HTTP interface to obtain the camera calibration parameters, including the camera intrinsic matrix camera_matrix, distortion parameter dist_coeffs, and pixel mapping ratio ppm;

[0059] Step 4: Perform camera calibration on the observation area. Based on the camera calibration parameters obtained in Step 3, use the undistortPoints function provided by the OpenCV repository to perform camera calibration on the coordinates of the point set in the specified observation area, and perform extreme value correction on the calibrated coordinate values.

[0060] Step 5: Correct the flow direction of the observation area by manually specifying the water flow direction and rotating the corrected image of the camera so that the flow direction is upward.

[0061] Step 6: Perform fine-grained cropping of the observation area, and calculate the coordinates of multiple ROIs to be cropped based on the observation area after camera correction and flow direction correction;

[0062] Step 7: Perform camera correction and flow direction correction on the input image. Based on the camera calibration parameters obtained in Step 3, use the undistort function built into the OpenCV library to perform camera correction on each frame of the image. Simultaneously, use warpAffine to rotate the corrected image so that the flow direction is upward.

[0063] Step 8: Traverse the ROI set obtained in Step 6, and for each ROI, extract 32 consecutive frames of images to input into the model for inference. Obtain the maximum flow velocity value and the flow field distribution array, and multiply them to obtain the dense flow velocity distribution information for a single inference.

[0064] Step 9: Integrate the results of the reasoning and map them back to the original image for output, such as... Figure 4 As shown.

[0065] Preferably, the Unet network receives input from 32 consecutive frame images, uses feature fusion to overlap and merge the channels of adjacent frames to learn the temporal information of the video frames in the time dimension, outputs the first supervision information, namely the maximum flow velocity value, through channel dimensionality reduction after downsampling, gradually restores the spatial dimension of the data through channel skip connections and upsampling, and outputs a 18080 flow field distribution array as the second supervision information after upsampling.

[0066] Preferably, the calculation of multiple ROI coordinates to be clipped includes:

[0067] Based on the observed region, the start and end points clips to be clipped are obtained, including clip_l, clip_r, and clip_y, which represent the horizontal start point, horizontal end point, and vertical center point coordinates of the region to be clipped, respectively. model_size represents the model input size, i.e., the clipping size. The coordinates of each ROI region are calculated as follows:

[0068]

[0069] in, Indicates the first There are ROI coordinates, where l, t, r, and b represent the left, top, right, and bottom coordinates of the ROI, respectively.

[0070] Preferably, the camera calibration service built in step 1 enables edge devices to directly call the service interface to obtain camera calibration parameters.

[0071] Preferably, when correcting the flow direction of the observation area in step 5, the water flow direction is upward by default.

[0072] Preferably, in step 6, multiple ROIs are cropped in the observation area to adapt to the size input requirements of the model.

[0073] In this embodiment, the network structure diagram of the present invention is as follows: Figure 2 As shown, the network receives 32 consecutive frames of images as input. It uses feature fusion to overlap and merge the channels of adjacent frames, thereby learning the temporal information of the video frames in the time dimension. After downsampling, channel dimensionality reduction is used to output the network's first supervisory information—the maximum flow velocity value. The spatial dimension of the data is gradually restored through channel skip connections and upsampling. After upsampling, the network outputs a 1*80*80 flow field distribution array as the model's second supervisory information. This model introduces two supervisory information components, enabling the improved Unet network to better guide training, accelerate convergence, and improve segmentation capabilities.

[0074] In this embodiment, the camera calibration service is a set of programs or systems built on the server to provide camera parameter calibration functions for smart water conservancy monitoring and sensing devices (such as camera devices deployed beside rivers to monitor water flow).

[0075] Edge sensing devices refer to devices located at the edge of a water monitoring scenario, used to collect water flow-related data (such as capturing images of water flow). Examples include smart cameras installed on riverbanks. Camera calibration software is installed on a server, and network services are configured so that the edge sensing devices can access the camera calibration service on the server via a network (such as a local area network). For instance, a simple web service can be built using Python's Flask framework, allowing the edge devices to obtain calibration parameters by sending HTTP requests.

[0076] Water flow velocity refers to the distance water travels per unit of time. For example, if the flow velocity of a river is 2 meters per second, it means the water travels 2 meters per second. A large amount of image data containing annotations of water flow velocity is prepared and input into an improved Unet network for training. For instance, water flow images of different rivers under different flow conditions are collected, and the flow velocity in different regions of the images is manually annotated. Then, a deep learning framework (such as PyTorch) is used to build and train the improved Unet network.

[0077] The HTTP interface is an application programming interface based on the HTTP protocol. Edge devices communicate with the camera calibration service on the server through this interface to obtain the required data.

[0078] Distortion parameters are used to describe the distortion of a camera lens, including parameters such as radial distortion and tangential distortion, in order to correct image distortion caused by lens distortion.

[0079] The observation area refers to the specific area where water flow monitoring is required, such as a cross-sectional area of ​​a certain section of a river.

[0080] The `undistortPoints` function, included in the OpenCV repository, is a function in the OpenCV library used to correct distortion of image point coordinates. Based on the camera intrinsic matrix and distortion parameters, it corrects the distorted point coordinates to distort-free coordinates.

[0081] Extreme value correction is necessary because after camera correction, coordinates may exceed the image range. Therefore, it is necessary to check the corrected coordinate values ​​and adjust the coordinates that exceed the image range to the image boundary to ensure that the corrected points are within the image range. Specifically, the coordinates of the point set in the observation area are determined (for example, the coordinates of several vertices of the observation area are marked on the image). These coordinates, the camera intrinsic matrix, and the distortion parameters are input into the undistortPoints function for correction. Then, each corrected coordinate is traversed to determine whether it is within the width and height range of the image. If it exceeds the range, it is set as the coordinate value of the image boundary.

[0082] Flow direction correction is necessary because the model is trained with the water flow direction assumed to be upwards. Therefore, to ensure the model accurately identifies the flow velocity, the image of the observed area after camera correction needs to be rotated so that the water flow direction aligns with the direction used during model training (i.e., upwards). Specifically, the actual water flow direction is manually determined (by observing the movement of floating objects in the water flow, etc.), and then an image rotation algorithm (such as the `warpAffine` function in OpenCV) is used to rotate the camera-corrected image according to the determined rotation angle, ensuring the water flow direction is upwards. For example, if the actual water flow direction is 30 degrees to the left of the bottom, after calculating the rotation angle, the `warpAffine` function is used to rotate the image.

[0083] Fine-grained cropping refers to precisely cropping the observed area into multiple small regions of interest (ROIs) suitable for the model input size based on the actual water flow conditions. An ROI is a region in an image that requires focused attention and processing; here, it refers to the cropped image region used as input to the model for flow velocity inference. First, the horizontal start point, horizontal end point, and vertical center point of the observed area are determined. Then, based on the model input size, the left, top, right, and bottom coordinates of each ROI are calculated using a given formula. Finally, an image cropping function is used to crop the observed area, resulting in multiple ROIs. For example, if clip_l is 100 pixels, clip_r is 500 pixels, clip_y is 300 pixels, and model_size is 224, the coordinates of multiple ROIs can be calculated and cropped.

[0084] The `undistort` function in the OpenCV library is used to correct distortion in an entire image. It corrects each pixel of the image based on the camera intrinsic matrix and distortion parameters to obtain a distortion-free image.

[0085] The `warpAffine` function is a function in the OpenCV library used to perform affine transformations (including rotation and translation) on images. Here, it is used to rotate a camera-corrected image so that the water flow direction is upward. Specifically, for each frame of the input image, the `undistort` function is first used to perform camera correction on the image, passing in the camera intrinsic matrix and distortion parameters; then, the rotation angle is determined, and the `warpAffine` function is used to rotate the corrected image so that the flow direction is upward.

[0086] Dense velocity distribution information refers to the velocity information at each location in the image obtained through model inference. It is a relatively fine velocity distribution data. Specifically, each ROI obtained in step 6 is extracted sequentially, and 32 frames containing the ROI are extracted from consecutive image frames. These 32 frames are input into the trained improved Unet model. The model outputs the maximum velocity value and the flow field distribution array. The two are multiplied to obtain the dense velocity distribution information of the ROI region.

[0087] The results integration and output involves integrating the dense velocity distribution information of each ROI in step 8, then mapping this information to the corresponding positions in the original image through coordinate mapping, and finally outputting the integrated results in the form of images or data files for easy viewing and analysis by staff.

[0088] The beneficial effects of the above technical solution are as follows: For practical straight channel flow measurement scenarios, a flow measurement model is proposed that discards flow direction information and focuses only on monitoring flow velocity information. Furthermore, the flow velocity information is decomposed into two types: maximum flow velocity and normalized flow field distribution, accelerating model convergence and increasing model accuracy. Integrating camera calibration operations into the central server allows for unified management of camera calibration data and visualization of calibration results, ensuring the accuracy of camera calibration. On the other hand, if camera calibration is performed at the edge, recalibration is required when camera parameters change (such as lens replacement or camera position adjustment), further increasing maintenance costs and workload. For models based on fixed camera calibration and flow direction as prerequisites, an adapted camera calibration and flow direction correction solution is proposed to ensure the accuracy and effectiveness of the input image. For the fixed input size problem of the model, a multi-ROI cropping and separate inference approach is used. The flow measurement algorithm formed in this way is more adaptable to solving practical problems, and the inference results can be converted into real flow velocity values.

[0089] This invention provides a flow measurement method for intelligent water conservancy monitoring and sensing equipment, which includes the following steps before fine-grained cropping of the observation area:

[0090] Extract the edge lines of the camera-corrected image, cut the edge lines into several sub-lines based on the image inflection points, and fit each sub-line.

[0091] Obtain the set of connecting lines for each sub-line, and determine the auxiliary fitting function of each connecting line in the set and the main fitting function of the corresponding sub-line;

[0092] Based on the first intersection point of each connecting line in the connecting line set with the corresponding sub-line, the first deviation of the corresponding auxiliary fitting function based on the first intersection point, and the second deviation of the corresponding main fitting function based on the first intersection point, a deviation sequence of the corresponding sub-line is constructed.

[0093] The deviation sequence is input into the sequence analysis model to obtain the auxiliary complexity coefficients of the corresponding sub-lines. At the same time, the main complexity coefficients are determined based on the edge pixel distribution of the corresponding sub-lines. ,in, The number of pixels at the edge of the sub-line; The area of ​​the region containing the sub-line; These are the weighting coefficients; It is the gradient direction The probability of occurrence; The number of intervals quantized for direction;

[0094] The global flow complexity coefficient is obtained by analyzing all auxiliary complexity coefficients and all principal complexity coefficients in the camera-corrected image. ,in, This represents the total number of all sub-lines; The main complexity coefficient of the j-th sub-line; Let j be the distance from the j-th sub-line to the center point of the image; This is the average distance of all sub-lines from the center point of the image; This is the adjustment coefficient;

[0095] Determine the number of lines in each set of connecting lines, and lock the sub-line corresponding to the largest number of lines as the first line. At the same time, select the sub-line with the largest coefficient from all major complexity coefficients and regard it as the second line.

[0096] The search range is defined by all sub-lines in the image after camera correction, with the first line as the initial search point and the second line as the end search point. The first closed search boundary is obtained according to the shortest distance principle, and the first local water flow complexity coefficient is determined. At the same time, the second closed search boundary is obtained according to the longest distance principle, and the second local water flow complexity coefficient is determined.

[0097] Connect the center points of the first line and the second line to obtain a connecting line, and determine the number of other lines that pass through the connecting line to obtain the third local water flow complexity coefficient.

[0098] The final trimming coefficient is determined based on the global flow complexity coefficient, the first local flow complexity coefficient, the second local flow complexity coefficient, and the third local flow complexity coefficient.

[0099] Fine-grained cropping is performed on the camera-corrected image by matching the cropping granularity with the global flow complexity coefficient and the final cropping coefficient from the dual-coefficient-granularity lookup table.

[0100] In this embodiment, the image after camera correction is the water flow monitoring image after step 4 (eliminating lens distortion). For example, the original image of the river has curved edges due to lens distortion, and the corrected edges are closer to the real shape.

[0101] In this embodiment, the edge line is the boundary between the water flow area and the non-water flow area (such as riverbank or dam) in the image, and can be extracted by an edge detection algorithm. For example, the boundary between the river and the riverbank can be detected using OpenCV's Canny algorithm, and the continuous white lines in the output black and white image are the edge lines.

[0102] In this embodiment, the inflection point is a point on the edge line where the direction changes significantly (the point of maximum curvature). For example, the turning point where the edge line first bends to the left and then suddenly bends to the right can be determined by calculating the curvature of each point on the edge line (such as the angle change of three adjacent points).

[0103] In this embodiment, the sub-line is a segmented line formed by cutting the edge line at the inflection point. For example, if an edge line has two inflection points, it can be cut into three independent sub-lines.

[0104] In this embodiment, the fitting approximates the shape of the sub-line using mathematical functions (such as straight lines or polynomials). For example, if a sub-line approximates a straight line, it can be fitted using a linear function y=kx+b. The coefficients k and b can be calculated using the least squares method. Specifically, the edge lines are extracted using OpenCV's Canny() function.

[0105] In this embodiment, the curvature of each point on the edge line is calculated (e.g., the change in the tangent angle is calculated by the coordinates of 5 adjacent points), and points with curvature greater than a threshold are selected as inflection points; the edge line is cut into sub-lines by using the split() function according to the coordinates of the inflection points; and each sub-line is polynomial fitted (e.g., first-order or second-order fitting) using numpy.polyfit().

[0106] In this embodiment, the connecting line set is a collection of other sub-lines that are spatially related to a certain sub-line. For example, sub-lines B and C (segments of the right riverbank edge) near sub-line A (left riverbank edge) form the connecting line set of A by connecting lines to B and A and C.

[0107] In this embodiment, the connecting line is a single line in a set of connecting lines, that is, a line segment connecting the target sub-line with other sub-lines. For example, a straight line connecting the midpoint of sub-line A with the midpoint of sub-line B.

[0108] In this embodiment, the auxiliary fitting function is a mathematical function used to fit the connecting line. For example, when the connecting line is a curve, the auxiliary fitting function is obtained by fitting the curve with a quadratic function y=ax²+bx+c.

[0109] In this embodiment, the master fitting function is a mathematical function that fits the current sub-line itself (same as the fitting result in step 1). For example, the linear function y=0.5x+3 obtained by fitting sub-line A is the master fitting function. The Euclidean distance between sub-lines is calculated, and other sub-lines with a distance less than a threshold (e.g., 50 pixels) are selected as associated lines to form a set of connecting lines;

[0110] In this embodiment, feature points (such as midpoints) connecting sub-lines and associated lines are used to generate connecting lines;

[0111] Use numpy.polyfit() to fit the connecting line and the sub-line respectively to obtain the auxiliary fitting function and the main fitting function.

[0112] In this embodiment, the first intersection point is the intersection point of the connecting line and the corresponding sub-line. For example, the coordinates of the intersection point of the connecting line (AB) and the sub-line A are (100, 200). The first deviation is the deviation between the predicted value of the auxiliary fitting function at the first intersection point and the actual value of the connecting line. For example, if the auxiliary fitting function predicts a y-value of 200 at (100, 200), and the actual y-value of the connecting line is 202, then the first deviation = 202 - 200 = 2.

[0113] In this embodiment, the second deviation is the deviation between the predicted value of the master fitting function at the first intersection point and the actual value of the sub-line. For example, if the master fitting function predicts an x-value of 100 at (100, 200) and the actual x-value of the sub-line is 98, the second deviation is 98 - 100 = -2.

[0114] In this embodiment, the deviation sequence is an array that arranges all first and second deviations of the same sub-line in order. For example, sub-line A has 3 connecting lines, resulting in 6 deviation values, forming the sequence [2,-2,1,-1,3,-3]. Specifically, the coordinates of the first intersection point are obtained by solving the system of equations (connecting line equations and sub-line equations); the difference between the predicted value and the actual value of the auxiliary fitting function and the main fitting function at the intersection point is calculated to obtain the first and second deviations; and the deviation values ​​are combined into a deviation sequence according to the connecting line order.

[0115] In this embodiment, the sequence analysis model is a model (such as an LSTM neural network) used to analyze the fluctuation characteristics of the deviation sequence and outputs an index that quantifies the complexity. For example, given the deviation sequence [2,-2,1,-1,3,-3], the model outputs a coefficient reflecting the complexity of the line association.

[0116] In this embodiment, the auxiliary complexity coefficient is a coefficient output by the sequence analysis model that reflects the complexity of the relationship between the sub-line and other lines (the larger the value, the more complex the relationship). For example, the model outputs 0.7 (range 0~1).

[0117] In this embodiment, edge pixel distribution refers to the number and spatial distribution characteristics (such as density and directional diversity) of pixels on the edge of a sub-line. For example, if the pixels on the edge of a sub-line are dense and their directions are disordered, it indicates that the water flow is more complex.

[0118] In this embodiment, an LSTM model is built using Keras, with a deviated sequence as input (length fixed at 10) and an auxiliary complexity coefficient as output (the training data consists of manually labeled complexity labels).

[0119] In this embodiment, the number of lines is the total number of connecting lines contained in each set of connecting lines. For example, the set of connecting lines for sub-line A has 6 connecting lines, so the number of lines is 6.

[0120] In this embodiment, the first line is the sub-line with the largest number of connecting lines (reflecting the densest association with other lines). For example, if sub-line B has a connecting line set of 8 lines (the most), then B is the first line.

[0121] The second line is the sub-line with the largest primary complexity coefficient (reflecting the most complex edge of itself). For example, if sub-line C has Cmain=1.8 (maximum), then C is the second line. Specifically, the process is as follows: count the number of connecting lines contained in the connecting line set of each sub-line, and take the sub-line corresponding to the maximum value as the first line; traverse all primary complexity coefficients, and take the sub-line corresponding to the maximum value as the second line.

[0122] In this embodiment, the search range is the area covered by all sub-lines in the image (i.e., the water flow boundary area where the edge lines are located).

[0123] The initial search point is the geometric center of the first line, such as the center point coordinates (150, 300).

[0124] The search ends at the geometric center of the second line, such as the center point coordinates (450, 500).

[0125] The shortest distance principle is to connect the initial point and the end point, and form a closed boundary (such as the minimum convex hull) that contains all sub-lines and has the shortest perimeter.

[0126] The first closed search boundary is the boundary of a closed region generated according to the shortest distance principle (such as the smallest polygon surrounding the first line, the second line, and the intermediate sub-lines).

[0127] The longest distance principle is to connect the initial point and the end point, and form the closed boundary with the longest perimeter that includes all sub-lines (such as the largest polygon that includes the outer sub-lines).

[0128] The second closed search boundary is a closed region boundary generated according to the longest distance principle (its range is larger than the first boundary).

[0129] The center point is the geometric center of the sub-line (such as the midpoint between the two endpoints of the sub-line). For example, the endpoints of the first line are (100, 200) and (200, 400), and the center point is (150, 300). The connecting line is the line that connects the center points of the first and second lines, such as the two-point equation y = 0.5x + 225.

[0130] The remaining lines are sub-lines other than the first and second lines.

[0131] In this embodiment, the dual-coefficient-granularity lookup table is a pre-established correspondence table between the global flow complexity coefficient, the final clipping coefficient, and the clipping granularity. For example, when Cglobal is 1.4~1.5 and the final clipping coefficient is 2.4~2.5, the clipping granularity is 64×64 pixels. The clipping granularity is the size of the ROI (region of interest) after clipping (e.g., 32×32, 64×64 pixels).

[0132] Fine-grained cropping divides an image into multiple Regions of Interest (ROIs) according to a matching granularity. For example, a granularity of 64×64 uniformly crops the image into 64×64 sub-regions.

[0133] In this embodiment, the first local flow complexity coefficient ,in, The set of sub-lines within the first closed search boundary; These are the weighting coefficients; for The number of neutron lines; for The actual arc length of the j-th sub-line; for The ideal straight line length of the j-th sub-line; This represents the number of intersections between sub-lines within the first closed search boundary; The length of the first closed search boundary; The length of a single intersection point.

[0134] Second local flow complexity coefficient ,in, The set of sub-lines within the second closed search boundary; for The number of neutron lines; for The direction angle of the j2th sub-line; for The average value of the direction angles of all sub-lines; These are the weighting coefficients; for The standard deviation of all sub-lines in the middle; for The average value of all sub-lines in the middle;

[0135] Third Local Flow Complexity Coefficient ,in, The number of sub-lines that the connecting line passes through; The length of the connecting line; The length of the line; For the set of sub-lines that the connecting straight line passes through; The average width of the j3rd sub-line;

[0136] The final cropping factor is the sum of the products of each calculated factor and its corresponding weight, where, + + =1, and varies depending on the actual application scenario. , , The values ​​can be different.

[0137] The beneficial effects of the above technical solution are: by refining the analysis of the water flow edge line and combining multi-dimensional complexity coefficients (global and local) to quantify the complexity of the water flow, the cropping granularity is accurately matched to achieve targeted fine-grained cropping of the image. This not only ensures the effectiveness of the model input (preserving key water flow features) but also improves the accuracy and efficiency of subsequent flow velocity inference, providing more suitable technical support for flow monitoring in smart water conservancy monitoring.

[0138] This invention provides a flow measurement system for intelligent water conservancy monitoring and sensing equipment, such as... Figure 3 As shown, it includes:

[0139] The service setup module is used to set up camera calibration services, deploy camera calibration services on servers accessible to edge sensing devices, and provide camera calibration services for smart water conservancy monitoring and sensing devices.

[0140] The training module is used for model training and employs an improved Unet network to train the recognition of water flow velocity.

[0141] The parameter acquisition module is used to call the camera calibration service. On the edge device, the camera calibration service HTTP interface is called to obtain camera calibration parameters, including the camera intrinsic matrix camera_matrix, distortion parameter dist_coeffs and pixel mapping ratio ppm;

[0142] The camera calibration module is used to perform camera calibration on the observation area. Based on the obtained camera calibration parameters, it uses the undistortPoints function from the OpenCV repository to perform camera calibration on the coordinates of the point set in the specified observation area, and performs extreme value correction on the calibrated coordinate values.

[0143] The flow direction correction module is used to correct the flow direction in the observation area. The water flow direction can be manually specified and the corrected image from the camera can be rotated so that the flow direction is upward.

[0144] The fine-grained cropping module is used to perform fine-grained cropping of the observation area, and calculates the coordinates of multiple ROIs to be cropped based on the observation area after camera correction and flow direction correction;

[0145] The synchronization module is used to perform camera correction and flow direction correction on the input image. Based on the obtained camera calibration parameters, it uses the undistort function built into the OpenCV library to perform camera correction on each frame of the image, and synchronously uses warpAffine to rotate the corrected image so that the flow direction is upward.

[0146] The distribution information inference module is used to traverse the obtained ROI set. For each ROI, 32 consecutive frames of images are extracted and input into the model for inference. The maximum flow velocity value is obtained and multiplied with the flow field distribution array to obtain the dense flow velocity distribution information of a single inference.

[0147] The integration module is used to integrate the results of the inference and map them back to the original graph for output.

[0148] The beneficial effects of the above technical solution are as follows: For practical straight channel flow measurement scenarios, a flow measurement model is proposed that discards flow direction information and focuses only on monitoring flow velocity information. Furthermore, the flow velocity information is decomposed into two types: maximum flow velocity and normalized flow field distribution, accelerating model convergence and increasing model accuracy. Integrating camera calibration operations into the central server allows for unified management of camera calibration data and visualization of calibration results, ensuring the accuracy of camera calibration. On the other hand, if camera calibration is performed at the edge, recalibration is required when camera parameters change (such as lens replacement or camera position adjustment), further increasing maintenance costs and workload. For models based on fixed camera calibration and flow direction as prerequisites, an adapted camera calibration and flow direction correction solution is proposed to ensure the accuracy and effectiveness of the input image. For the fixed input size problem of the model, a multi-ROI cropping and separate inference approach is used. The flow measurement algorithm formed in this way is more adaptable to solving practical problems, and the inference results can be converted into real flow velocity values.

[0149] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A flow measurement method for a smart water conservancy monitoring and sensing device, characterized in that, Comprise: Step 1: build a camera calibration service, deploy a camera calibration service on a server accessible to edge perception devices, provide camera calibration services for smart water conservancy monitoring and perception devices; Step 2: model training, using improved Unet network to train to identify water flow velocity; Step 3: call the camera calibration service, call the camera calibration service http interface on the edge device end, get the camera calibration parameters, including camera intrinsic matrix camera_matrix, distortion parameter dist_coeffs and pixel mapping ratio ppm; Step 4: camera correction for observation area, based on the camera calibration parameters obtained in step 3, using the undistortPoints function provided by opencv repository to correct the point set coordinates of the specified observation area, and performing extreme value correction on the corrected coordinate values; Step 5: flow direction correction for observation area, manually specify the water flow direction, rotate the camera corrected image to make the flow direction upward; Step 6: fine-grained cropping of observation area, calculate multiple ROI coordinates to be cropped based on the observation area after camera correction and flow direction correction; Step 7: camera correction and flow direction correction for input image, based on the camera calibration parameters obtained in step 3, using the undistort function provided by opencv library to correct each frame of image, and using warpAffine to rotate the corrected image to make the flow direction upward; Step 8: traverse the ROI set obtained in step 6, for each ROI, take 32 consecutive frames of image as input to the model for inference, get the maximum flow velocity value and flow field distribution array and multiply them to get the dense flow velocity distribution information of single inference; Step 9: integrate the inference results and map them to the original image for output.

2. The flow measuring method for a smart water conservancy monitoring and sensing device according to claim 1, characterized in that, The Unet network receives the input of 32 consecutive frames of image, uses feature fusion to overlap and merge the channels of adjacent frames to learn the time sequence information of video frames in time dimension, outputs the first supervision information, i.e. the maximum flow velocity value, through channel dimension reduction after downsampling, gradually restores the spatial dimension of data through channel skip connection and upsampling, and outputs the flow field distribution array of 18080 as the second supervision information after upsampling.

3. The flow measuring method for a smart water conservancy monitoring and sensing device according to claim 1, characterized in that, The multiple ROI coordinates to be cropped include: Based on the observation area, get the start and end points clips to be cropped, including clip_l, clip_r and clip_y, which represent the horizontal start point, horizontal end point and vertical center point coordinates of the region to be cropped, respectively, and model_size represents the model input size, i.e. the cropping size, the calculation method of each ROI region coordinate is as follows: ； wherein, represents the first ROI coordinate, l, t, r, b represent left, top, right, bottom coordinates of the ROI, respectively.

4. The flow measuring method for a smart water conservancy monitoring and sensing device according to claim 1, characterized in that, The camera calibration service built in step 1 enables the edge device to directly call the service interface to obtain the camera calibration parameters. 5.The flow measurement method for the smart water conservancy monitoring and sensing device of claim 1, wherein, In step 5, the flow direction of the observation area is corrected to be upward by default.

6. The flow measuring method for a smart water conservancy monitoring and sensing device according to claim 1, characterized in that, In step 6, multiple ROI cropping is performed on the observation area to adapt to the size input requirements of the model.

7. The flow measuring method for a smart water conservancy monitoring and sensing device according to claim 1, characterized in that, Before fine-grained cropping of the observation area, An edge line of the image corrected by the camera is extracted, and the edge line is cut according to image inflection points to obtain a plurality of sub-lines, and each sub-line is fitted; A connection line set of each sub-line is obtained, and an auxiliary fitting function of each connection line in the connection line set and a main fitting function of the corresponding sub-line are determined; A deviation sequence of the corresponding sub-line is constructed according to a first intersection point of each connection line in the connection line set and the corresponding sub-line, a first deviation of the corresponding auxiliary fitting function based on the first intersection point, and a second deviation of the corresponding main fitting function based on the first intersection point; inputting the deviated sequence into a sequence analysis model to obtain an auxiliary complexity coefficient corresponding to the sub-line, and determining a main complexity coefficient according to an edge pixel distribution of the corresponding sub-line wherein, is an edge pixel number of the sub-line; is an area of a region where the sub-line is located; is a weight coefficient; is a gradient direction occurrence probability; is an interval number of direction quantization; According to all auxiliary complexity coefficients and all main complexity coefficients in the image corrected by the camera, a global water flow complexity coefficient is obtained wherein, is the total number of all sub-lines; is the main complexity coefficient of the jth sub-line; is the distance from the jth sub-line to the center point of the image; is the average distance from all sub-lines to the center point of the image; is an adjustment coefficient; The number of lines in each connection line set is determined, and the sub-line corresponding to the maximum number of lines is locked as a first line, and the sub-line corresponding to the maximum coefficient is selected from all main complex coefficients as a second line; All sub-lines in the image corrected by the camera are taken as a search range, the first line is taken as a search initial position point, the second line is taken as a search end position point, a first closed search boundary is obtained according to the shortest distance principle to determine a first local flow complexity coefficient, and a second closed search boundary is obtained according to the longest distance principle to determine a second local flow complexity coefficient; A connection straight line is obtained by connecting the center points of the first line and the second line, and the number of remaining lines passing through the connection straight line is determined to obtain a third local flow complexity coefficient; The final cropping coefficient is determined depending on the global flow complexity coefficient, the first local flow complexity coefficient, the second local flow complexity coefficient and the third local flow complexity coefficient; A cropping granularity consistent with the global flow complexity coefficient and the final cropping coefficient is matched from a double-coefficient-granularity matching table, and the image corrected by the camera is finely cropped.

8. A flow measurement system for a smart water conservancy monitoring and sensing device, characterized in that, It comprises: a service building module for building a camera calibration service, deploying a camera calibration service on a server accessible by an edge perception device, and providing a camera calibration service for a smart water conservancy monitoring and perception device; a training module for model training, and using an improved Unet network to train the recognition of water flow velocity; a parameter acquisition module for calling a camera calibration service, calling a camera calibration service http interface at an edge device end, and acquiring camera calibration parameters including a camera intrinsic matrix camera_matrix, distortion parameters dist_coeffs, and a pixel mapping ratio ppm; a camera correction module for camera correction of an observation area, using an undistortPoints function provided by an opencv repository to perform camera correction on point set coordinates of a specified observation area based on obtained camera calibration parameters, and performing extreme value correction on the corrected coordinate values; a flow direction correction module for flow direction correction of an observation area, manually specifying a water flow direction, and rotating a camera-corrected image to make the flow direction upward; a fine-grained cropping module for fine-grained cropping of an observation area, calculating a plurality of ROI coordinates to be cropped based on the observation area after camera correction and flow direction correction; A synchronization module is configured to perform camera correction and flow direction correction on the input image. Based on the obtained camera calibration parameters, an undistort function provided by the opencv library is used to perform camera correction on each frame of image. The corrected image is synchronously rotated by using warpAffine, so that the flow direction is upward. A distribution information reasoning module is configured to traverse the obtained ROI set, intercept 32 continuous frames of image for each ROI, input the image into a model for reasoning, obtain a maximum flow rate value and a flow field distribution array, and multiply the maximum flow rate value and the flow field distribution array to obtain dense flow rate distribution information of single reasoning. An integration module is configured to integrate the reasoning result and map the result to an original image for output.

Images