A non-contact river flow velocity calculation method and system based on super points

Abstract

The present invention discloses a non-contact river velocity calculation method and system based on super points, which belongs to the field of deep learning and velocity calculation. The method steps are as follows: collecting dynamic video images of river water surface under different lighting and climate conditions; processing the images by super point algorithm to extract water surface feature points, and generating descriptor information of feature points; constructing a descriptor enhancement model, mapping the original descriptor to a high-dimensional feature space by a multi-layer perceptron, and optimizing the descriptor information of feature points; generating enhanced descriptors of feature points by a deep feature network, dividing feature points by DBSCAN algorithm, and screening out feature areas where close points or similar points are located by density-based clustering method; combining with optical flow method, calculating the speed and direction of the river under different flow conditions, and performing real-time velocity analysis and river monitoring, which can meet the needs of flood prevention and disaster reduction and water environment management, and is suitable for unmanned long-term river monitoring tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning and flow velocity calculation, and in particular to a non-contact river flow velocity calculation method and system based on super points. Background Art

[0002] Traditional methods for measuring river flow velocity mainly include contact particle velocity measurement, sensor velocity measurement, etc. Although these methods have a certain degree of accuracy, they have many shortcomings in practical applications. First, the equipment installation is complex and costly, especially in mountainous rivers with complex terrain and inconvenient transportation, where the layout and maintenance of equipment are extremely difficult. Secondly, contact measurement methods usually require the equipment to be placed in the water body, which is not only easy to affect the river's ecological environment, but also increases the risk of equipment being damaged by floods or floating objects. In addition, the data acquisition range of traditional methods is limited and cannot provide comprehensive spatial coverage. Since these devices need to be regularly maintained and replaced, the operating costs are further increased. Real-time performance is also a major disadvantage of traditional methods. These measurement methods often rely on manual intervention, making it difficult to achieve continuous and automated real-time monitoring, which limits their application in flood warning.

[0003] With the development of science and technology, non-contact flow measurement technology has gradually emerged and become an important direction to solve the limitations of traditional methods. Non-contact technology uses sound, light, electricity and image processing technology to estimate the surface velocity of water bodies through telemetry, which greatly improves the safety and convenience of measurement. Methods such as radar and laser Doppler have realized the ability to obtain water surface velocity data without contacting the water body. These technologies not only avoid the complex operation of installing equipment in the water body, but also enhance the spatial coverage of the measurement, and can monitor the flow rate of rivers in a wider range. At the same time, with the development of deep learning and computer vision technology, river flow rate monitoring methods based on image processing have begun to be applied to actual scenarios. This type of method extracts water surface feature points through deep learning algorithms and combines optical flow methods for velocity estimation, which can achieve more accurate and real-time velocity calculations, reduce human intervention, and greatly improve the level of automation and real-time performance.

[0004] Therefore, it is of great research significance to further study non-contact river velocity monitoring, reduce computational complexity, maintain high real-time and automation, and apply it to unmanned long-term river monitoring tasks. Summary of the invention

[0005] The problem to be solved by the present invention is to provide a non-contact river flow velocity calculation method and system based on super points, which estimates the river flow velocity in real time and accurately through a non-contact method to meet the needs of flood prevention and disaster reduction and water environment management, while greatly reducing the calculation complexity and maintaining a high degree of real-time and automation, and can be suitable for unmanned long-term river monitoring tasks.

[0006] The present invention adopts the following technical solution: a non-contact river flow velocity calculation method based on super points, comprising the following steps:

[0007] S1. Data collection and preprocessing: Use video equipment to collect dynamic video images of the river surface under different lighting and climate conditions, capture the changes in the flowing water surface, and perform data cleaning;

[0008] S2, image feature extraction: the collected video images are processed by the super point algorithm to extract the water surface feature point information, including the feature point position coordinates and feature point descriptors;

[0009] S3, model construction and training: build a deep feature network model, map the input original descriptor to a high-dimensional feature space through a multi-layer perceptron, optimize the descriptor information of the feature points, and perform model training;

[0010] S4, feature point matching: using the deep feature network model to generate enhanced descriptors of feature points, matching feature points between adjacent images according to the Euclidean distance, dividing feature points using the DBSCAN algorithm, and screening feature areas where close points or similar points are located as feature salient areas through a density-based clustering method;

[0011] S5. Calculation of river flow velocity: In the feature-significant area, the optical flow method is combined to calculate the overall optical flow size in the feature area; and the speed of the river in the three-dimensional real world is calculated based on the transmission transformation principle to perform real-time flow velocity analysis and river monitoring.

[0012] Preferably, in step S1, data cleaning includes: noise filtering, contrast and brightness enhancement, and color balancing of the original image.

[0013] Preferably, step S3 model building includes the following sub-steps:

[0014] S301, constructing a deep feature network, including an input layer, multiple hidden layers and an output layer, each hidden layer performs nonlinear transformation through an activation function to enhance the descriptor's expressiveness;

[0015] S302, through the multi-layer perceptron, the feature point's own coordinate information is converted into a descriptor, for the initial descriptor , get the descriptor of the enhanced feature point .

[0016] Preferably, step S3 model training includes the following sub-steps:

[0017] S311. Balance training speed and stability by setting learning rate and batch size;

[0018] S312, optimize the average precision through the loss function and calculate the precision AP of each descriptor, ;

[0019] S313, Use represents the loss, averaging the average precision of all descriptor matches;

[0020] S304, reducing overfitting through regularization strategies, and monitoring the validation set by dynamically adjusting hyperparameters; the regularization strategies include L2 regularization and Dropout.

[0021] Preferably, step S4, feature point matching, includes the following sub-steps:

[0022] S401, using the enhanced descriptor generated by the deep feature network, selecting two frames of images within a certain time interval of the river video to obtain feature point descriptors of the two frames of images;

[0023] S402, using nearest neighbor matching to match the feature point descriptors of the two frames of images, and using the Euclidean method to calculate the distance to obtain the corresponding closest feature point distance;

[0024] S403, using a ratio test to compare the distances between the closest and second closest feature points, if the ratio is less than a preset threshold, the match is considered valid, otherwise it is discarded;

[0025] S404, using the DBSCAN clustering algorithm to screen the matched feature points, find the significant distribution area of ​​the feature points, calculate the upper left coordinate and the lower right coordinate of the significant feature area, and use them for river velocity estimation.

[0026] Preferably, for a stable river, step S5 adopts the mean method to calculate the river flow rate, including the following sub-steps:

[0027] S501. Assume that after classification by DBSCAN clustering algorithm, the set of salient region feature points is P, and the number of salient region feature points is ;

[0028] S502: After using the nearest neighbor matching based on the set P, obtain the feature point position coordinate set of the first image. , the feature point position coordinate set of the second image , the coordinates of the feature points after matching the first and second images are subtracted to obtain the optical flow set, and the average speed is calculated by summing the Euclidean magnitude of the optical flow;

[0029] S503, by summing up the Euclidean magnitude of the optical flow, calculate the overall average speed of the optical flow .

[0030] Preferably, for an unstable river, step S5, the river flow velocity calculation includes the following sub-steps:

[0031] S511, based on the upper left coordinates and lower right coordinates of the feature salient area, generate a global optical flow matrix using the raft optical flow method , obtain the optical flow size of the area;

[0032] S512, calculate the average optical flow size of the pixels in the area, filter out the optical flow that is larger than the average size, filter out the feature points with non-obvious motion, and calculate the temporary value of the speed ;

[0033] S513, using secondary search, for the optical flow matrix F, re-traverse the area, select the pixel optical flow vector whose optical flow size is greater than half of temp, and obtain the pixel optical flow set , execute steps S502 to S503 to calculate the overall optical flow size.

[0034] The technical solution of the present invention also includes: a non-contact river flow velocity calculation system based on super points, including: a video data acquisition unit, an image feature extraction unit, and a flow velocity calculation unit;

[0035] The video data acquisition unit uses high-resolution video equipment to obtain dynamic video image data of the river surface in real time, captures small changes in the flowing water surface under different lighting and climate conditions, and provides a data basis for river flow velocity calculation;

[0036] The image feature extraction unit processes the video image using a super point algorithm after the collected video image data is collected, extracts the water surface feature points, and generates descriptor information of the feature points;

[0037] The flow velocity calculation unit adopts a deep feature network framework and optimizes the descriptor information of feature points through a multi-layer perceptron. After matching the feature points, the DBSCAN algorithm is used to divide the feature points to obtain different feature point areas. The density-based clustering method is used to screen the feature-significant areas where similar or close points are located. By calculating the optical flow size of the feature-significant area, the river speed and direction under different flow conditions are obtained.

[0038] Preferably, it also includes a back-end server unit and a front-end display unit;

[0039] The back-end server unit receives the real-time video images collected by the video data collection unit through the real-time streaming protocol, implements WebSocket support by using the Flask framework and the Flask-SocketIO extension, saves the river speed and direction calculated by the flow velocity calculation unit through the flow velocity result, and pushes it to the front-end display unit through SocketIO;

[0040] The front-end display unit receives and displays river data in real time for unattended long-term monitoring tasks.

[0041] Preferably, the backend server unit includes: an image data receiving module, a deep learning reasoning module, and a flow rate result storage query and return module;

[0042] The image data receiving module, the camera device pushes the original image video stream data to the server through the real-time streaming protocol, the server extracts the video stream data frame by frame, stores it in the server's memory or file system, and segments each frame of the image and transmits it to the deep learning reasoning module for processing;

[0043] The deep learning inference module uses a super point algorithm to extract feature points from each frame of the image, processes the feature point descriptors through a deep feature network model, enhances the descriptor information of the feature points, calculates the motion vectors of the feature points in the image in combination with the optical flow method, and calculates the real-time flow velocity;

[0044] The flow rate result saving query and return module stores the flow rate data in real time in the database, and manages and queries it based on the timestamp and geographic location index mechanism. The query data is returned in JSON format; and the Flask-SocketIO extension is used to push the latest flow rate information to the front-end display unit in real time based on the WebSocket protocol.

[0045] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0046] 1. The non-contact river flow velocity calculation method of the present invention effectively avoids direct contact between the equipment and the water body through remote video acquisition and image processing, improves the system's adaptability to complex environments and the stability of data acquisition; by introducing video image processing technology and using computer vision methods to achieve non-contact flow velocity measurement, it effectively simplifies the flow velocity monitoring process and improves operational efficiency.

[0047] 2. The present invention designs a super point feature extraction method enhanced by a multi-layer perceptron (MLP), and combines the density clustering algorithm (DBSCAN) to segment the water surface feature points, and calculates the flow velocity through the optical flow method, so that the system can maintain high measurement accuracy and data stability in the flow field with large fluctuations and unevenness of the river.

[0048] 3. The non-contact river flow rate calculation system of the present invention realizes flow rate measurement based on low-cost camera equipment. Through visual algorithms, computer vision and deep learning frameworks are used to replace expensive hardware systems, which significantly reduces hardware investment and maintenance costs. At the same time, it avoids on-site operations, improves measurement safety in high-risk environments, and meets the needs of unattended flow rate measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 The framework diagram of the non-contact river velocity calculation system based on super points of the present invention;

[0050] Figure 2 This is a deep feature network model diagram of the present invention;

[0051] Figure 3 The present invention is a flow chart for the registration of river surface feature areas;

[0052] Figure 4 The DBSCAN feature region partition diagram is applied in the present invention;

[0053] Figure 5 This is a matching result diagram based on a deep feature network according to an embodiment of the present invention;

[0054] Figure 6 This is a diagram of river velocity estimation results according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the application is further elaborated in detail below in conjunction with the accompanying drawings. The described embodiments are only a part of the embodiments involved in the present invention. All non-innovative embodiments of other researchers in the field on this embodiment belong to the protection scope of the present invention. At the same time, for the step numbering in the embodiment of the present invention, it is only set for the convenience of explanation, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0056] In one embodiment of the present invention, a non-contact river flow velocity calculation system based on super points, such as Figure 1 As shown, it includes: a video data acquisition unit, an image feature extraction unit, and a flow rate calculation unit.

[0057] The video data acquisition unit acquires dynamic video images of the river surface in real time through the video data acquisition unit of the high-resolution camera equipment. This unit is not only highly adaptable and can work stably under different lighting and climatic conditions, but also can capture small changes in the flowing water surface, providing a high-quality data basis for subsequent feature extraction and flow velocity calculation.

[0058] Image feature extraction unit, after the video data collection is completed, the system uses the Superpoint algorithm to process the video image through the image feature extraction unit to extract the water surface feature points. Generate the descriptor information of the feature points to prepare for the subsequent deep learning network.

[0059] The velocity calculation unit uses a deep learning-based registration framework, the Deep Feature Network, to optimize the descriptors of feature points through a multi-layer perceptron, thereby improving the matching accuracy. This enables more accurate matching in the subsequent process.

[0060] After the feature points are matched, the DBSCAN algorithm will be used to divide the feature points. Because the regions of the feature points are different, the density-based clustering method can be used to screen out the feature regions where similar or close points are located. In this way, subsequent calculations only need to calculate the optical flow size of a certain area, reducing the calculation complexity. The flow velocity calculation process combines the optical flow method, which can accurately estimate the speed and direction of the river under different flow velocity conditions, fully demonstrating the adaptability of the present invention in dynamic environments.

[0061] Preferably, the non-contact river flow velocity calculation system based on super points in this embodiment further includes: a back-end server unit and a front-end display unit.

[0062] The backend server unit is responsible for receiving the real-time video stream transmitted via the RTSP protocol. This unit uses the Flask framework and the Flask-SocketIO extension to implement WebSocket support, ensuring that after the flow rate calculation is completed, the system can push the latest flow rate data to the front-end display system with low latency.

[0063] The front-end display unit has the ability to receive and display data in real time, providing users with intuitive flow rate monitoring results and improving the real-time and convenience of data analysis.

[0064] Furthermore, this embodiment is based on the above-mentioned non-contact river flow velocity calculation system, and estimates the river flow velocity in real time and accurately through a non-contact method.

[0065] 1. Build a flow velocity estimation system platform:

[0066] To ensure the stability and computational efficiency of the entire system, this implementation uses a high-performance multi-core CPU, such as Intel Xeon, to support multi-threaded parallel processing and can efficiently handle large amounts of data input, scheduling model training and inference tasks.

[0067] In terms of memory configuration, at least 64GB of RAM is used to ensure that data can be quickly loaded and preprocessed during big data processing and model training, and to support the training and reasoning of deep learning models.

[0068] In terms of graphics cards, the NVIDIA RTX 3090 graphics card is used as the core computing device for model training. The RTX3090 has 24GB of video memory, can handle large-scale data sets and complex deep learning tasks, and supports CUDA parallel computing, which greatly improves the training speed and reasoning efficiency. It is particularly suitable for the high-precision river velocity estimation model training of this embodiment, which helps to process the feature points extracted from the river surface video and the optical flow calculation tasks in real time.

[0069] In terms of software configuration, the Ubuntu 18.04 operating system is used, and the virtual environment is managed through Conda to build a deep learning training platform based on Python 3.8. It mainly relies on PyTorch 1.9.0 and its supported CUDA 11.1 to ensure the efficiency of GPU acceleration. The system also integrates common libraries such as NumPy, Pandas, and OpenCV for data processing and image processing. In addition, the subsequent service deployment uses the Flask framework to ensure the efficient deployment and calling of the flow velocity estimation model.

[0070] 2. Build and train a deep feature network, such as Figure 2 shown.

[0071] 2.1 Data collection and preprocessing

[0072] The camera position and angle are fixed, and the distances between the four vertices of the rectangle shown by the camera in the three-dimensional world are recorded for subsequent transmission transformation. After that, characteristic video data of the river surface is collected and data cleaning is performed to ensure that the data is suitable for model input.

[0073] In particular, this embodiment is to correct the original video image captured by the camera. The transmission transformation realizes the mapping from the two-dimensional image plane to the three-dimensional real world by obtaining the coordinates of the four vertices of the river surface in the image, so as to calculate the real river speed.

[0074] 2.2 Feature Extraction

[0075] Based on the super point algorithm, water surface feature points are extracted, and then feature points and descriptor information are generated to provide training data for the deep feature network.

[0076] 2.3 Model Training and Optimization

[0077] The universal approximation theorem states that a feedforward neural network with a linear output layer and at least one hidden layer with a nonlinear activation function with a "squeezing" property can approximate any Borel-measurable function from one finite-dimensional space to another with arbitrary accuracy, as long as the network is given a sufficient number of hidden units.

[0078] Therefore, the coordinate information of the feature points themselves can also be used as descriptor information, which can be converted into high-dimensional descriptors through multi-layer perceptrons. The improved descriptors can be expressed as follows:

[0079] ;

[0080] in, represents the enhanced descriptor, is the initial descriptor, represents the weight, = (x, y) represents the coordinate position information of the i-th feature point, is the descriptor encoder, Represents the keypoint encoder.

[0081] In model training, a learning rate of 0.01 and a batch size of 32 were used to balance training speed and stability. The initial high learning rate accelerated convergence, dynamic decay improved accuracy, and the batch size optimized video memory usage and maintained gradient stability. In terms of the loss function, by optimizing the average accuracy, the calculation was simplified and the model's sorting ability was enhanced, thereby improving the generalization performance of the model under different river conditions and the accuracy of flow velocity estimation.

[0082] AP is used to represent the accuracy of each descriptor, where , the formula is as follows:

[0083] ;

[0084] in, Represents the decision threshold, which is used to adjust the confidence of the model output. Recall is the ratio of all actual positive samples that are correctly identified as positive samples. It represents accuracy, which is defined as the proportion of positive samples predicted by the model that are actually positive samples. Represents the domain of recall, which is in the range of [0,1];

[0085] For the loss part, we can sum and average the average precision of all descriptor matches. For the convenience of minimization, we can express it as follows:

[0086] ;

[0087] in, represents the enhanced descriptor, Indicates the number of descriptors;

[0088] Finally, regularization strategies (L2 regularization, Dropout) are used to reduce overfitting and ensure stable performance. Validation set monitoring is used to dynamically adjust hyperparameters to ensure the generalization performance of the model in different environments.

[0089] 3. Design the velocity estimation method and the registration process of the characteristic areas on the river surface, e.g. Figure 3 shown.

[0090] 3.1 Feature Point Matching

[0091] Using the enhanced descriptor generated by the deep feature network, two frames of images within a certain time interval of the river video are selected to obtain information on the feature points of the two frames of images. In particular, the two frames of images are not necessarily continuous, such as the first frame and the fourth frame of images, and the two frames of images can be within a certain time interval.

[0092] In this embodiment, two frames of images are selected, and the feature point information is respectively , ,in, , Respectively represent the feature point position coordinate sets of the two frames of images, , Respectively represent the descriptor information sets of two frames of images.

[0093] The feature point descriptors of the two frames are matched, and the nearest neighbor matching and the combined ratio test are used to prevent false matching. Assume that the descriptor sets of the two frames are: , , the distance can be calculated using the Euclidean method as follows:

[0094] ;

[0095] in, , They represent the descriptor sets of two frames of images respectively, N is the number of descriptors, t represents the subscript index number, , Respectively , The i-th and j-th descriptors in ;

[0096] For each Need to find the closest corresponding , calculated using the following method:

[0097] ;

[0098] in, Indicates the distance of the closest feature point, Indicates the corresponding minimum distance parameter value.

[0099] In order to avoid false matches, a ratio test is needed to compare the distances between the closest and second closest feature points. If the ratio is less than a certain threshold (such as 0.7 or 0.8), the match is considered valid, otherwise it is discarded. The specific method is as follows:

[0100] ;

[0101] in, Indicates the distance of the next closest feature point, The function represents the calculation of the Euclidean distance between two vectors.

[0102] 3.2 Feature Region Clustering

[0103] The DBSCAN clustering algorithm is used to screen the matching feature points. By setting appropriate thresholds (eps=0.35, minSamples=4 in this example), the area with the most significant feature point distribution is found. Figure 4 Scattered area shown.

[0104] Figure 4 (a) shows a river video image collected at a certain moment. Figure 4 (b) in the figure shows the feature points after enhanced descriptors and nearest neighbor matching. In this figure, the DBSCAN clustering method is used to classify the feature points into regions, where the region with the largest number of feature points of the same color is identified as a significant region, and then the upper left and lower right coordinates of the region are calculated for the estimation of river flow velocity.

[0105] 3.3 Optical flow estimation method

[0106] When estimating river water speed, the deep feature network is used in combination with the optical flow method to ensure more accurate motion estimation of feature point areas. The raft optical flow method uses a recursive network structure to efficiently calculate the motion between all pixel pairs, thereby generating a dense optical flow field, ensuring higher accuracy in the flow speed calculation process.

[0107] The optical flow vector in the image plane is converted to the actual physical speed through the projection transformation. The river speed is projected from the image space to the physical space through the transformation matrix to calculate the real river speed, taking into account factors such as the camera angle, shooting distance and transmission transformation.

[0108] For stable rivers, the mean method is used for calculation, where it is assumed that the set of significant area points obtained after the feature points are classified by the DBSCAN algorithm is P. is a set of feature points, and k is the number of feature points in the salient area.

[0109] After using the nearest neighbor matching based on the set P, the feature point position coordinate set of the first frame image is obtained , the feature point position coordinate set of the second frame image , subtract the coordinates of the feature points after matching the first and second images to get the optical flow set ; The average velocity is calculated by summing the Euclidean magnitude of the optical flow.

[0110] ;

[0111] in, Represents the coordinate difference between the corresponding positions of points2 and points1, and the pixel optical flow generated by this;

[0112] For unstable rivers, such as those with too fast movement speed, since the significant area has been calculated above, we only need to calculate the optical flow size of the area. The optical flow here can be obtained by the raft optical flow method to get F. However, some pixels have a small optical flow, because they are not real feature points, so here we first calculate the average size of the vector of the entire pixel points, and then filter out the optical flow larger than the average size. This method can filter out those feature points with non-obvious movement.

[0113] Assume that the coordinates of the upper left and lower right corners of the region are , , where the calculation method is as follows:

[0114] ;

[0115] in, , are the coordinates of the upper left corner and lower right corner of the feature salient area; Represents the optical flow vector corresponding to the image (i, j) position; Indicates the calculated temporary value of speed;

[0116] Since not all pixels in the salient area are moving, there is obviously an error between the above temp and the true value. In order to solve this problem, a secondary search is proposed to re-traverse , In the area enclosed by the two points, find the pixel whose optical flow size is greater than half of temp.

[0117] Get pixel set , and then use the above-mentioned stabilization method. Compared with the optical flow on the entire image, this approach significantly reduces the CPU calculation overhead.

[0118] This implementation is based on the matching results of the deep feature network, such as Figure 5As shown: the left side is the matching effect diagram of the optimized feature points, and the latter is the displacement difference of the pixels of the previous and next frames directly using the deep feature network and the speed measurement result diagram combined with the optical flow method. It can be seen that the effects are similar.

[0119] The river velocity estimation result diagram of this embodiment is as follows: Figure 6 As shown, the horizontal axis represents the specific moment of each unit time, and the vertical axis represents the speed in meters per second. Figure 6 (a) in the figure represents the velocity result calculated directly using the steady river calculation method. Figure 6 (b) shows the speed result calculated only by the raft optical flow method. Figure 6 (c) is the velocity result calculated by the non-stationary method. Figure 6 (d) in the figure is the result measured by the flow meter.

[0120] 4. Build the backend system

[0121] 4.1 Image data receiving module

[0122] In this embodiment, the camera pushes the video stream to the server via the Real-Time Streaming Protocol (RTSP). The server is designed with a streaming media receiving module, which is responsible for receiving the real-time video stream and extracting and processing it frame by frame.

[0123] The specific process is that the camera continuously pushes the video stream, and after the server receives the streaming data, it segments each frame of the image and temporarily stores each frame of the image in the server's memory or file system. In this way, the system can process real-time video streams and is suitable for flow rate monitoring tasks with high continuity requirements.

[0124] 4.2 Deep Learning Inference Module

[0125] After the server extracts each frame of image from the receiving module, it immediately calls the deep learning model for processing. First, the super point algorithm is used to extract feature points from each frame; then the deep feature network model is called to process the feature points to enhance the descriptor information of the feature points; finally, the motion vector of the feature points in the image is calculated by combining the optical flow method to estimate the flow rate.

[0126] This process is real-time and continuous, ensuring that the river speed can be dynamically calculated as new video frames are continuously input.

[0127] 4.3 Saving, querying and returning flow rate results

[0128] After the flow rate calculation is completed, the back-end system stores the data in real time in the database, and based on index mechanisms such as timestamps and geographic locations, it facilitates efficient management and on-demand query of flow rate data.

[0129] To achieve seamless push of real-time data, the backend system uses the Flask-SocketIO extension to push the latest flow rate information to the front end in real time based on the WebSocket protocol, allowing users to obtain real-time flow rate changes at the millisecond level, ensuring the timeliness of monitoring and response.

[0130] In addition, the backend system provides a historical data query interface based on the Flask REST API, allowing users to obtain flow rate information within a specific time period. The query data is returned in JSON format, which is convenient for front-end display and further analysis and processing. Through this design, the system not only supports real-time flow rate monitoring, but also meets the user's access needs for historical data, ensuring the continuity, reliability and traceability of flow rate information, and is suitable for unattended long-term monitoring tasks.

[0131] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A non-contact river velocity calculation method based on super points, characterized in that: The steps include: S1. Data collection and preprocessing: Use video equipment to collect dynamic video images of the river surface under different lighting and climate conditions, capture the changes in the flowing water surface, and perform data cleaning; S2, image feature extraction: the collected video images are processed by the super point algorithm to extract the water surface feature point information, including the feature point position coordinates and feature point descriptors; S3, model construction and training: build a deep feature network model, map the input original descriptor to a high-dimensional feature space through a multi-layer perceptron, optimize the descriptor information of the feature points, and perform model training, including the following sub-steps: S311. Balance training speed and stability by setting learning rate and batch size; S312, optimize the average precision through the loss function, and use AP to represent the precision of each descriptor. , the formula is as follows: ； in, Represents the decision threshold, which is used to adjust the confidence of the model output. Recall is the ratio of all actual positive samples that are correctly identified as positive samples. It represents accuracy, which is defined as the proportion of positive samples predicted by the model that are actually positive samples. Represents the definition domain of recall rate, with a value range of [0,1]; S313, Use Represents the loss, averaging the average precision of all descriptor matches: ； in, represents the enhanced descriptor, Indicates the number of descriptors; S304, reducing overfitting through a regularization strategy, and monitoring the validation set by dynamically adjusting hyperparameters; the regularization strategy includes L2 regularization and Dropout; S4, feature point matching: using the deep feature network model to generate enhanced descriptors of feature points, matching feature points between adjacent images according to the Euclidean distance, dividing feature points using the DBSCAN algorithm, and screening feature areas where close points or similar points are located as feature salient areas through a density-based clustering method; S5. Calculation of river velocity: In the feature-significant area, the optical flow method is combined to calculate the overall optical flow size in the feature area; and the velocity of the river in the three-dimensional real world is calculated based on the transmission transformation principle, and real-time velocity analysis and river monitoring are performed; For stable rivers, the mean method is used to calculate the river velocity, which includes the following sub-steps: S501, assuming that after classification by DBSCAN clustering algorithm, the set of significant region feature points is P, , is the number of feature points in the salient area; S502: After using the nearest neighbor matching based on the set P, the feature point position coordinate sets of the two frames of images are obtained as follows: , , subtract the coordinates of the feature points after matching the two frames of images to obtain the optical flow set ; Calculate the average speed by summing the Euclidean magnitude of the optical flow; S503, by summing up the Euclidean magnitude of the optical flow, calculate the overall average speed of the optical flow : ； in, Indicates based on The pixel optical flow generated by the coordinate difference of the corresponding position of points1; For an unsteady river, the river velocity calculation includes the following sub-steps: S511, based on the upper left coordinates and lower right coordinates of the feature salient area, generate a global optical flow matrix using the raft optical flow method , get the optical flow size of the area; S512, calculate the average size of the optical flow of the pixels in the area, filter out the optical flow that is larger than the average size, and filter out the feature points with non-obvious motion, the method is as follows: ； in, , are the coordinates of the upper left corner and lower right corner of the feature salient area; Represents the optical flow vector corresponding to the image (i, j) position; Indicates the calculated temporary value of speed; S513, using secondary search, for the optical flow matrix F, re-traverse the feature-significant areas, and filter out the areas with optical flow greater than Half of the pixel optical flow vectors are obtained to obtain the pixel optical flow set , execute steps S502 to S503 to calculate the overall optical flow size.

2. The non-contact river flow velocity calculation method based on super points according to claim 1 is characterized in that: In step S1, the data cleaning includes noise filtering, contrast and brightness enhancement, and color balancing of the original image.

3. The non-contact river flow velocity calculation method based on super points according to claim 1 is characterized in that: Step S3: Model construction, including the following sub-steps: S301, constructing a deep feature network, including an input layer, multiple hidden layers and an output layer, each hidden layer performs nonlinear transformation through an activation function to enhance the descriptor's expressiveness; S302, through the multi-layer perceptron, the feature point's own coordinate information is converted into a descriptor, for the initial descriptor , the descriptor of the improved feature point is expressed as follows: ； in, represents the enhanced descriptor, represents the weight, Represents the coordinate position information of the i-th feature point, is the descriptor encoder, Represents the keypoint encoder.

4. The non-contact river flow velocity calculation method based on super points according to claim 3 is characterized in that: Step S4, feature point matching, includes the following sub-steps: S401, using the enhanced descriptor generated by the deep feature network, selecting two frames of images within a certain time interval of the river video to obtain feature point descriptors of the two frames of images; S402, using nearest neighbor matching to match the feature point descriptors of the two frames of images, assuming that the feature point descriptor sets of the two frames of images are , , , , using the Euclidean method to calculate the distance: ； Where N is the number of descriptors, t represents the subscript index number, , Respectively , The i-th and j-th descriptors in ; For each , find the corresponding closest , the method is as follows: ； in, Indicates the distance of the closest feature point, Indicates the corresponding minimum distance parameter value; S403, using a ratio test to compare the distances between the closest and second closest feature points. If the ratio is less than a preset threshold, the match is considered valid, otherwise it is discarded. The method is as follows: ； in, Indicates the distance of the second closest feature point, The function represents the calculation of the Euclidean distance between two vectors; S404, using the DBSCAN clustering algorithm to screen the matched feature points, find the significant distribution area of ​​the feature points, calculate the upper left coordinate and the lower right coordinate of the significant feature area, and use them for river velocity estimation.

5. A non-contact river velocity calculation system based on super points, used to implement the river velocity calculation method according to any one of claims 1 to 4, characterized in that: include: Video data acquisition unit, image feature extraction unit, flow velocity calculation unit; The video data acquisition unit acquires dynamic video image data of the river surface in real time through a high-resolution camera device, captures small changes in the flowing water surface under different lighting and climatic conditions, and provides a data basis for river flow velocity calculation; The image feature extraction unit processes the video image using a super point algorithm after the collected video image data is collected, extracts the water surface feature points, and generates descriptor information of the feature points; The velocity calculation unit adopts a deep feature network framework, optimizes the descriptor information of the feature points through a multi-layer perceptron, divides the feature points using a DBSCAN algorithm after matching the feature points, obtains different feature point regions, and uses a density-based clustering method to screen the feature regions where close or similar points are located, and obtains the river speed and direction under different flow conditions by calculating the optical flow size of the feature region; It also includes a back-end server unit and a front-end display unit; The back-end server unit receives the real-time video image collected by the video data collection unit through the real-time streaming protocol, uses the Flask framework and the Flask-SocketIO extension to implement WebSocket support, saves the river speed and direction calculated by the flow velocity calculation unit through the flow velocity result, and pushes it to the front-end display unit through SocketIO; The front-end display unit receives and displays river data in real time for unattended long-term monitoring tasks.

6. The non-contact river flow velocity calculation system based on super points according to claim 5, characterized in that: The back-end server unit includes: an image data receiving module, a deep learning reasoning module, and a flow rate result storage query and return module; The image data receiving module, the camera device pushes the original image video stream data to the server through the real-time streaming protocol, the server extracts the video stream data frame by frame, stores it in the server's memory or file system, and segments each frame of the image and transmits it to the deep learning reasoning module for processing; The deep learning inference module uses a super point algorithm to extract feature points from each frame of the image, enhances the descriptors of the feature points through a deep feature network, enhances the descriptor information of the feature points, calculates the motion vectors of the feature points in the image in combination with the optical flow method, and calculates the real-time flow velocity; The flow rate result saving query and return module stores the flow rate data in real time in the database, and manages and queries it based on the timestamp and geographic location index mechanism. The query data is returned in JSON format; and the Flask-SocketIO extension is used to push the latest flow rate information to the front-end display unit in real time based on the WebSocket protocol.

Images