A river surface flow velocity identification method

Abstract

The present application relates to the technical field of image recognition, and particularly relates to a river surface flow speed identification method, comprising: acquiring a frame sequence of a river video; performing spatial alignment processing on the frame sequence; performing texture enhancement processing on the frame sequence after the spatial alignment processing; dividing the frame sequence after the texture enhancement processing into multiple blocks, and extracting a time sequence signal of each block; performing normalization processing on the time sequence signal to obtain a pretreatment signal; inputting the pretreatment signal into a pre-trained time sequence one-dimensional convolutional neural network model to obtain a river surface flow speed. The above scheme uses a one-dimensional convolutional network to model long-range dependence of the time sequence signal, replaces traditional optical flow or three-dimensional convolution, can reduce the amount of calculation and storage demand while maintaining the sensitive capture of long-term flow speed changes, and realizes real-time inference.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically, to a method for recognizing the surface flow velocity of a river. Background Technology

[0002] River surface velocity is a key parameter for water resource management, flood warning, and aquatic ecosystem assessment. Traditional measurement methods, including buoy methods, propeller current meters, and acoustic Doppler current meters, all require placing sensors or floats into the water, resulting in complex deployment, high maintenance costs, and difficulties in continuous monitoring. In recent years, non-contact solutions based on computer vision have gradually emerged. These solutions use fixed or airborne cameras to collect video of the water surface and then combine this with image processing technology to retrieve the velocity. They offer advantages such as flexible deployment, good real-time performance, and low cost, making them a research hotspot in the field of hydrological monitoring.

[0003] Existing vision-based solutions primarily rely on optical flow methods or two-dimensional convolutional neural networks to process single frames or short-time frame sequences. Optical flow methods obtain velocity fields through pixel-by-pixel displacement estimation, which is computationally intensive and susceptible to interference from water surface reflections, waves, and floating objects, resulting in insufficient robustness. Two-dimensional convolutional neural networks utilize only single-frame spatial information, making it difficult to capture the continuous dynamic evolution of water surface textures and limiting their ability to model long-range temporal dependencies, thus causing a significant decrease in flow velocity estimation accuracy with environmental changes. Therefore, the most significant technical challenge of existing solutions lies in how to simultaneously achieve high accuracy and real-time performance in complex water surface environments, accurately capturing and modeling the long-term evolution characteristics of dynamic water flow textures. Summary of the Invention

[0004] The purpose of this invention is to provide a method for identifying river surface flow velocity in order to solve the above-mentioned technical problems.

[0005] This invention provides a method for identifying river surface velocity. The method includes: acquiring a frame sequence of a river video; performing spatial alignment processing on the frame sequence; performing texture enhancement processing on the spatially aligned frame sequence; dividing the texture-enhanced frame sequence into multiple blocks and extracting a temporal signal from each block; performing normalization processing on the temporal signal to obtain a preprocessed signal; and inputting the preprocessed signal into a pre-trained temporal one-dimensional convolutional neural network model to obtain the river surface velocity output by the temporal one-dimensional convolutional neural network model.

[0006] Further, the spatial alignment processing of the frame sequence includes: detecting key points in two adjacent frames of the frame sequence and generating corresponding description vectors; pairing the description vectors to obtain an initial set of matching point pairs; iteratively filtering the initial matching point pairs in the initial set of matching point pairs, calculating the optimal geometric transformation matrix and retaining the interior points that satisfy the optimal geometric transformation matrix; performing affine mapping on the pixel coordinates of the entire frame image based on the optimal geometric transformation matrix to generate a corrected image with the same spatial position as the reference frame, thus completing the inter-frame alignment.

[0007] Further, the texture enhancement processing of the frame sequence after spatial alignment includes: dividing each frame image in the frame sequence after spatial alignment into multiple non-overlapping local image blocks; for each local image block, obtaining the gray-level distribution of the local image block and generating a local histogram; limiting the upper end of the local histogram based on a preset cropping threshold; evenly distributing the number of cropped pixels to each gray level to obtain a corrected histogram; constructing a gray-level mapping function based on the corrected histogram and performing gray-level transformation on each pixel in the local image block; and re-stitching all the local image blocks after gray-level transformation back to their original spatial positions to obtain the texture-enhanced full frame image.

[0008] Further, the step of dividing the frame sequence after texture enhancement into multiple blocks and extracting the temporal signal of each block includes: obtaining a multi-scale block set based on a pre-defined spatial grid of at least two different pixel sizes; performing gridded segmentation on each frame image in the frame sequence after texture enhancement based on the multi-scale block set to obtain multiple blocks; configuring a unique block identifier for each block and recording the spatial coordinate range of the block in the original image; reading the pixel values ​​of the same block identifier in all frames along the time dimension to construct the temporal signal corresponding to the block; and obtaining a block-level temporal signal set for all the blocks based on the temporal signal.

[0009] Further, the step of obtaining a multi-scale block set based on at least two pre-defined spatial grids of different pixel sizes includes: obtaining a pre-defined first-scale grid, a second-scale grid, and a third-scale grid; wherein the first scale is smaller than the second scale, and the second scale is smaller than the third scale; selecting at least two scales of grids from the first-scale grid, the second-scale grid, and the third-scale grid based on the water flow turbulence level and / or a preset monitoring accuracy index to obtain a target-scale grid; generating a uniform grid according to the corresponding pixel scale for each pixel scale of the target-scale grid, with the upper left corner of the image as the origin, to obtain a grid instance corresponding to each pixel scale; configuring a globally unique identifier for each cell in each grid instance, and recording the starting row and column coordinates, width pixel value, and height pixel value of the cell in the image coordinate system to construct a block metadata structure; calculating the spatial intersection-union ratio (CIU) of any two cells in the grid instances of different pixel scales, and if the CIU exceeds a preset CIU threshold, retaining the cell with the larger area and discarding the cell with the smaller area to obtain a multi-scale block set.

[0010] Further, the normalization processing of the time series signal to obtain the preprocessed signal includes: traversing the time series signal corresponding to each block, calculating the arithmetic mean and standard deviation of the time series signal at all time steps; subtracting the arithmetic mean from the value of each time step of the time series signal to obtain a zero-mean signal; dividing the value of each time step of the zero-mean signal by the standard deviation to obtain a unit variance signal; if the standard deviation is lower than a preset lower threshold, setting the entire time series signal of the block to zero or filling it with a preset constant; obtaining the normalized time series signal of each block to construct the preprocessed signal.

[0011] Furthermore, the temporal one-dimensional convolutional neural network model includes a first convolutional block, a second convolutional block, a third convolutional block, a global average pooling layer, and a fully connected layer connected in sequence; the first convolutional block, the second convolutional block, and the third convolutional block each include multiple one-dimensional convolutional layers;

[0012] The step of inputting the preprocessed signal into a pre-trained temporal one-dimensional convolutional neural network model to obtain the river surface velocity output by the temporal one-dimensional convolutional neural network model includes: inputting the preprocessed signal into the first convolutional block to obtain a primary temporal feature map; inputting the primary temporal feature map into the second convolutional block to obtain an intermediate temporal feature map; inputting the intermediate temporal feature map into the third convolutional block to obtain a high-level temporal feature map; inputting the high-level temporal feature map into the global average pooling layer that performs mean operation along the time dimension to obtain a two-dimensional feature vector; and inputting the two-dimensional feature vector into the fully connected layer to obtain the initial velocity value corresponding to each block.

[0013] Furthermore, the method further includes: for each block, reading the width pixel value and height pixel value in the block metadata structure, and determining the actual pixel area of ​​the block based on the width pixel value and the height pixel value; determining the normalized weight coefficient of the current block based on the sum of the actual pixel areas of all the blocks and the actual pixel area of ​​the current block; determining the weighted flow velocity value based on the normalized weight coefficient and the initial flow velocity value; and summing the weighted flow velocity values ​​of all the blocks to obtain the flow velocity value of the river surface.

[0014] Furthermore, the method further includes: acquiring historical flow velocity data and historical video frame sequences of the river; establishing a timestamp association between the historical flow velocity data at the same moment and its corresponding historical video frame sequences to obtain training samples; randomly occluding a portion of the temporal signal set in the training samples according to a preset ratio to simulate missing information; inputting the occluded temporal signal into the temporal one-dimensional convolutional neural network model to calculate the error between the flow velocity value output by the temporal one-dimensional convolutional neural network model and the actual flow velocity; iteratively updating the network weights of the temporal one-dimensional convolutional neural network model using a backpropagation algorithm until the verification error stably converges within a continuous preset period to obtain the trained temporal one-dimensional convolutional neural network model.

[0015] Furthermore, the step of inputting the occluded time-series signal into the one-dimensional convolutional neural network model and calculating the error between the flow velocity value output by the one-dimensional convolutional neural network model and the actual flow velocity includes: inputting the occluded time-series signal into the one-dimensional convolutional neural network model and calculating the error between the flow velocity value output by the one-dimensional convolutional neural network model and the actual flow velocity using the Huber loss function.

[0016] The above scheme effectively suppresses noise from water surface reflections, waves, and floating objects through cascaded processing of frame alignment and CLAHE texture enhancement, improving the recognizability of water surface textures in low-contrast environments and making the velocity feature extraction more environmentally robust. On the other hand, it uses a one-dimensional convolutional network to model the long-range dependency of temporal signals, replacing traditional optical flow or three-dimensional convolution, which can reduce computational and storage requirements while maintaining a keen capture of long-term velocity changes, enabling real-time inference. Furthermore, the weighted fusion mechanism based on the actual area of ​​the blocks can adaptively compensate for spatial coverage differences caused by different scale blocks, so that the velocity contributions of the mainstream and local turbulence are balanced, improving the physical rationality of the overall estimation. Finally, the introduction of random occlusion enhancement and anomaly-resistant loss function during the training phase enables the model to still have robust output capabilities under conditions such as floating object occlusion and sudden changes in illumination, enhancing the robustness and environmental adaptability of actual deployment.

[0017] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A schematic flowchart of the river surface velocity identification method provided in an embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram illustrating the application process of the video frame preprocessing module provided in an embodiment of the present invention in a certain application scenario.

[0021] Figure 3 This is a schematic diagram of the architecture of a temporal one-dimensional convolutional neural network model provided in an embodiment of the present invention. Detailed Implementation

[0022] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0023] Please see Figure 1 This invention provides a method for identifying river surface velocity, comprising:

[0024] Step S110: Obtain the frame sequence of the river video.

[0025] It should be noted that, to obtain raw data for subsequent flow velocity identification, a vertically top-down imaging device can be used to continuously photograph the surface of the target river. The deployed camera unit captures the dynamics of the water area at a fixed frame rate of no less than 30 frames per second, forming a continuous image sequence. This imaging device can be mounted on a drone, a fixed mast, or the bottom of a bridge, with its optical axis perpendicular to the water surface being measured to ensure constant pixel resolution within the field of view and reduce perspective distortion. During the acquisition process, the exposure time, gain, and white balance parameters of the camera unit are automatically adjusted according to the ambient light, while the absolute timestamp of each frame is recorded to ensure accurate alignment with the actual flow velocity data measured by the Doppler radar current meter. The resulting video stream is stored in its original lossless format and forms a temporally continuous image sequence in the local buffer, providing a complete and synchronous input source for subsequent spatial alignment, texture enhancement, block segmentation, and flow velocity estimation.

[0026] Step S120: Perform spatial alignment processing on the frame sequence.

[0027] Preferably, in one embodiment of the present invention, step S120 may include: detecting key points in two adjacent frames of the frame sequence and generating corresponding description vectors; pairing the description vectors to obtain an initial set of matching point pairs; iteratively filtering the initial matching point pairs in the initial set of matching point pairs, calculating the optimal geometric transformation matrix and retaining the interior points that satisfy the optimal geometric transformation matrix; performing affine mapping on the pixel coordinates of the entire frame image based on the optimal geometric transformation matrix to generate a corrected image that is consistent with the spatial position of the reference frame, thereby completing the inter-frame alignment.

[0028] It should be noted that, for a continuously acquired river video frame sequence, feature detection is first performed on adjacent frames: stable keypoints are extracted in each frame using a scale-invariant feature transformation algorithm, and a high-dimensional description vector is generated for each keypoint; then, a nearest neighbor search strategy is used to pair the description vectors of the two frames to form an initial set of matching point pairs. To eliminate mismatches, a random sampling consensus algorithm is used to iteratively filter this set: in each iteration, candidate affine transformation matrices are estimated, the number of interior points satisfying the matrix is ​​counted, and the matrix with the highest proportion of interior points is retained as the optimal geometric transformation matrix. Based on this matrix, affine mapping is performed on the pixel coordinates of the entire frame image to achieve pixel-level spatial resampling, thereby outputting a corrected image that is precisely aligned with the spatial position of the reference frame, completing the inter-frame alignment.

[0029] Step S130: Perform texture enhancement processing on the frame sequence after spatial alignment.

[0030] Preferably, in one embodiment of the present invention, step S130 may include: dividing each frame image in the frame sequence after spatial alignment into multiple non-overlapping local image blocks; for each local image block, obtaining the gray-level distribution of the local image block and generating a local histogram; limiting the upper end of the local histogram based on a preset cropping threshold; uniformly distributing the number of cropped pixels to each gray level to obtain a corrected histogram; constructing a gray-level mapping function based on the corrected histogram and performing gray-level transformation on each pixel in the local image block; and re-stitching all the gray-level transformed local image blocks according to their original spatial positions to obtain a texture-enhanced full frame image.

[0031] It should be noted that, based on the spatially aligned frame sequence, each grayscale image is first divided into uniformly sized local image blocks without overlap. Then, the grayscale distribution within each local image block is statistically analyzed, and a corresponding local histogram is generated. For this local histogram, the amplitude of its high-end grayscale levels is limited according to a preset cropping threshold to suppress extreme grayscale values ​​caused by water surface reflections or noise. The total number of cropped pixels is evenly redistributed to the grayscale levels of the histogram, forming a corrected histogram. A monotonically increasing grayscale mapping function is constructed using the corrected histogram, and grayscale transformation is performed point-by-point on all pixels within the local image block to enhance contrast. All grayscale-transformed local image blocks are reassembled according to their original spatial coordinates, ultimately obtaining a complete frame image with enhanced texture details and controlled noise, for subsequent block segmentation and flow velocity feature extraction.

[0032] Step S140: Divide the frame sequence after texture enhancement into multiple blocks and extract the temporal signal of each block.

[0033] Preferably, in one embodiment of the present invention, step S140 may include: obtaining a multi-scale block set based on a pre-defined spatial grid of at least two different pixel sizes; performing grid-based segmentation on each frame image in the frame sequence after texture enhancement processing based on the multi-scale block set to obtain multiple blocks; configuring a unique block identifier for each block and recording the spatial coordinate range of the block in the original image; reading the pixel values ​​of the same block identifier in all frames along the time dimension to construct the temporal signal corresponding to the block; and obtaining a set of temporal signals for all blocks based on the temporal signal.

[0034] It should be noted that for frame sequences with texture enhancement completed, the aforementioned river surface velocity recognition method can first call a preset multi-scale grid dictionary, which contains at least two discrete scale parameters of different pixel sizes. Based on the weights of detail resolution and global trend requirements of the current monitoring task, two or more scales are selected to construct an effective grid set for this segmentation. Subsequently, using a Cartesian coordinate system with the top left corner of the image as the origin and pixels as the unit, each frame image is divided into non-overlapping grids according to the effective grid set at each scale, resulting in a block array with a unique index. Each block is assigned a globally unique identifier upon generation, and its top left corner row and column coordinates, width in pixels, and height in pixels are recorded simultaneously to form the metadata structure of the block. Next, all frames are traversed along the time axis, and the pixel grayscale values ​​of the corresponding spatial locations are extracted based on the block identifiers and arranged in chronological order to form a one-dimensional temporal signal. The temporal signals of all blocks are centrally stored as a temporal signal set, serving as a unified data source for subsequent normalization and network input.

[0035] Preferably, in one embodiment of the present invention, the above-mentioned acquisition of a multi-scale block set based on at least two pre-set spatial grids of different pixel sizes includes: acquiring a pre-set first-scale grid, a second-scale grid, and a third-scale grid; wherein the first scale is smaller than the second scale, and the second scale is smaller than the third scale; selecting at least two scales of grids from the first-scale grid, the second-scale grid, and the third-scale grid based on the water flow turbulence level and / or a preset monitoring accuracy index to acquire a target-scale grid; generating a uniform grid according to the corresponding pixel scale for each pixel scale of the target-scale grid, with the upper left corner of the image as the origin, to obtain a grid instance corresponding to each pixel scale; configuring a globally unique identifier for each cell in each grid instance, and recording the starting row and column coordinates, width pixel value, and height pixel value of the cell in the image coordinate system to construct a block metadata structure; calculating the spatial intersection-union ratio (CIU) of any two cells in grid instances of different pixel scales, and if the CIU exceeds a preset CIU threshold, retaining the cell with the larger area and discarding the cell with the smaller area to acquire a multi-scale block set.

[0036] It should be noted that when constructing a multi-scale block set, a three-level scale dictionary can be pre-maintained in memory, corresponding to fine, medium, and coarse spatial resolutions. The fine scale captures micro-scale turbulent textures, the medium scale considers local velocity gradients, and the coarse scale depicts the overall flow trend. During runtime, based on real-time acquired quantitative indicators of water flow turbulence or user-preset monitoring accuracy requirements, two or three levels of scale are strategically selected for the target scale set, thus achieving a balance between computational resources and information richness. Subsequently, a pixel coordinate system is established with the top-left corner of the image as the origin. Each selected scale is uniformly stepped along the row and column directions according to the pixel size corresponding to that scale, quickly generating non-overlapping grid instances. Each cell of the grid instance is then assigned a globally unique identifier, and its top-left row and column coordinates and width and height pixel values ​​are registered in the metadata record, forming a traceable block identity. To avoid redundant spatial coverage between grid instances of different scales, the spatial intersection-union ratio between any two cells is further calculated: when this ratio exceeds a preset threshold, the smaller cell is automatically removed, while the larger cell is retained. After this conflict resolution mechanism, a multi-scale block set that is both non-overlapping and covers the entire field of view is finally obtained, providing an accurate and non-redundant spatial basis for subsequent extraction of block-level time-series signals along the time axis.

[0037] Step S150: Normalize the timing signal to obtain a preprocessed signal.

[0038] Preferably, in one embodiment of the present invention, step S150 may include: traversing the time series signals corresponding to each block, calculating the arithmetic mean and standard deviation of the time series signals at all time steps; subtracting the arithmetic mean from the value of each time step of the time series signal to obtain a zero-mean signal; dividing the value of each time step of the zero-mean signal by the standard deviation to obtain a unit variance signal; if the standard deviation is lower than a preset lower threshold, setting the time series signal of the block to zero or filling it with a preset constant; obtaining the normalized time series signal of each block and constructing a preprocessed signal.

[0039] It should be noted that, to transform the original time-series signals of each block into preprocessed signals with a uniform numerical distribution suitable for network input, the system first traverses the complete time series corresponding to each block, calculating its arithmetic mean and standard deviation over all time steps. Then, the arithmetic mean is subtracted from each value in the sequence to obtain a centered signal with a mean of zero. Next, each centered value is divided by its corresponding standard deviation to normalize the variance, resulting in a unit variance signal with a mean of zero and a variance of one. If the standard deviation is below a preset lower threshold, to avoid division by zero or numerical instability, all time-series signals of that block are directly set to zero or filled with predefined constant values ​​to mark invalid blocks and prevent abnormal input. After the above operations, the normalized time-series signals of all blocks are integrated to form a set of preprocessed signals without redundancy and with consistent values, which can be directly used by the downstream one-dimensional temporal convolutional neural network.

[0040] It is understood that steps S120 to S150 above can be processed by the video frame preprocessing module. Please refer to [link / reference]. Figure 2 The following is a specific application example of the video frame preprocessing module in a certain application scenario, mainly including the following steps:

[0041] Step 1: Apply SIFT (Scale-Invariant Feature Transform) feature point detection to the original unaligned adjacent frames to obtain the key feature points of the corresponding frames:

[0042]

[0043]

[0044] in, This indicates the initial size of the Gaussian kernel, used to balance noise suppression and detail preservation; the default value is 1.6. Indicates the number of pyramid groups, covering a scale range of 1.6 to 6.4, used to adapt to different flow velocity structures on the river surface; the default value is 3. This indicates the number of layers in each pyramid group; the default value is 3.

[0045] It should be noted that SIFT feature point detection is a computer vision algorithm for stably extracting local feature points in an image. This algorithm is a relatively mature and well-known technology in the field. For its specific implementation, please refer to the relevant technologies. The embodiments of this invention will not be described in detail here.

[0046] Step 2: Use FLANN (Fast Library for Approximate Nearest Neighbors) or a brute-force matcher to match the SIFT descriptors of the two frames, generating a preliminary set of matching pairs:

[0047]

[0048] Then, RANSAC (Random Sample Consensus) is used for iterative optimization to maximize the number of matching points retained, resulting in the optimized affine transformation matrix. RANSAC parameters include: number of iterations: 1000, interior point threshold: 3 pixels.

[0049] It should be noted that FLANN is an open-source algorithm library for efficient approximate nearest neighbor search, specifically designed for fast matching of high-dimensional feature vectors. Its core function is to accelerate the matching process of feature point descriptors through approximate search strategies (such as KD-trees, hierarchical clustering trees, and other data structures), reducing computation time while maintaining accuracy. In this invention, FLANN acts as a feature point matcher, used to quickly pair SIFT descriptor vectors of adjacent frames, thereby generating an initial set of matching point pairs, providing an efficient and robust data foundation for subsequent RANSAC optimization and geometric transformation matrix calculation. It is understood that FLANN is a relatively mature and well-known technology in this field; for its specific implementation, please refer to relevant technologies, and this invention will not elaborate further.

[0050] It should be further noted that the brute-force matcher is a very basic feature point matching algorithm. It calculates the pairwise distance for each element in two sets of feature descriptors, exhaustively searching for all possible pairings to find the matching point pair with the smallest distance or that meets a specific threshold condition. Since it does not use any acceleration data structures, its computational complexity is proportional to the square of the number of descriptors, making it suitable for scenarios with a small number of features or where matching accuracy is extremely high but real-time performance is not critical. In this invention, the brute-force matcher can be used as an alternative to FLANN to perform high-precision pairing of SIFT descriptors in adjacent frames to obtain an initial set of matching point pairs. It is understood that the brute-force matcher is a relatively mature and well-known technology in this field; its specific implementation can be found in related technologies, and will not be elaborated upon in this embodiment.

[0051] It should be further noted that RANSAC iterative optimization is a robust parameter estimation algorithm used to estimate the optimal geometric transformation model from a set of matching point pairs containing noise and outliers. It is understood that RANSAC iterative optimization is a well-established and mature technology in this field; for its specific implementation, please refer to relevant technologies, and the embodiments of this invention will not elaborate further.

[0052] Step 3: Using the affine transformation matrix To achieve frame alignment, see the following formula:

[0053]

[0054] in, Indicates the aligned first frame; Indicates the first The original image of the frame; It is a linear transformation matrix. For translation vectors, all are derived from... supply.

[0055] Step 4: Obtain the aligned frames Then, texture enhancement is performed;

[0056] Texture enhancement can be achieved using the CLAHE (Contrast Limited Adaptive Histogram Equalization) method, which clips and limits the grayscale histogram within each 8×8 block, as shown in the following formula:

[0057]

[0058] in, This indicates a limit on the local histogram clipping range to prevent excessive noise amplification; in this scenario, it will... Set to 2.0; This indicates the block size, used to balance local contrast enhancement and computational efficiency. In this scene, it will... Set it to 8×8.

[0059] It should be noted that the CLAHE method described above is a local histogram enhancement technique, which is a well-known and mature technique in the field. For its specific implementation, please refer to the relevant technologies. The embodiments of this invention will not be described in detail here.

[0060] Step 5: The spatiotemporal dynamic information of the video frames needs to be converted into an input format suitable for temporal convolutional networks. Therefore, for the original video frame sequence... The image enhanced by CLAHE is It is divided into S×S blocks (S∈{4,8,16}), and the spatiotemporal slice signal of each block is... Defined as:

[0061]

[0062] in, The block size is indicated by a multi-scale design (4×4 captures detailed turbulence, 16×16 reflects the mainstream trend). This is the coordinate index of the region block, the range of which is determined by the video resolution (e.g., a 640×480 video). ).

[0063] Step 6: Eliminate the influence of illumination variations on the time-series signal, and standardize the time-series signal of each block to obtain the network input tensor. :

[0064]

[0065] in, This represents the average value of the block's time-series signal. The standard deviation of the block timing signal is represented.

[0066] It should be noted that the above scheme effectively suppresses water surface reflection, wave noise, and floating object interference through frame alignment (affine transformation matrix optimization) and CLAHE texture enhancement technology (limiting the local histogram clipping amplitude). CLAHE's dynamic equalization processing of the grayscale histogram within the block (such as 8×8 block design) can enhance local texture contrast while avoiding excessive noise amplification. Combined with multi-scale block segmentation (from 4×4 to 16×16 pixel blocks), it preserves turbulent details (small blocks) while capturing the mainstream trend (large blocks), solving the failure problem of traditional optical flow methods caused by dynamic texture blurring.

[0067] Step S160: Input the preprocessed signal into the pre-trained temporal one-dimensional convolutional neural network model to obtain the river surface velocity output by the temporal one-dimensional convolutional neural network model.

[0068] Please see Figure 3 Preferably, in one embodiment of the present invention, the temporal one-dimensional convolutional neural network model includes a first convolutional block, a second convolutional block, a third convolutional block, a global average pooling layer, and a fully connected layer connected in sequence; the first convolutional block, the second convolutional block, and the third convolutional block each include multiple one-dimensional convolutional layers;

[0069] Step S160 above may include: inputting the preprocessed signal into a first convolutional block to obtain a primary temporal feature map; inputting the primary temporal feature map into a second convolutional block to obtain a secondary temporal feature map; inputting the secondary temporal feature map into a third convolutional block to obtain a high-level temporal feature map; inputting the high-level temporal feature map into a global average pooling layer that performs mean calculation along the time dimension to obtain a two-dimensional feature vector; and inputting the two-dimensional feature vector into a fully connected layer to obtain the initial flow rate value corresponding to each block. An example of this implementation is:

[0070] Receive spatiotemporal slice signal [N] blocks [,1,T], where N blocks This represents the total number of video frame blocks, calculated as (H / S) × (W / S). For example:

[0071] Assuming the video resolution is 640×480, then the block size S = 16 → (480 / 16)×(640 / 16) = 30×40 = 1200. T represents the time step, which defaults to 120 steps, corresponding to a 4-second video at 30 FPS.

[0072] The aforementioned one-dimensional temporal convolutional neural network models include:

[0073] The first convolutional block contains 3 stacked convolutional layers with a kernel size of 3, a stride of 1, and 64 channels from input 1 to output, resulting in a feature map with a size of 1200×64×120.

[0074] The second convolutional block contains 5 stacked convolutional layers with a kernel size of 3, a stride of 1, and the number of channels from 64 inputs to 128 outputs, resulting in a feature map with a size of 1200×128×120.

[0075] The third convolutional block contains 3 stacked convolutional layers with a kernel size of 3, a stride of 1, and the number of channels increases from 64 to 128, resulting in a feature map with a size of 1200×256×120.

[0076] A global average pooling layer is used to obtain the output by taking the average value along the time dimension. :

[0077]

[0078] in, This represents the output feature map of the third convolutional block. The output size ranges from 1200×256×1 to 1200×256, compressing the time dimension while preserving spatial features.

[0079] A fully connected layer is used to map the features of each block to the predicted flow rate. The fully connected neuron unit is set to [256,1] and the output size is 1200×1.

[0080] It should be noted that the above scheme uses a one-dimensional convolutional network (1D-CNN) to replace the traditional optical flow method or complex three-dimensional convolution, reducing the number of parameters while improving inference speed. Under the same hardware conditions, the river surface velocity identification method provided in this embodiment of the invention can process 640×640 resolution video at a frame rate of up to 30 FPS, meeting the real-time requirements of hydrological monitoring, and its computational complexity is also lower than that of the optical flow method.

[0081] Preferably, in one embodiment of the present invention, the above-mentioned river surface velocity identification method may further include: for each block, reading the width pixel value and height pixel value in the block metadata structure, and determining the actual pixel area of ​​the block based on the width pixel value and height pixel value; determining the normalized weight coefficient of the current block based on the sum of the actual pixel areas of all blocks and the actual pixel area of ​​the current block; determining the weighted velocity value based on the normalized weight coefficient and the initial velocity value; and summing the weighted velocity values ​​of all blocks to obtain the velocity value of the river surface. For example, this implementation calculates the weight based on the actual coverage area of ​​the blocks, including cases where the block sizes are uneven. The calculation method is as follows:

[0082]

[0083] in, Indicates the first The area of ​​the block Indicates the first Divide into blocks and output the final flow rate. , This represents the total number of blocks.

[0084] The training method for the above-mentioned one-dimensional temporal convolutional neural network model is described below:

[0085] Preferably, in one embodiment of the present invention, the above-mentioned river surface velocity identification method may further include: acquiring historical river velocity data and historical video frame sequences; establishing a timestamp association between the historical velocity data at the same moment and its corresponding historical video frame sequences to obtain training samples; randomly occluding a portion of the temporal signal set in the training samples according to a preset ratio to simulate missing information; inputting the occluded temporal signal into a temporal one-dimensional convolutional neural network model, calculating the error between the velocity value output by the temporal one-dimensional convolutional neural network model and the actual velocity; iteratively updating the network weights of the temporal one-dimensional convolutional neural network model using a backpropagation algorithm until the verification error stably converges within a continuous preset period, thereby obtaining a trained temporal one-dimensional convolutional neural network model. Examples of this implementation include:

[0086] The UAV is equipped with a Doppler radar current meter to emit electromagnetic waves perpendicular to the water surface and calculate the surface flow velocity V. real Measurement error <0.05m / s, while simultaneously acquiring river video. Each video contains frame The Canny edge detection algorithm is used to select regions with significant flow velocity characteristics. :

[0087]

[0088] in, This represents the edge strength, calculated using the Sobel operator. As an adaptive threshold, it is generally adjusted within the range of 0.5 to 0.8.

[0089] Then, the selected ROI region is preprocessed using the aforementioned video frame preprocessing module to obtain the segmented timing signal.

[0090] Add random masking data augmentation, i.e. randomly occluding 20% ​​of the block area (simulating the occlusion of floating objects on the water surface) during model training, to optimize the error between the predicted flow velocity value output by the model and the actual measured flow velocity value;

[0091] The trained model is then validated in real-world scenarios.

[0092] Preferably, in one embodiment of the present invention, the above-mentioned inputting the occluded time-series signal into a one-dimensional temporal convolutional neural network model and calculating the error between the flow velocity value output by the one-dimensional temporal convolutional neural network model and the actual flow velocity includes: inputting the occluded time-series signal into a one-dimensional temporal convolutional neural network model and calculating the error between the flow velocity value output by the one-dimensional temporal convolutional neural network model and the actual flow velocity using the Huber loss function.

[0093] It should be noted that the Huber loss function described above is a robust loss function for regression tasks. By defining it piecewise, it exhibits L2 (squared error) characteristics when the residuals are small and transforms into L1 (absolute error) characteristics when the residuals are large, thus balancing insensitivity to outliers and convergence efficiency. It is understood that the Huber loss function is a relatively mature and well-known technology in this field; for its specific implementation, please refer to relevant technologies, and this embodiment will not elaborate further.

[0094] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings are intended to cover non-exclusive inclusion. In the description of embodiments of the invention, technical terms such as "first," "second," etc., are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, particular order, or primary or secondary relationship of the indicated technical features. In the description of embodiments of the invention, "a plurality of" means two or more, unless otherwise expressly and specifically defined. The reference to "embodiment" herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0095] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying surface flow velocities in a river, characterized by, The method comprises: obtaining a frame sequence of river video; performing spatial alignment processing on the frame sequence; performing texture enhancement processing on the frame sequence after the spatial alignment processing; dividing the frame sequence after the texture enhancement processing into multiple blocks, and extracting a time sequence signal of each block; performing normalization processing on the time sequence signal to obtain a pretreatment signal; inputting the pretreatment signal into a pre-trained time sequence one-dimensional convolutional neural network model to obtain a river surface flow rate output by the time sequence one-dimensional convolutional neural network model; The time sequence one-dimensional convolutional neural network model comprises a first convolutional block, a second convolutional block, a third convolutional block, a global average pooling layer and a full connection layer connected in sequence; the first convolutional block, the second convolutional block and the third convolutional block each comprise multiple one-dimensional convolutional layers; The method further comprises: for each block, reading a width pixel value and a height pixel value in a block metadata structure, and determining an actual pixel area of the block based on the width pixel value and the height pixel value; determining a normalization weight coefficient of the current block based on the sum of the actual pixel areas of all the blocks and the actual pixel area of the current block; determining a weighted flow rate value based on the normalization weight coefficient and the initial flow rate value; and accumulating and summing the weighted flow rate values of all the blocks to obtain a flow rate value of the river surface. The spatial alignment processing on the frame sequence comprises:

2. The river surface flow velocity identification method according to claim 1, characterized by, detecting key points in each of two adjacent frames of the frame sequence and generating corresponding description vectors; pairing each description vector to obtain an initial matching point pair set; iteratively screening the initial matching point pairs in the initial matching point pair set, calculating an optimal geometric transformation matrix and retaining an inlier point satisfying the optimal geometric transformation matrix; performing affine mapping on pixel coordinates of the whole frame image based on the optimal geometric transformation matrix to generate a corrected image consistent with the spatial position of a reference frame, and completing inter-frame alignment. The texture enhancement processing on the frame sequence after the spatial alignment processing comprises:

3. The river surface flow velocity identification method according to claim 1, characterized by, dividing each frame image in the frame sequence after the spatial alignment processing into multiple non-overlapping local image blocks; for each local image block, obtaining a gray level distribution of the local image block and generating a local histogram; clipping an upper end of the local histogram based on a preset clipping threshold; ​ The number of pixels to be cropped is evenly distributed to each gray level to obtain a corrected histogram; A gray scale mapping function is constructed based on the corrected histogram, and each pixel in the local image block is subjected to a gray scale transformation; All the local image blocks subjected to the gray scale transformation are reassembled according to the original spatial position to obtain an entire frame of image subjected to texture enhancement.

4. The river surface flow velocity identification method according to claim 1, characterized by, The frame sequence subjected to the texture enhancement processing is divided into a plurality of blocks, and a time sequence signal of each block is extracted, including: Based on the pre-set spatial grid of at least two different pixel sizes, a multi-scale block set is obtained; Based on the multi-scale block set, each frame image in the frame sequence subjected to the texture enhancement processing is subjected to grid cutting to obtain a plurality of blocks; Each block is configured with a unique block identifier, and the spatial coordinate range of the block in the original image is recorded; The pixel values of the same block identifier in all frames are read along the time dimension to construct a time sequence signal corresponding to the block; Based on the time sequence signal, a block-level time sequence signal set of all the blocks is obtained.

5. The river surface flow velocity identification method according to claim 4, characterized by, The multi-scale block set is obtained based on the pre-set spatial grid of at least two different pixel sizes, including: A first scale grid, a second scale grid and a third scale grid are obtained; wherein the first scale is smaller than the second scale, and the second scale is smaller than the third scale; Based on the water flow turbulence level and / or the pre-set monitoring accuracy index, at least two scale grids are selected from the first scale grid, the second scale grid and the third scale grid to obtain a target scale grid; Based on each pixel scale of the target scale grid, a uniform grid is generated from the top left corner of the image as the origin according to the corresponding pixel scale to obtain a grid instance corresponding to each pixel scale; Each cell in each grid instance is configured with a globally unique identifier, and the starting row and column coordinates, the width pixel value and the height pixel value of the cell in the image coordinate system are recorded to construct a block metadata structure; The spatial intersection ratio of any two cells in the grid instances of different pixel scales is calculated, if the intersection ratio exceeds a pre-set intersection ratio threshold, the cell with a larger area is retained, and the cell with a smaller area is removed to obtain a multi-scale block set.

6. The river surface flow velocity identification method according to claim 1, characterized by, The time sequence signal is normalized to obtain a pre-processed signal, including: Each time step value of the time sequence signal is subtracted by the arithmetic mean value to obtain a zero-mean signal; Each time step value of the zero-mean signal is divided by the standard deviation to obtain a unit variance signal; If the standard deviation is lower than a pre-set lower threshold, the time sequence signal of the block is set to zero or filled with a pre-set constant; The normalized time sequence signal of each block is obtained to construct a pre-processed signal. The method further includes:

7. The river surface flow velocity identification method according to any one of claims 1 to 6, characterized by, Historical flow rate data and historical video frame sequences of the river are obtained; ​ The historical flow rate data at the same moment is time-stamped associated with the corresponding historical video frame sequence, and a training sample is obtained; The time sequence signal set in the training sample is randomly occluded by a preset proportion to occlude part of the block signal to simulate missing information; The occluded time sequence signal is input into the time sequence one-dimensional convolutional neural network model, and the error between the flow rate value output by the time sequence one-dimensional convolutional neural network model and the true flow rate is calculated; The network weight of the time sequence one-dimensional convolutional neural network model is iteratively updated by using a back propagation algorithm until the verification error is stably converged in a continuous preset period, and a trained time sequence one-dimensional convolutional neural network model is obtained.

8. The river surface flow velocity identification method according to claim 7, characterized by, The occluded time sequence signal is input into the time sequence one-dimensional convolutional neural network model, and the error between the flow rate value output by the time sequence one-dimensional convolutional neural network model and the true flow rate is calculated, comprising: The occluded time sequence signal is input into the time sequence one-dimensional convolutional neural network model, and the error between the flow rate value output by the time sequence one-dimensional convolutional neural network model and the true flow rate is calculated by using a Huber loss function.

Images