A video flow measurement method and system based on texture clipping and double-flow fusion network

Abstract

The application discloses a video flow measurement method and system based on texture clipping and a double-flow fusion network, and comprises the following steps: acquiring a video sequence of a water surface to be measured, generating a space-time graph according to the video sequence, performing texture clipping on the space-time graph to obtain a space-time clipping graph; inputting the space-time clipping graph into a double-flow fusion network, wherein the double-flow fusion network comprises a spatial domain branch for extracting local texture features and a Hough global branch for extracting global arrangement features; performing fusion processing on the local texture features and the global arrangement features to obtain fusion features; performing decoding processing on the fusion features to obtain a continuous water flow direction angle, and calculating a water surface flow rate based on the continuous water flow direction angle and space sampling intervals and time sampling intervals corresponding to the space-time graph. The application can accurately identify texture directionality information in the space-time graph, and significantly improve the accuracy and stability of video flow measurement in a complex scene.

Description

Technical Field

[0001] This invention belongs to the field of video flow measurement technology, specifically relating to a video flow measurement method and system based on texture clipping and dual-stream fusion network. Background Technology

[0002] Current velocity monitoring is a fundamental technical means in water resource management, flood control and disaster reduction, water ecological protection, and river operation management. Traditional methods such as contact current meters, buoy methods, or radar velocity measurement have drawbacks such as high deployment costs, strong dependence on on-site conditions, and difficulty in long-term continuous deployment. Video-based flow measurement methods based on visual perception have significant advantages such as being non-contact, flexible in deployment, and capable of remote continuous monitoring, and have gradually become an important technical direction in the field of intelligent hydrological monitoring.

[0003] However, existing video flow measurement methods still face the following technical bottlenecks in practical applications: First, the spatiotemporal map preprocessing stage often uses fixed-size scaling operations, which easily destroys the directional structural information of the texture stripes, leading to systematic biases in subsequent angle estimation; Second, existing methods mostly use a single network structure for feature extraction, making it difficult to simultaneously capture local water surface texture direction features and global fluctuation arrangement features, resulting in insufficient adaptability to complex flow states; Third, the water flow direction angle has a periodic boundary characteristic from 0° to 180°, and traditional regression methods or ordinary classification methods are prone to prediction jumps or gradient misleading at the periodic boundaries, seriously affecting the convergence of model training and prediction accuracy.

[0004] Therefore, there is an urgent need for a video flow measurement method and system that can extract local texture features and global arrangement features while effectively preserving spatiotemporal texture information, and continuously and stably decode periodic angles. Summary of the Invention

[0005] The purpose of this invention is to provide a video flow measurement method and system based on texture clipping and dual-stream fusion networks, so as to solve the technical problems in the prior art where spatiotemporal map preprocessing destroys texture structure, a single network cannot simultaneously capture local and global features, and periodic angle prediction jumps.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A video flow measurement method based on texture cropping and dual-stream fusion network includes the following steps:

[0008] A video sequence of the surface of the water body to be tested is acquired, a spatiotemporal map is generated based on the video sequence, and texture clipping is performed on the spatiotemporal map to obtain a spatiotemporal clipped map.

[0009] The spatiotemporal clipping map is input into a two-stream fusion network, which includes a spatial branch for extracting local texture features and a Hough global branch for extracting global permutation features.

[0010] The local texture features and the global arrangement features are fused together to obtain the fused features;

[0011] The fused features are decoded to obtain the continuous water flow direction angle. Based on the continuous water flow direction angle and the spatial and temporal sampling intervals corresponding to the spatiotemporal map, the surface velocity of the water body is calculated.

[0012] Optionally, the texture clipping includes:

[0013] Multiple candidate regions are set in the spatiotemporal graph, and a corresponding structure tensor is constructed based on the horizontal and vertical gradients of the candidate regions.

[0014] The structural tensor is used to calculate the principal direction of texture, direction consistency, and texture intensity of the candidate region, and the texture score of the candidate region is generated by combining the direction weighting information.

[0015] The candidate region with the highest texture score is used as the clipping region to clip the spatiotemporal map, resulting in the spatiotemporal clipped map.

[0016] Optionally, the direction weighting information includes:

[0017] The direction weighting information is calculated based on the main direction of the texture. The direction weighting information is used to suppress the influence of candidate regions whose main direction of the texture is close to horizontal or vertical on the cropping region determination process.

[0018] Optionally, the Hough global branch includes:

[0019] Extract the edge features of the spatiotemporal cropping map to generate an edge feature map;

[0020] Layered sampling is performed on the edge points in the edge feature map to obtain sampled edge points;

[0021] A learnable voting mechanism is used to map the sampled edge points to obtain an angle space feature map;

[0022] Sequence modeling is performed on the angular space feature map to output the global permutation features.

[0023] Optionally, the learnable voting mechanism includes:

[0024] The sampled edge points are point-encoded, and voting weights are calculated based on the relationship between the point codes and angle codes. The sampled edge points are then mapped to the angle space according to the voting weights to form the angle space feature map.

[0025] Optionally, the fusion process includes:

[0026] The local texture features and the global arrangement features are dimensionally aligned and stitched together;

[0027] Gating coefficients representing the contribution ratio of two types of features are generated using a gating network, and the fused features are obtained based on the gating coefficients.

[0028] Optionally, the dimension alignment includes:

[0029] At least one of the following methods is used: linear transformation, convolutional transformation, upsampling, or downsampling, to map the local texture features and the global arrangement features to the same feature dimension and spatial scale.

[0030] Optionally, the decoding process includes:

[0031] A periodic angle probability distribution is generated based on the fusion features, and an initial predicted angle is obtained based on the periodic angle probability distribution.

[0032] Centered on the initial predicted angle, the periodic angle probability distribution is weighted by distance, and the weighted periodic angle probability distribution is processed by distance-weighted circumferential expectation decoding to obtain the continuous water flow direction angle.

[0033] Optionally, the method further includes:

[0034] A composite loss function is constructed using divergence loss and periodic absolute value loss to train the dual-stream fusion network.

[0035] This invention also provides a video flow measurement system based on texture clipping and dual-stream fusion networks, comprising the following modules:

[0036] The spatiotemporal cropping module is used to acquire a video sequence of the surface of the water body to be tested, generate a spatiotemporal map based on the video sequence, and perform texture cropping on the spatiotemporal map to obtain a spatiotemporal cropped map.

[0037] The feature extraction module is used to input the spatiotemporal cropping map into a dual-stream fusion network, which includes a spatial branch for extracting local texture features and a Hough global branch for extracting global permutation features.

[0038] The feature fusion module is used to fuse the local texture features and the global arrangement features to obtain fused features;

[0039] The result output module is used to decode the fused features to obtain the continuous water flow direction angle. Based on the continuous water flow direction angle and the spatial sampling interval and time sampling interval corresponding to the spatiotemporal map, the surface flow velocity of the water body is calculated.

[0040] The advantages of this invention compared to existing methods are:

[0041] (1) This invention replaces the traditional fixed-size scaling preprocessing with a texture clipping mechanism. Based on the structural tensor, it calculates the principal direction, directional consistency, and texture intensity of the candidate region, and combines directional weighted information to perform texture scoring, automatically selecting the region with the richest texture directional information as the clipping region. This method effectively preserves the texture stripe structure, which is most critical for estimating the direction of water flow in the spatiotemporal image, and avoids the destruction of directional information by fixed scaling operations.

[0042] (2) This invention constructs a dual-flow fusion network architecture with parallel spatial branch and Hough global branch. The spatial branch focuses on extracting local texture features and capturing subtle changes in the flow direction texture of the water surface; the Hough global branch maps edge points to angle space and performs sequence modeling through a learnable voting mechanism, extracting global arrangement features and capturing the macroscopic motion law of water flow. The dual-flow architecture achieves complementary extraction of local and global features, significantly enhancing the adaptability to complex flow states.

[0043] (3) The present invention uses distance-weighted circular expectation decoding to decode the fused features. It uses the initial prediction angle as the center to perform distance weighting on the periodic angle probability distribution, which effectively solves the problem of prediction jump of the water flow direction angle at the periodic boundary, realizes continuous and stable angle estimation, and avoids the gradient misleading at the boundary of the traditional regression method. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating a video flow measurement method based on texture clipping and dual-stream fusion network in an embodiment of the present invention.

[0046] Figure 2 This is a schematic diagram of the texture trimming process in an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of the architecture of the dual-stream fusion network in an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram illustrating the principle of Hough global branching in an embodiment of the present invention;

[0049] Figure 5 This is a schematic diagram illustrating the principle of distance-weighted circumferential expectation decoding in an embodiment of the present invention;

[0050] Figure 6 This is a schematic diagram of the structure of a video flow measurement system based on texture clipping and dual-stream fusion network in an embodiment of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention more apparent and understandable, the technical solutions of this invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0052] Please see Figure 1 As shown, the present invention provides a video flow measurement method based on texture clipping and dual-stream fusion network, comprising the following steps:

[0053] S1. Obtain a video sequence of the surface of the water body to be tested, generate a spatiotemporal map based on the video sequence, and perform texture clipping on the spatiotemporal map to obtain a spatiotemporal clipped map.

[0054] Step S1 in this embodiment of the invention specifically includes:

[0055] S11. Based on the video sequence of the surface of the water body to be tested acquired by the camera equipment, the grayscale value of the video sequence is extracted frame by frame according to the preset spatial sampling line to generate a spatiotemporal image.

[0056] Specifically, by using camera equipment installed above or on the bank of the water body, and employing a fixed viewing angle and frame rate, a video sequence of the surface of the water body to be measured is continuously acquired. The video sequence contains multiple consecutive water surface images I(x,y,t), where (x,y) represents spatial coordinates and t represents frame time.

[0057] In the video sequence, a sampling line L is set along the direction perpendicular to the water flow. Pixel intensity values ​​of different water surface images are extracted frame by frame along the sampling line, and the pixel intensity values ​​are stitched together from top to bottom in chronological order to obtain a two-dimensional spatiotemporal map.

[0058]

[0059] In the formula, u represents the spatial position coordinates along the sampling line L, which serves as the horizontal axis of the spatiotemporal graph; τ represents the frame time coordinates, which serve as the vertical axis of the spatiotemporal graph.

[0060] The grayscale value of each pixel in the spatiotemporal graph reflects the water surface brightness information at that spatial location within the corresponding frame time. Therefore, when there is water flow, the water surface texture presents a directional stripe structure in the spatiotemporal graph, and the tilt direction of the stripes corresponds to the direction of water flow.

[0061] S12. Set candidate regions and construct a structure tensor. Calculate the texture score of the candidate regions based on the structure tensor. Select the candidate region with the highest texture score as the clipping region and perform clipping processing on the spatiotemporal graph to obtain a spatiotemporal clipped graph.

[0062] Please see Figure 2 As shown, step S12 in this embodiment of the invention specifically includes:

[0063] S121. Set up candidate regions using a sliding window method, and construct a structure tensor by calculating the horizontal and vertical gradients of the candidate regions.

[0064] Different candidate regions are set in the spatiotemporal graph using a sliding window approach. For each pixel (u, τ) in the candidate region, its horizontal gradient, vertical gradient, and structure tensor are calculated using the Sobel operator.

[0065]

[0066]

[0067]

[0068] In the formula, S u S represents the horizontal gradient; τ G represents the vertical gradient; G represents the structure tensor; ω represents the vertical gradient. c (u,τ) represents the Gaussian weight function with the center of the candidate region as the origin, and its expression is:

[0069]

[0070] In the formula, (u c ,τ c ) represents the center coordinates of the candidate region, σ c This represents the Gaussian kernel standard deviation.

[0071] S122. Perform feature decomposition on the structure tensor, calculate the main texture direction, direction consistency and texture intensity of the candidate region, introduce direction weighting information, and obtain the texture score of the candidate region.

[0072] Eigenvalues ​​λ1 and λ2 are obtained by performing eigenvalue decomposition on the structure tensor, where λ1 ≥ λ2 ≥ 0. The eigenvalues ​​can be expressed as:

[0073]

[0074] The texture principal direction, orientation consistency, and texture intensity of the candidate region are calculated using the eigenvalues. The texture principal direction reflects the dominant tilt direction of the texture stripes within the candidate region. The orientation consistency characterizes the concentration of texture orientations in the candidate region. The texture intensity characterizes the salience of the texture structure in the candidate region. Their expressions are as follows:

[0075]

[0076]

[0077]

[0078] In the formula, θ represents the main direction of the texture, C represents the direction consistency, and E represents the texture intensity.

[0079] Since texture stripes cannot effectively reflect water flow direction information when the main texture direction is close to horizontal (0° or 180°) or vertical (90°), direction weighting information is calculated based on the main texture direction. This information is used to suppress the influence of candidate regions with main texture directions close to horizontal or vertical on the cropping region determination process. The expression for the direction weighting information is as follows:

[0080]

[0081] In the formula, K represents the directional weighting information, and γ represents the directional attenuation coefficient.

[0082] Based on the main texture direction, the direction consistency, the texture intensity, and the direction-weighted information, a texture score for the candidate region is generated:

[0083]

[0084] In the formula, S core This represents the texture score corresponding to the candidate region.

[0085] S123. Based on the texture score, the candidate region with the highest texture score is used as the clipping region, and the spatiotemporal map is clipped to obtain a spatiotemporal clipped map.

[0086] Based on steps S121 and S122, texture scores corresponding to all candidate regions are calculated. The candidate region with the highest texture score is selected as the clipping region, which represents the region in the spatiotemporal map with the richest texture directionality information and is most conducive to water flow direction estimation. The spatiotemporal map is clipped according to the clipping region to obtain the spatiotemporal clipped map R(u,τ).

[0087] This step automatically filters and preserves the region with the richest texture directionality information in the spatiotemporal map through a texture clipping mechanism, effectively avoiding the destruction of the texture stripe directionality structure by traditional fixed-size scaling operations, and providing high-quality input for subsequent feature extraction and angle estimation.

[0088] S2. Input the spatiotemporal clipping map into a dual-stream fusion network, which includes a spatial branch for extracting local texture features and a Hough global branch for extracting global arrangement features.

[0089] Step S2 in this embodiment of the invention specifically includes:

[0090] Please see Figure 3 As shown, the dual-stream fusion network includes a parallel spatial branch and a Hough global branch. The spatiotemporal clipping map is synchronously input into the two branches for processing, and the local texture features and global arrangement features are output respectively.

[0091] S21. Using spatial branching, the spatiotemporal cropping map is sequentially processed by convolution, downsampling, and average pooling to extract local texture features.

[0092] The spatial branch employs a convolutional neural network architecture, including a 3×3 convolutional layer, a downsampling layer, and an average pooling layer. The convolutional layers progressively process the local texture features in the spatiotemporal cropping image, capturing the local direction of texture stripes and outputting a texture feature map. This texture feature map preserves the details of texture direction and edge information changes in the spatiotemporal cropping image. The texture feature map is then mapped sequentially through the downsampling layer and the average pooling layer to obtain the local texture feature F. local .

[0093] S22. Using Hough global branching, the spatiotemporal clipping map is sequentially subjected to edge feature extraction, hierarchical sampling, mapping processing, and sequence modeling to extract global permutation features.

[0094] Please see Figure 4 As shown, the Hough global branch includes an edge feature extraction module, a hierarchical sampling module, a mapping processing module, and a sequence modeling module, which are used to capture the macroscopic motion patterns of water flow.

[0095] S221. The edge features in the spatiotemporal cropping map are extracted by the edge feature extraction module to generate an edge feature map.

[0096] The edge features of the spatiotemporal clipping map R(u,τ) are extracted using the Canny operator to generate an edge feature map Z(u,τ), which contains significant texture boundaries and undulating contours in the spatiotemporal clipping map.

[0097] S222. The edge points in the edge feature map are sampled using the layered sampling module to obtain sampled edge points.

[0098] The edge points in the edge feature map are divided into layers according to edge strength, and random sampling is performed within each layer according to a preset ratio to obtain a set of sampled edge points:

[0099]

[0100] In the formula, p i =(u,τ) represents the coordinates of the i-th sampling edge point.

[0101] In this embodiment, the preset ratio is one-tenth of the total number of edge points in each layer. The hierarchical sampling process aims to preserve the global distribution characteristics while effectively reducing the computational load of subsequent processing.

[0102] S223. Introduce a voting learning mechanism to map the sampled edge points to obtain an angle space feature map.

[0103] Using a multilayer perceptron (MLP) for each sampled edge point p i Perform point encoding to obtain the point feature vector e p (i). Transform the range of texture direction angles [0°, 180°] into M discrete angle intervals. (e.g., α1 represents [0°, 10°)), and angle encoding is performed on each angle interval to obtain the angle feature vector e. a (j).

[0104] The similarity between the point feature vector and the angle feature vector is calculated to obtain the voting weight, which can be expressed as:

[0105]

[0106] In the formula, d ij Let b represent the voting weight, b represent the feature dimension, Softmax represent the normalization process along the angle dimension, and D represent the learnable projection matrix.

[0107] Based on the voting weights, the sampled edge points are mapped to the angle intervals to form an angle space feature map, i.e.:

[0108]

[0109] In the formula, S pace (j) represents the angle space feature map corresponding to the j-th angle interval; φ(p i ) represents the sampling edge point p i Edge strength characteristics.

[0110] S224. Perform sequence modeling on the angle space feature map and output global permutation features.

[0111] The angular space feature map is modeled using a Transformer encoder or a bidirectional long short-term memory network (Bi-LSTM) to capture the long-range dependencies and global permutation patterns between different angles, generating a global permutation feature F. global .

[0112] S3. The local texture features and the global arrangement features are fused to obtain fused features.

[0113] Step S3 in this embodiment of the invention specifically includes:

[0114] The local texture features and the global arrangement features are dimensionally aligned and concatenated. Then, a gating network is used to generate gating coefficients that represent the contribution ratio of the two types of features. Based on the gating coefficients, the local texture features and the global arrangement features are fused to obtain fused features.

[0115] S31. Using at least one of linear transformation, convolution transformation, upsampling or downsampling, map the local texture features and the global arrangement features to the same feature dimension and spatial scale, and then concatenate them to obtain concatenated features.

[0116] Since the feature dimensions and spatial scales output by the spatial domain branch and the Hough global branch may differ, the local texture features and the global arrangement features are first mapped to the same feature dimensions and spatial scale through at least one or more of linear transformation, convolutional transformation, upsampling, or downsampling. The mapped features are then concatenated to obtain the concatenated feature F. concat .

[0117] S32. Use a gating network to generate gating coefficients that represent the contribution ratio of the two types of features, and obtain the fused features based on the gating coefficients.

[0118] The gated network consists of two fully connected layers and a sigmoid activation function. Its input is the concatenated features, and its output is the gating coefficients, i.e.:

[0119]

[0120] In the formula, g represents the gating coefficient, which takes values ​​in the range of [0,1]; W1 and W2 both represent weight matrices; B1 and B2 both represent bias coefficients.

[0121] Based on the gating coefficient, the fusion feature is obtained:

[0122]

[0123] In the formula, F fuse This indicates the fusion feature. , These represent the local texture features and global arrangement features after feature mapping, respectively.

[0124] The gated network can adaptively adjust the contribution ratio of local details and global patterns, so that the fused features have both micro-texture sensitivity and macro-fluidity adaptability.

[0125] S4. Decode the fused features to obtain the continuous water flow direction angle. Based on the continuous water flow direction angle and the spatial sampling interval and time sampling interval corresponding to the spatiotemporal map, calculate the surface flow velocity of the water body.

[0126] Step S4 in this embodiment of the invention specifically includes:

[0127] S41. Generate a periodic angle probability distribution based on the fusion features, and obtain the initial predicted angle according to the periodic angle probability distribution.

[0128] A decoding network is constructed by using a fully connected layer and a Softmax activation function. The fused features are input into the decoding network to generate a periodic angular probability distribution η(β), where β∈[0°,180°), and the periodic probability distribution satisfies η(β)=η(β+180°).

[0129] Based on the periodic probability distribution, the angle corresponding to the highest probability is selected as the initial prediction angle β0.

[0130] S42. Using the initial predicted angle as the center, the periodic angle probability distribution is weighted by distance, and the weighted periodic angle probability distribution is processed by distance-weighted circumferential expectation decoding to obtain the continuous water flow direction angle.

[0131] Please see Figure 5 As shown, based on the periodic characteristics of the water flow direction angle, the angular distance function is defined as:

[0132]

[0133] In the formula, This represents the angular distance function.

[0134] Construct distance weight v using high and low functions. Its expression is:

[0135]

[0136] In the formula, σ v This represents the distance-weighted bandwidth parameter.

[0137] The weighted periodic angle probability distribution is processed using distance-weighted circular expectation decoding to obtain the continuous water flow direction angle. The circular expectation decoding is performed in the complex plane to avoid periodic jumps during angle averaging.

[0138]

[0139] In the formula, The arg() method represents the angle of the continuous water flow direction, and arg() represents the complex argument operation. This decoding method solves the problem of prediction jumps in the water flow direction angle at the periodic boundary, achieving continuous and stable angle estimation.

[0140] S43. Based on the angle of the continuous water flow direction and the spatial and temporal sampling intervals corresponding to the spatiotemporal diagram, the surface velocity of the water body is calculated:

[0141]

[0142] In the formula, v s Δu represents the surface flow velocity of the water body; Δu represents the spatial sampling interval, which is determined by the calibration relationship between the pixel spacing of the sampling lines and the actual physical distance; Δτ represents the temporal sampling interval, which is determined by the video frame rate.

[0143] In some embodiments, the present invention can also construct a composite loss function by using divergence loss and periodic absolute loss to perform end-to-end training on the dual-stream fusion network.

[0144] The divergence loss is used to measure the difference between the predicted periodic angular probability distribution η(β) and the true angular distribution η. r The difference between (β). In this embodiment, Kullback-Leibler (KL) divergence is used as the divergence loss, and its expression is:

[0145]

[0146] In the formula, L KL This represents the divergence loss.

[0147] The periodic absolute value loss is used to directly constrain the angle of the continuous water flow direction. Compared to the true angle θ rThe deviation between them is expressed as:

[0148]

[0149] In the formula, L per This represents the periodic absolute value loss.

[0150] The composite loss function L total The weighted sum of divergence loss and periodic absolute value loss:

[0151]

[0152] In the formula, μ is the balance coefficient, which is set to 0.5 in this embodiment.

[0153] During training, the Adam optimizer is used to iteratively update the network parameters. The initial learning rate is set to 1×10⁻⁶. −4 The batch size is 16, and a learning rate decay and early stopping mechanism are introduced. Training stops and the optimal model parameters are saved when the composite loss function fails to decrease for 10 consecutive rounds.

[0154] The following describes a video flow measurement system based on texture clipping and dual-stream fusion network provided by the present invention. The video flow measurement system based on texture clipping and dual-stream fusion network described below and the video flow measurement method based on texture clipping and dual-stream fusion network described above can be compared and referenced with each other.

[0155] Please see Figure 6 As shown, it includes a spatiotemporal cropping module 61, a feature extraction module 62, a feature fusion module 63, and a result output module 64, wherein:

[0156] The spatiotemporal cropping module is used to acquire a video sequence of the surface of the water body to be tested, generate a spatiotemporal map based on the video sequence, and perform texture cropping on the spatiotemporal map to obtain a spatiotemporal cropped map.

[0157] The feature extraction module is used to input the spatiotemporal cropped image into a two-stream fusion network, which includes a spatial branch for extracting local texture features and a Hough global branch for extracting global permutation features.

[0158] The feature fusion module is used to fuse the local texture features and the global arrangement features to obtain fused features.

[0159] The result output module is used to decode the fused features to obtain the continuous water flow direction angle. Based on the continuous water flow direction angle and the spatial sampling interval and time sampling interval corresponding to the spatiotemporal map, the surface flow velocity of the water body is calculated.

[0160] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of the present invention.

Claims

1. A video flow measurement method based on texture clipping and dual-stream fusion network, characterized in that, Includes the following steps: A video sequence of the surface of the water body to be tested is acquired, a spatiotemporal map is generated based on the video sequence, and texture clipping is performed on the spatiotemporal map to obtain a spatiotemporal clipped map. The spatiotemporal clipping map is input into a two-stream fusion network, which includes a spatial branch for extracting local texture features and a Hough global branch for extracting global permutation features. The local texture features and the global arrangement features are fused together to obtain the fused features; The fused features are decoded to obtain the continuous water flow direction angle. Based on the continuous water flow direction angle and the spatial and temporal sampling intervals corresponding to the spatiotemporal map, the surface velocity of the water body is calculated.

2. The video flow measurement method based on texture clipping and dual-stream fusion network according to claim 1, characterized in that, The texture clipping includes: Multiple candidate regions are set in the spatiotemporal graph, and a corresponding structure tensor is constructed based on the horizontal and vertical gradients of the candidate regions. The structural tensor is used to calculate the principal direction of texture, direction consistency, and texture intensity of the candidate region, and the texture score of the candidate region is generated by combining the direction weighting information. The candidate region with the highest texture score is used as the clipping region to clip the spatiotemporal map, resulting in the spatiotemporal clipped map.

3. The video flow measurement method based on texture clipping and dual-stream fusion network according to claim 2, characterized in that, The direction-weighted information includes: The direction weighting information is calculated based on the main direction of the texture. The direction weighting information is used to suppress the influence of candidate regions whose main direction of the texture is close to horizontal or vertical on the cropping region determination process.

4. The video flow measurement method based on texture clipping and dual-stream fusion network according to claim 1, characterized in that, The Hough global branch includes: Extract the edge features of the spatiotemporal cropping map to generate an edge feature map; Layered sampling is performed on the edge points in the edge feature map to obtain sampled edge points; A learnable voting mechanism is used to map the sampled edge points to obtain an angle space feature map; Sequence modeling is performed on the angular space feature map to output the global permutation features.

5. The video flow measurement method based on texture clipping and dual-stream fusion network according to claim 4, characterized in that, The learnable voting mechanism includes: The sampled edge points are point-encoded, and voting weights are calculated based on the relationship between the point codes and angle codes. The sampled edge points are then mapped to the angle space according to the voting weights to form the angle space feature map.

6. The video flow measurement method based on texture clipping and dual-stream fusion network according to claim 1, characterized in that, The fusion process includes: The local texture features and the global arrangement features are dimensionally aligned and stitched together; Gating coefficients representing the contribution ratio of two types of features are generated using a gating network, and the fused features are obtained based on the gating coefficients.

7. The video flow measurement method based on texture clipping and dual-stream fusion network according to claim 6, characterized in that, The dimension alignment includes: At least one of the following methods is used: linear transformation, convolutional transformation, upsampling, or downsampling, to map the local texture features and the global arrangement features to the same feature dimension and spatial scale.

8. The video flow measurement method based on texture clipping and dual-stream fusion network according to claim 1, characterized in that, The decoding process includes: A periodic angle probability distribution is generated based on the fusion features, and an initial predicted angle is obtained based on the periodic angle probability distribution. Centered on the initial predicted angle, the periodic angle probability distribution is weighted by distance, and the weighted periodic angle probability distribution is processed by distance-weighted circumferential expectation decoding to obtain the continuous water flow direction angle.

9. A video flow measurement method based on texture clipping and dual-stream fusion network according to claim 1, characterized in that, The method further includes: A composite loss function is constructed using divergence loss and periodic absolute value loss to train the dual-stream fusion network.

10. A video flow measurement system based on texture clipping and dual-stream fusion network, characterized in that, Includes the following modules: The spatiotemporal cropping module is used to acquire a video sequence of the surface of the water body to be tested, generate a spatiotemporal map based on the video sequence, and perform texture cropping on the spatiotemporal map to obtain a spatiotemporal cropped map. The feature extraction module is used to input the spatiotemporal cropping map into a dual-stream fusion network, which includes a spatial branch for extracting local texture features and a Hough global branch for extracting global permutation features. The feature fusion module is used to fuse the local texture features and the global arrangement features to obtain fused features; The result output module is used to decode the fused features to obtain the continuous water flow direction angle. Based on the continuous water flow direction angle and the spatial sampling interval and time sampling interval corresponding to the spatiotemporal map, the surface flow velocity of the water body is calculated.

Images