Methods, devices, equipment and media for detecting water surface velocity

Abstract

This disclosure relates to the field of image recognition technology, and discloses a method, apparatus, device, and medium for detecting water surface velocity. The method includes: acquiring multiple frames of image data to be detected; using a target detection network to extract features, fuse features, and predict the image data of the current frame to obtain a first target detection box corresponding to the image data of the current frame; performing feature extraction, feature fusion, and prediction on the image data of the next frame to obtain a second target detection box corresponding to the image data of the next frame; using a target tracking algorithm based on the first and second target detection boxes to update the historical trajectories of all water surface tracers in the current frame to obtain the target trajectories of all water surface tracers in the current frame; and obtaining the water surface velocity of the image data of the current frame based on the target trajectories. This disclosure improves the accuracy of water surface velocity detection by integrating an attention mechanism into the target detection network and using a target tracking algorithm to track water surface tracers.

Description

Technical Field

[0001] This disclosure relates to the field of image recognition technology, specifically to methods, devices, equipment, and media for detecting water surface flow velocity. Background Technology

[0002] Currently, the field of water surface velocity detection mainly relies on hardware devices such as current meters, which determine the water surface velocity by comparing the frequency difference between transmitted and reflected signals. This method of velocity measurement requires significant manpower and maintenance costs, and its measurement accuracy is greatly affected by environmental factors and equipment parameters.

[0003] Current techniques typically involve deploying surface tracer particles into the water flow and tracking their positions at different times, calculating the surface velocity based on their displacement and time variations. However, because single-type surface tracer particles are susceptible to errors caused by factors such as changes in illumination, water surface fluctuations, and background noise, it is necessary to use multiple types of surface tracer particles simultaneously for surface velocity calculation.

[0004] In recent years, with the rapid development of deep learning, image recognition methods based on convolutional neural networks (CNNs) have been widely applied and have shown significant advantages in various recognition fields such as object detection and semantic segmentation. However, when using CNNs to detect water surface tracer particles, the accuracy of the model often fluctuates in complex scenes due to environmental factors such as lighting, shadows, and complex backgrounds. Furthermore, although CNNs can detect the category and location of water surface tracer particles, they struggle to distinguish between particles of the same category, resulting in low detection accuracy. Summary of the Invention

[0005] In view of this, the present disclosure provides a method, apparatus, equipment and medium for detecting water surface velocity, in order to solve the problem of low accuracy in detecting water surface velocity.

[0006] In a first aspect, this disclosure provides a method for detecting water surface velocity, the method comprising:

[0007] Acquire multiple frames of image data to be detected, which include water surface tracers for water surface velocity detection;

[0008] The object detection network is used to extract, fuse, and predict features of the image data of the current frame to obtain the first object detection box corresponding to the image data of the current frame. The object detection network is used to extract, fuse, and predict features of the image data of the next frame to obtain the second object detection box corresponding to the image data of the next frame. The object detection network includes a feature extraction layer, an attention mechanism, a feature fusion layer, and a prediction layer. The attention mechanism is located between the feature extraction layer and the feature fusion layer.

[0009] Using a target tracking algorithm based on the first and second target detection boxes, the historical trajectories of all water surface tracers in the current frame are updated to obtain the target trajectories of all water surface tracers in the current frame;

[0010] The water surface flow velocity is obtained from the image data of the current frame based on the target trajectory.

[0011] In this embodiment, multiple frames of image data to be detected are acquired. These image data include surface tracers used for water surface velocity detection. A target detection network is used to extract, fuse, and predict features from the current frame's image data to obtain a first target detection box. The same network is then used to extract, fuse, and predict features from the next frame's image data to obtain a second target detection box. The target detection network includes a feature extraction layer, an attention mechanism, a feature fusion layer, and a prediction layer, with the attention mechanism located between the feature extraction and feature fusion layers. A target tracking algorithm is used to update the historical trajectories of all surface tracers in the current frame based on the first and second target detection boxes, obtaining the target trajectories of all surface tracers in the current frame. The water surface velocity of the current frame's image data is then obtained based on the target trajectories. Because this embodiment improves water surface velocity detection accuracy by fusing an attention mechanism in the target detection network and using a target tracking algorithm to track surface tracers, this method achieves higher accuracy.

[0012] In one optional implementation, the feature extraction layer includes multiple convolutional layers. The object detection network performs feature extraction, feature fusion, and prediction on the image data of the current frame to obtain a first object detection box corresponding to the image data of the current frame. The object detection network then performs feature extraction, feature fusion, and prediction on the image data of the next frame to obtain a second object detection box corresponding to the image data of the next frame, including:

[0013] Based on the feature extraction layer, feature extraction is performed on the image data of the current frame to obtain the first feature, and feature extraction is performed on the image data of the next frame to obtain the second feature;

[0014] The image data of the current frame is convolved by the convolutional layer to obtain the third feature, and the first network feature from the preset layer is selected from the third feature. The image data of the next frame is convolved to obtain the fourth feature, and the second network feature from the preset layer is selected from the fourth feature.

[0015] The first feature is weighted based on the attention mechanism to obtain the weighted first feature, and the second feature is weighted to obtain the weighted second feature.

[0016] Based on the feature fusion layer, the first network feature and the weighted first feature are fused to obtain the first fused feature, and the second network feature and the weighted second feature are fused to obtain the second fused feature;

[0017] The prediction layer detects the first fusion feature of the input and outputs the first target detection box corresponding to the image data of the current frame. The second fusion feature of the input is then detected, and the second target detection box corresponding to the image data of the next frame is output.

[0018] In this embodiment of the disclosure, by utilizing the feature extraction layer, attention mechanism, feature fusion layer and prediction layer in the target detection network, feature extraction, feature fusion and prediction are performed on the image data of the current frame and the image data of the next frame. This enables the accurate capture of the features of water surface tracers in the image data at different scales, obtaining accurate target detection boxes and improving the accuracy of water surface velocity detection.

[0019] In one optional implementation, the first feature is weighted based on an attention mechanism to obtain a weighted first feature, and the second feature is weighted to obtain a weighted second feature, including:

[0020] The first feature is pooled using an average pooling layer to obtain a first tensor, and the second feature is pooled to obtain a second tensor.

[0021] The first tensor is convolved in one dimension using a one-dimensional convolutional layer to obtain the third tensor, and the second tensor is convolved in one dimension to obtain the fourth tensor.

[0022] The third tensor is transformed to a preset interval using an activation function to obtain the first weight, and the fourth tensor is transformed to a preset interval to obtain the second weight.

[0023] The first feature and the first weight are fused to obtain the weighted first feature, and the second feature and the second weight are fused to obtain the weighted second feature.

[0024] In this embodiment of the disclosure, by weighting the first feature and the second feature based on an attention mechanism, the channel information of the first feature and the second feature can be fully utilized to enhance the fitting ability of the target detection network and improve the accuracy of water surface velocity detection.

[0025] In one optional implementation, a target tracking algorithm is used to update the historical trajectories of all water surface tracers in the current frame based on a first target detection box and a second target detection box, to obtain the target trajectories of all water surface tracers in the current frame, including:

[0026] Obtain the status information of the first target detection box;

[0027] Based on the state information, the trajectories of all water surface tracers in the current frame are predicted for the next frame, and the tracking boxes of all water surface tracers in the current frame for the next frame are obtained.

[0028] The second target detection box is matched with the tracking box in the next frame to obtain the matching result;

[0029] The target trajectory is obtained by updating the historical trajectories of all water surface tracers in the current frame based on the matching results.

[0030] In this embodiment of the disclosure, by using a target tracking algorithm to update the historical trajectories of all water surface tracers in the current frame, it is possible to continuously and stably track all water surface tracers, obtain accurate target trajectories, and improve the accuracy of water surface velocity detection.

[0031] In one optional implementation, the second target detection box is matched with the tracking box of the next frame to obtain a matching result, including:

[0032] Based on the number of historical uncorrelated frames of all surface tracers in the current frame, the state of the tracking box in the next frame is determined, resulting in a first tracking box with a determined state and a second tracking box with an undetermined state.

[0033] The first tracking box is matched with the second target detection box to obtain the first matching result. The first matching result includes the third tracking box and the fourth tracking box. The third tracking box is the tracking box that does not match the second target detection box, and the fourth tracking box is the tracking box that does not match the second target detection box when a new water surface tracer is detected.

[0034] The second tracking box, the third tracking box, and the fourth tracking box are matched in the second way to obtain the second matching result.

[0035] In this embodiment of the disclosure, by performing a first match and a second match between the second target detection box and the tracking box of the next frame, the accuracy and stability of tracking water surface tracers can be improved, and the accuracy of water surface velocity detection can be improved.

[0036] In an optional implementation, the first matching result further includes a fifth tracking box, which is a tracking box that successfully matches the second target detection box. The second matching result includes the third, fourth, and fifth tracking boxes. Based on the matching result, the historical trajectories of all water surface tracers in the current frame are updated to obtain the target trajectory, including:

[0037] The historical trajectories of all water surface tracers in the current frame are updated using the fifth tracking frame to obtain the trajectory of the first target.

[0038] Based on the number of historical unrelated frames of all surface tracers in the current frame corresponding to the third tracking box, the state of the third tracking box is determined, and the first and second tracking boxes are obtained.

[0039] The number of historical unrelated frames of the first tracking frame is compared with the preset number of frames to obtain the sixth and seventh tracking frames. The sixth tracking frame is the tracking frame with a number of historical unrelated frames greater than the preset number of frames, and the seventh tracking frame is the tracking frame with a number of historical unrelated frames less than the preset number of frames.

[0040] Using the fourth and seventh tracking boxes in the second matching result, new trajectories of all water surface tracers in the current frame are created to obtain the second target trajectory.

[0041] Delete the historical trajectories of all water surface tracers in the current frame corresponding to the second and sixth tracking frames;

[0042] The target trajectory is obtained based on the first target trajectory and the second target trajectory.

[0043] In this embodiment of the disclosure, by updating the historical trajectories of all surface tracers in the current frame based on the first matching result and the second matching result, it is possible to continuously and stably track all surface tracers, obtain accurate target trajectories, and improve the accuracy of surface velocity detection.

[0044] In one optional implementation, the water surface flow velocity of the current frame's image data is obtained based on the target trajectory, including:

[0045] Based on the target trajectory of each water surface tracer in the current frame, obtain the displacement information of each water surface tracer in the current frame;

[0046] Based on displacement information and the time interval between the current frame and the next frame, the flow velocity of each water surface tracer in the current frame is obtained;

[0047] The surface flow velocity of the current frame is obtained by fusing the flow velocities of each surface tracer in the current frame.

[0048] In this embodiment of the disclosure, the flow velocity of each water surface tracer in the current frame is obtained based on the target trajectory of each water surface tracer in the current frame, and the flow velocity of each water surface tracer in the current frame is fused to obtain the water surface flow velocity of the image data of the current frame. This can reduce the error of the water surface flow velocity, ensure the reliability of the water surface flow velocity, and improve the accuracy of water surface flow velocity detection.

[0049] Secondly, this disclosure provides a water surface velocity detection device, the device comprising:

[0050] The acquisition module is used to acquire multiple frames of image data to be detected. The image data includes water surface tracers used for water surface velocity detection.

[0051] The first obtaining module is used to extract, fuse, and predict features of the image data of the current frame using an object detection network to obtain a first object detection box corresponding to the image data of the current frame, and to extract, fuse, and predict features of the image data of the next frame using the object detection network to obtain a second object detection box corresponding to the image data of the next frame. The object detection network includes a feature extraction layer, an attention mechanism, a feature fusion layer, and a prediction layer, with the attention mechanism located between the feature extraction layer and the feature fusion layer.

[0052] The second obtaining module is used to update the historical trajectories of all water surface tracers in the current frame based on the first target detection box and the second target detection box using the target tracking algorithm, so as to obtain the target trajectories of all water surface tracers in the current frame.

[0053] The third module is used to obtain the surface flow velocity of the current frame's image data based on the target trajectory.

[0054] Thirdly, this disclosure provides a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the water surface flow velocity detection method described in the first aspect or any corresponding embodiment.

[0055] Fourthly, this disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to execute the water surface velocity detection method described in the first aspect or any corresponding embodiment.

[0056] Fifthly, this disclosure provides a computer program product, including computer instructions for causing a computer to execute the water surface velocity detection method described in the first aspect or any corresponding embodiment. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0058] Figure 1 This is a schematic flowchart of a water surface velocity detection method according to an embodiment of the present disclosure;

[0059] Figure 2 This is a schematic diagram of the structure of a target detection network according to an embodiment of the present disclosure;

[0060] Figure 3 This is a schematic diagram of the attention mechanism according to an embodiment of the present disclosure;

[0061] Figure 4 This is a flowchart illustrating a target tracking algorithm according to an embodiment of the present disclosure;

[0062] Figure 5 This is a schematic flowchart of another water surface velocity detection method according to an embodiment of the present disclosure;

[0063] Figure 6 This is a schematic flowchart of another water surface velocity detection method according to an embodiment of the present disclosure;

[0064] Figure 7 This is a structural block diagram of a water surface velocity detection device according to an embodiment of the present disclosure;

[0065] Figure 8 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present disclosure. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0067] Currently, the field of water surface velocity detection mainly relies on hardware devices such as current meters, which determine the water surface velocity by comparing the frequency difference between transmitted and reflected signals. This method of velocity measurement requires significant manpower and maintenance costs, and its measurement accuracy is greatly affected by environmental factors and equipment parameters.

[0068] Current techniques typically involve deploying surface tracer particles into the water flow and tracking their positions at different times, calculating the surface velocity based on their displacement and time variations. However, because single-type surface tracer particles are susceptible to errors caused by factors such as changes in illumination, water surface fluctuations, and background noise, it is necessary to use multiple types of surface tracer particles simultaneously for surface velocity calculation.

[0069] In recent years, with the rapid development of deep learning, image recognition methods based on convolutional neural networks (CNNs) have been widely applied and have shown significant advantages in various recognition fields such as object detection and semantic segmentation. However, when using CNNs to detect water surface tracer particles, the accuracy of the model often fluctuates in complex scenes due to environmental factors such as lighting, shadows, and complex backgrounds. Furthermore, although CNNs can detect the category and location of water surface tracer particles, they struggle to distinguish between particles of the same category, resulting in low detection accuracy.

[0070] To address the aforementioned problems, an embodiment of a water surface velocity detection method is provided according to the present disclosure. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0071] This embodiment provides a method for detecting water surface velocity, such as Figure 1 As shown, Figure 1 This is a schematic flowchart of a water surface velocity detection method according to an embodiment of the present disclosure. The process can be applied to a server and includes the following steps:

[0072] Step S101: Acquire multiple frames of image data to be detected. The image data includes water surface tracers used for water surface velocity detection.

[0073] Optionally, in this embodiment of the disclosure, the server can acquire the video data to be detected through methods such as web crawling or on-site filming. After acquiring the video data, the server performs frame-by-frame processing on the video data to obtain multiple frames of image data to be detected.

[0074] Specifically, surface tracers are objects used to mark and track water flow during surface velocity detection. Because surface tracers have good buoyancy and visibility, and can move naturally with the water flow, the server can intuitively understand the water flow trajectory by recording and analyzing the surface tracers at different times and locations. Surface tracers can be artificially made tracers such as plastic balls, foam products, and fluorescent markers, or tracers made of natural materials such as leaves and wood chips.

[0075] In addition, the server can preprocess the image data. For example, the server can first scale the image data to the same size, then normalize the image data, calculate the mean and standard deviation of the pixel values ​​of each channel of the image data, and then subtract the mean and divide by the standard deviation from the normalized image data to obtain the preprocessed image data.

[0076] Step S102: Use the object detection network to extract features, fuse features, and predict the image data of the current frame to obtain the first object detection box corresponding to the image data of the current frame. Use the object detection network to extract features, fuse features, and predict the image data of the next frame to obtain the second object detection box corresponding to the image data of the next frame. The object detection network includes a feature extraction layer, an attention mechanism, a feature fusion layer, and a prediction layer. The attention mechanism is located between the feature extraction layer and the feature fusion layer.

[0077] Optionally, in embodiments of this disclosure, such as Figure 2 As shown, the object detection network includes a feature extraction layer, an attention mechanism, a feature fusion layer, and a prediction layer. The attention mechanism is located between the feature extraction layer and the feature fusion layer. The first object detection box refers to the object detection box corresponding to the image data of the current frame, and the second object detection box refers to the object detection box corresponding to the image data of the next frame.

[0078] Specifically, the server first inputs the image data of the current frame and the image data of the next frame into the target detection network, and then uses the convolution kernel of the feature extraction layer to slide on the image data of the current frame and perform convolution calculation with the pixels of the image data of the current frame. The convolution kernel of the feature extraction layer is then used to slide on the image data of the next frame and perform convolution calculation with the pixels of the image data of the next frame.

[0079] Then, as Figure 3 As shown, the server uses an attention mechanism to weight the features of different channels in the current frame image data output by the feature extraction layer, and to weight the features of different channels in the next frame image data output by the feature extraction layer.

[0080] Next, the server uses a feature fusion layer to fuse multiple features output during feature extraction of the current frame's image data with the weighted features, and then fuses multiple features output during feature extraction of the next frame's image data with the weighted features.

[0081] Then, the server uses the prediction layer to predict the fusion features corresponding to the image data of the current frame to obtain the first target detection box, and uses the prediction layer to predict the fusion features corresponding to the image data of the next frame to obtain the second target detection box.

[0082] Step S103: Using a target tracking algorithm, based on the first target detection box and the second target detection box, update the historical trajectories of all water surface tracers in the current frame to obtain the target trajectories of all water surface tracers in the current frame.

[0083] Optionally, in this embodiment of the disclosure, the historical trajectory refers to the set of all trajectories of each surface tracer before the current frame image data, and the target trajectory refers to the set of all trajectories of each surface tracer before the current frame image data.

[0084] Specifically, such as Figure 4 As shown, the server first uses a Kalman filter to predict the state of the next frame based on the state information of the first target detection box, thus obtaining the tracking box for the next frame. Then, the server uses the Hungarian algorithm to match the tracking box of the next frame with the second target detection box, obtaining the matching result. Afterward, the server uses a Kalman filter to update the historical trajectories of all water surface tracers in the current frame based on the matching result, thus obtaining the target trajectories of all water surface tracers in the current frame.

[0085] Step S104: Obtain the water surface flow velocity of the current frame image data based on the target trajectory.

[0086] Optionally, in this embodiment of the present disclosure, the server first calculates the displacement of each water surface tracer from the current frame to the next frame based on the coordinate information of the first target detection box and the coordinate information of the second target detection box in the target trajectory of each water surface tracer in the current frame. Then, it calculates the time elapsed from the current frame to the next frame based on the frame rate of the video data to be detected. Next, it calculates the flow velocity of each water surface tracer based on the displacement and time elapsed from the current frame to the next frame. Finally, it performs a fusion calculation on the flow velocity of each water surface tracer (such as average calculation, median calculation, mode calculation, weighted calculation, etc.) to obtain the water surface flow velocity of the image data of the current frame.

[0087] In this embodiment, multiple frames of image data to be detected are acquired. These image data include surface tracers used for water surface velocity detection. A target detection network is used to extract, fuse, and predict features from the current frame's image data to obtain a first target detection box. The same network is then used to extract, fuse, and predict features from the next frame's image data to obtain a second target detection box. The target detection network includes a feature extraction layer, an attention mechanism, a feature fusion layer, and a prediction layer, with the attention mechanism located between the feature extraction and feature fusion layers. A target tracking algorithm is used to update the historical trajectories of all surface tracers in the current frame based on the first and second target detection boxes, obtaining the target trajectories of all surface tracers in the current frame. The water surface velocity of the current frame's image data is then obtained based on the target trajectories. Because this embodiment improves water surface velocity detection accuracy by fusing an attention mechanism in the target detection network and using a target tracking algorithm to track surface tracers, this method achieves higher accuracy.

[0088] In some optional implementations, this embodiment provides a method for detecting water surface velocity, such as... Figure 5 As shown, Figure 5 This is a schematic flowchart of another water surface velocity detection method according to an embodiment of the present disclosure. This process can be applied to a server and includes the following steps:

[0089] Step S501: Acquire multiple frames of image data to be detected. The image data includes water surface tracers used for water surface velocity detection. See details below. Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0090] Step S502: Use the object detection network to extract features, fuse features, and predict the image data of the current frame to obtain the first object detection box corresponding to the image data of the current frame. Use the object detection network to extract features, fuse features, and predict the image data of the next frame to obtain the second object detection box corresponding to the image data of the next frame. The object detection network includes a feature extraction layer, an attention mechanism, a feature fusion layer, and a prediction layer. The attention mechanism is located between the feature extraction layer and the feature fusion layer.

[0091] Specifically, step S502 includes:

[0092] Step S5021: Based on the feature extraction layer, perform feature extraction on the image data of the current frame to obtain the first feature, and perform feature extraction on the image data of the next frame to obtain the second feature.

[0093] Optionally, in this embodiment of the disclosure, the first feature refers to the feature obtained by processing the image data of the current frame through the feature extraction layer, and the second feature refers to the feature obtained by processing the image data of the next frame through the feature extraction layer. For example... Figure 2 As shown, the feature extraction layer includes a convolutional module, a residual module, and a pooling module. The convolutional module includes convolutional layers, normalization layers, and activation function layers. The residual module includes convolutional layers, a residual network, and a channel concatenation operator. The pooling module includes convolutional layers and pooling layers.

[0094] Specifically, the server first inputs the image data of the current frame and the image data of the next frame into the convolution module. Using convolution kernels of different sizes and numbers in the convolution layer, convolution operations are performed on the image data of the current frame and the image data of the next frame, respectively, to extract local features of the image data of the current frame and the image data of the next frame. Then, a normalization layer is used to normalize the local features of the image data of the current frame and the image data of the next frame output by the convolution layer in the channel dimension, so that the feature mean of each channel is 0 and the variance is 1. Finally, an activation function layer is used to perform nonlinear transformation on the normalized features of the image data of the current frame and the image data of the next frame, respectively, setting the feature values ​​less than 0 to 0 and keeping the feature values ​​greater than 0 unchanged.

[0095] Then, the server inputs the features of the current frame image data output by the convolutional module and the features of the next frame image data into the residual module. The convolutional layer then performs convolution operations on the features of the current frame image data and the features of the next frame image data to extract deeper features. At the same time, the residual network performs feature extraction and residual connection on the features of the current frame image data output by the convolutional layer and the image data of the next frame image data. Finally, the channel concatenation operator is used to concatenate the features of the current frame image data output by the convolutional layer and the features of the current frame image data output by the residual network, and concatenate the features of the next frame image data output by the convolutional layer and the features of the next frame image data output by the residual network.

[0096] Then, the server inputs the features of the current frame image data output by the residual module and the features of the next frame image data into the pooling module. The convolutional layer further performs convolution operations on the features of the current frame image data and the features of the next frame image data to extract deeper features. Then, the pooling layer downsamples the features of the current frame image data output by the convolutional layer and the features of the next frame image data to obtain the first feature corresponding to the current frame image data and the second feature corresponding to the next frame image data.

[0097] It should be noted that, as Figure 2 As shown, the server uses multiple convolutional modules, multiple residual modules, and pooling modules in the feature extraction layer to extract features from the input image data, resulting in multi-scale feature maps (such as feature maps downsampled by 2x, 4x, and 8x).

[0098] Step S5022: Perform convolution operation on the image data of the current frame based on the convolutional layer to obtain the third feature, and select the first network feature from the preset layer from the third feature. Perform convolution operation on the image data of the next frame to obtain the fourth feature, and select the second network feature from the preset layer from the fourth feature.

[0099] Optionally, in this embodiment of the disclosure, the third feature refers to the feature output during feature extraction of the current frame's image data in the feature extraction layer, the fourth feature refers to the feature output during feature extraction of the next frame's image data in the feature extraction layer, the first network feature refers to the feature from the preset layer in the third feature, and the second network feature refers to the feature from the preset layer in the fourth feature. The preset layer is included in the feature extraction layer, such as... Figure 2 As shown, the preset layer refers to Figure 2 Modules 4 and 6 in the document.

[0100] Specifically, the server uses the convolutional layer in the feature extraction layer to perform a convolution operation on the image data of the current frame to obtain a third feature, and selects from the third feature the features from the preset layer ( Figure 2 The first network features of modules 4 and 6 in the feature extraction layer are used to perform convolution operations on the image data of the next frame to obtain the fourth feature, and the fourth feature is selected from the preset layer (modules 4 and 6 in the feature extraction layer). Figure 2 The second network features of modules 4 and 6 in the network.

[0101] Step S5023: Weight the first feature based on the attention mechanism to obtain the weighted first feature, and weight the second feature to obtain the weighted second feature.

[0102] Optionally, in embodiments of this disclosure, such as Figure 3 As shown, the server uses an attention mechanism to weight the first feature and the second feature respectively, and adjusts the channel attention of the first feature and the channel attention of the second feature to obtain the weighted first feature and the weighted second feature.

[0103] In some optional implementations, step S5023 above includes:

[0104] Step a1: Perform pooling operation on the first feature based on the average pooling layer to obtain the first tensor, and perform pooling operation on the second feature to obtain the second tensor.

[0105] Step a2: Perform a one-dimensional convolution on the first tensor based on the one-dimensional convolutional layer to obtain the third tensor, and perform a one-dimensional convolution on the second tensor to obtain the fourth tensor.

[0106] Step a3: Use the activation function to transform the third tensor to the preset interval to obtain the first weight, and transform the fourth tensor to the preset interval to obtain the second weight.

[0107] Step a4: Fuse the first feature and the first weight to obtain the weighted first feature, and fuse the second feature and the second weight to obtain the weighted second feature.

[0108] Optionally, in this embodiment of the disclosure, the first tensor refers to the tensor after pooling the first feature, the second tensor refers to the tensor after pooling the second feature, the third tensor refers to the tensor after one-dimensional convolution of the first tensor, the fourth tensor refers to the tensor after one-dimensional convolution of the second tensor, the first weight refers to the weight of the first feature, and the second weight refers to the weight of the second feature. The preset interval refers to the value range of the first weight and the second weight, such as the interval (0, 1).

[0109] Specifically, the server first performs a pooling operation on the first feature using an average pooling layer, adjusting the size of the first feature from W×H×C to 1×1×C to obtain the first tensor. Then, it performs a pooling operation on the second feature, adjusting its size from W×H×C to 1×1×C to obtain the second tensor. Here, W represents the width of the feature map, H represents the height of the feature map, and C represents the number of channels in the feature map.

[0110] Then, the server uses a one-dimensional convolutional layer to perform a one-dimensional convolution on the first tensor to obtain the third tensor, and performs a one-dimensional convolution on the second tensor to obtain the fourth tensor.

[0111] Next, the server uses an activation function (such as the sigmoid activation function) to transform the third tensor to the (0, 1) interval to obtain the first weight, and transforms the fourth tensor to the (0, 1) interval to obtain the second weight.

[0112] Finally, the server multiplies the first feature and the first weight to obtain the weighted first feature, and multiplies the second feature and the second weight to obtain the weighted second feature.

[0113] In the above embodiments, by weighting the first feature and the second feature based on the attention mechanism, the channel information of the first feature and the second feature can be fully utilized to enhance the fitting ability of the target detection network and improve the accuracy of water surface velocity detection.

[0114] Step S5024: Based on the feature fusion layer, feature fusion is performed on the first network feature and the weighted first feature to obtain the first fused feature; feature fusion is performed on the second network feature and the weighted second feature to obtain the second fused feature.

[0115] Optionally, in this embodiment of the disclosure, the first fusion feature refers to the fusion feature corresponding to the image data of the current frame, and the second fusion feature refers to the fusion feature corresponding to the image data of the next frame. For example... Figure 2 As shown, the feature fusion layer includes a feature pyramid network (i.e., Figure 2 Modules 11-17 in the middle) and path aggregation network (i.e. Figure 2(Modules 18-24 in the network). The feature pyramid network includes a convolutional module, an upsampling module, a stitching module, and a residual module, while the path aggregation network includes a residual module, a convolutional module, and a stitching module.

[0116] Specifically, the server utilizes the feature pyramid network in the feature fusion layer to process the features of the first network (i.e., Figure 2 The features output by modules 4 and 6 in the model) and the weighted first feature (i.e. Figure 2 The features output by module 10 are subjected to convolution, upsampling, residual connections, and concatenation operations. Then, the path aggregation network in the feature fusion layer is used to further process the features. Figure 2 The features output from modules 11, 14, and 17 are subjected to residual connections, convolution operations, and concatenation operations to obtain the first fused feature (i.e., Figure 2 (Features output by modules 18, 21, and 24 in the module).

[0117] Simultaneously, the server utilizes the feature pyramid network in the feature fusion layer to process the features of the second network (i.e. Figure 2 The features output by modules 4 and 6 in the model) and the weighted second feature (i.e. Figure 2 The features output by module 10 are subjected to convolution, upsampling, residual connections, and concatenation operations. Then, the path aggregation network in the feature fusion layer is used to further process the features. Figure 2 The features output from modules 11, 14, and 17 are subjected to residual connections, convolution operations, and concatenation operations to obtain the second fused feature (i.e., Figure 2 (Features output by modules 18, 21, and 24 in the module).

[0118] Step S5025: Detect the first fusion feature of the input based on the prediction layer, output the first target detection box corresponding to the image data of the current frame, detect the second fusion feature of the input, and output the second target detection box corresponding to the image data of the next frame.

[0119] Optionally, in this embodiment of the present disclosure, the server first uses the prediction layer to perform classification calculation on the first fused feature, maps the first fused feature to the category space, outputs the score of the corresponding water surface tracer belonging to each water surface tracer category through a fully connected layer or a convolutional layer, then uses the Softmax function to normalize the score, converts the score into a probability distribution, so that the sum of the probabilities of all categories is 1, obtains the probability of the corresponding water surface tracer belonging to each category, and determines the category with the highest probability as the category of the corresponding water surface tracer.

[0120] Then, the server uses the prediction layer to perform regression calculation on the first fusion feature, calculates the position and size of the corresponding water surface tracer based on the first fusion feature, and obtains information on multiple first candidate detection boxes corresponding to the image data of the current frame, including the confidence of the candidate box (i.e. the probability that the candidate box contains the water surface tracer), the center position coordinates of the candidate box, the length and width of the candidate box, etc.

[0121] Then, the server can use non-maximum suppression to determine the first target detection box from multiple first candidate detection boxes. First, all first candidate detection boxes are sorted from high to low confidence. The first candidate detection box with the highest confidence is selected as the retained box, that is, the box most likely to contain the water surface tracer. Then, the intersection-union ratio (IUR) between the retained box and the other first candidate detection boxes is calculated to determine the degree of overlap between these boxes. First candidate detection boxes with an IUR greater than a preset threshold (such as 0.5) are discarded. The above steps of selecting retained boxes, calculating the overlap, and removing overlapping boxes are repeated to obtain the final remaining first candidate detection boxes, that is, the first target detection box.

[0122] Similarly, the server uses the prediction layer to classify the second fusion feature to obtain the category information of the water surface tracer in the image data of the next frame. It uses the prediction layer to perform regression calculation on the second fusion feature to obtain the information of multiple second candidate detection boxes corresponding to the image data of the next frame. Then, it uses the non-maximum suppression method to filter from the multiple second candidate detection boxes to obtain the second target detection box.

[0123] Step S503: Using a target tracking algorithm based on the first and second target detection boxes, update the historical trajectories of all water surface tracers in the current frame to obtain the target trajectories of all water surface tracers in the current frame. For details, please refer to [link to relevant documentation]. Figure 1 Step S103 of the illustrated embodiment will not be described again here.

[0124] Step S504: Obtain the water surface flow velocity from the image data of the current frame based on the target trajectory. For details, please refer to [link to relevant documentation]. Figure 1 Step S104 of the illustrated embodiment will not be described again here.

[0125] In this embodiment of the disclosure, by utilizing the feature extraction layer, attention mechanism, feature fusion layer and prediction layer in the target detection network, feature extraction, feature fusion and prediction are performed on the image data of the current frame and the image data of the next frame. This enables the accurate capture of the features of water surface tracers in the image data at different scales, obtaining accurate target detection boxes and improving the accuracy of water surface velocity detection.

[0126] In some optional implementations, this embodiment provides a method for detecting water surface velocity, such as... Figure 6 As shown, Figure 6This is a flowchart illustrating another water surface velocity detection method according to an embodiment of the present disclosure. This process can be applied to a server and includes the following steps:

[0127] Step S601: Acquire multiple frames of image data to be detected. The image data includes water surface tracers used for water surface velocity detection. See details below. Figure 5 Step S501 of the illustrated embodiment will not be described again here.

[0128] Step S602: The object detection network is used to extract, fuse, and predict features from the image data of the current frame to obtain a first object detection box corresponding to the image data of the current frame. The same network is then used to extract, fuse, and predict features from the image data of the next frame to obtain a second object detection box corresponding to the image data of the next frame. The object detection network includes a feature extraction layer, an attention mechanism, a feature fusion layer, and a prediction layer. The attention mechanism is located between the feature extraction layer and the feature fusion layer. For details, please refer to [link to details]. Figure 5 Step S502 of the illustrated embodiment will not be described again here.

[0129] Step S603: Using a target tracking algorithm based on the first target detection box and the second target detection box, update the historical trajectories of all water surface tracers in the current frame to obtain the target trajectories of all water surface tracers in the current frame.

[0130] Specifically, step S603 includes:

[0131] Step S6031: Obtain the status information of the first target detection box.

[0132] Optionally, in this embodiment of the disclosure, the state information of the first target detection box is an 8-dimensional vector, including the center position coordinates (u, v), aspect ratio y, height h, center position horizontal coordinate velocity (i.e., the horizontal movement speed of the corresponding water surface tracer), center position vertical coordinate velocity (i.e., the vertical movement speed of the corresponding water surface tracer), aspect ratio change rate, and height change rate.

[0133] Step S6032: Based on the state information, predict the trajectory of all water surface tracers in the current frame for the next frame to obtain the tracking box of all water surface tracers in the current frame for the next frame.

[0134] Optionally, in this embodiment of the present disclosure, the server uses a Kalman filter to first construct a state transition matrix and an observation matrix based on the state information of the first target detection box, and then predicts the trajectory of all surface tracers in the current frame for the next frame based on the state information of the first target detection box and the state transition matrix, thereby obtaining the tracking boxes of all surface tracers in the current frame for the next frame.

[0135] Step S6033: Match the second target detection box with the tracking box of the next frame to obtain the matching result.

[0136] Optionally, in embodiments of this disclosure, such as Figure 2 As shown, the server uses the Hungarian algorithm to match the second target detection box with the tracking box in the next frame to obtain the matching result.

[0137] In some optional implementations, step S6033 includes:

[0138] Step b1: Based on the number of historical unrelated frames of all water surface tracers in the current frame, determine the state of the tracking box in the next frame, and obtain the first tracking box with a determined state and the second tracking box with an undetermined state.

[0139] Step b2: Perform a first match between the first tracking box and the second target detection box to obtain a first matching result. The first matching result includes a third tracking box and a fourth tracking box. The third tracking box is the tracking box that does not match the second target detection box, and the fourth tracking box is the tracking box that does not match the second target detection box when a new water surface tracer is detected.

[0140] Step b3: Perform a second matching on the second tracking box, the third tracking box, and the fourth tracking box to obtain the second matching result.

[0141] Optionally, in this embodiment of the disclosure, the number of historically unassociated frames refers to the number of frames since each surface tracer was last successfully associated. The tracking frame of the next frame includes a first tracking frame in a determined state and a second tracking frame in an undetermined state. The first matching result includes a third tracking frame that does not match the second target detection frame, and a fourth tracking frame that does not match the second target detection frame if a new surface tracer is detected.

[0142] Specifically, when the server uses the Kalman filter to track all surface tracers, for each surface tracer, it calculates the number of frames since the last successful association to obtain the number of historically unassociated frames. If the current frame is successfully associated with the next frame (i.e., the second target detection box is successfully matched with the tracking box of the next frame), the number of historically unassociated frames is reset to 0; if the current frame is not successfully associated with the next frame (i.e., the second target detection box is not matched with the tracking box of the next frame), the number of historically unassociated frames is incremented by 1.

[0143] First, the server determines the state of the tracking frame for the next frame based on the number of historically unassociated frames for all surface tracers in the current frame. The tracking frame states for the next frame include a definite state, an indefinite state, and a deleted state. For each surface tracer, if its historical unassociated frame count exceeds a preset number, the tracking frame state for the next frame is deleted; if it has been successfully associated in every frame before the current frame, the tracking frame state for the next frame is definite (the first tracking frame); if its historical unassociated frame count does not exceed the preset number, and there are frames before the current frame where association was unsuccessfully completed, the tracking frame state for the next frame is indefinite (the second tracking frame).

[0144] Then, the server performs a first match (i.e., cascaded match) between the first tracking box with the second target detection box, resulting in a first match result (i.e., cascaded match result). More specifically, the server classifies the first tracking box according to the occlusion time it has experienced, and determines the matching order according to the principle that the shorter the occlusion time, the higher the matching level (for example, starting the matching from the first tracking box with the shortest occlusion time). Then, the server performs a first match between the first tracking box and the second target detection box according to the matching order, resulting in a third tracking box that does not match the second target detection box, and a fourth tracking box that does not match the second target detection box if a new water surface tracer is detected.

[0145] Next, the server performs a second matching (i.e., intersection-union matching) on ​​the second, third, and fourth tracking boxes to obtain the second matching result (i.e., the intersection-union matching result). More specifically, the server calculates the intersection-union ratio between the second tracking box and the third and fourth tracking boxes respectively to obtain the second matching result.

[0146] In the above embodiments, by performing a first match and a second match between the second target detection box and the tracking box of the next frame, the accuracy and stability of tracking water surface tracers can be improved, and the accuracy of water surface velocity detection can be improved.

[0147] Step S6034: Update the historical trajectories of all water surface tracers in the current frame based on the matching results to obtain the target trajectory.

[0148] Optionally, in this embodiment of the disclosure, the matching result includes a first matching result (i.e., a cascaded matching result) and a second matching result (i.e., an intersection-union matching result). The server uses a Kalman filter to update the historical trajectories of all surface tracers in the current frame based on the matching results to obtain the target trajectory.

[0149] In some optional implementations, step S6034 above includes:

[0150] Step c1: Update the historical trajectories of all water surface tracers in the current frame using the fifth tracking frame to obtain the trajectory of the first target.

[0151] Step c2: Based on the number of historical unrelated frames of all surface tracers in the current frame corresponding to the third tracking frame, determine the state of the third tracking frame and obtain the first and second tracking frames.

[0152] Step c3: Compare the number of historical unrelated frames of the first tracking frame with the preset number of frames to obtain the sixth and seventh tracking frames. The sixth tracking frame is the tracking frame with a number of historical unrelated frames greater than the preset number of frames, and the seventh tracking frame is the tracking frame with a number of historical unrelated frames less than the preset number of frames.

[0153] Step c4: Using the fourth and seventh tracking boxes in the second matching result, create new trajectories for all water surface tracers in the current frame to obtain the second target trajectory.

[0154] Step c5: Delete the historical trajectories of all water surface tracers in the current frame corresponding to the second and sixth tracking frames.

[0155] Step c6: Based on the first target trajectory and the second target trajectory, obtain the target trajectory.

[0156] Optionally, in this embodiment of the disclosure, the first matching result includes, in addition to the third and fourth tracking boxes mentioned in the above embodiments, a fifth tracking box that successfully matches the second target detection box. The second matching result includes the third, fourth, and fifth tracking boxes.

[0157] The third tracking box in the second matching result includes the first tracking box and the second tracking box. The first tracking box includes the sixth tracking box with a historical unrelated frame count greater than the preset frame count, and the seventh tracking box with a historical unrelated frame count less than the preset frame count.

[0158] Specifically, the server uses the fifth tracking frame to update the historical trajectories of all surface tracers in the current frame to obtain the first target trajectory, and uses the fourth and seventh tracking frames in the second matching result to create new trajectories of all surface tracers in the current frame to obtain the second target trajectory. Finally, the server combines the first target trajectory and the second target trajectory to obtain the target trajectory.

[0159] In addition, the server also needs to delete the historical trajectories of all water surface tracers in the current frame corresponding to the second and sixth tracking frames.

[0160] In the above embodiments, by updating the historical trajectories of all surface tracers in the current frame based on the first matching result and the second matching result, it is possible to continuously and stably track all surface tracers, obtain accurate target trajectories, and improve the accuracy of surface velocity detection.

[0161] Step S604: Obtain the water surface flow velocity of the current frame image data based on the target trajectory.

[0162] Specifically, step S604 includes:

[0163] Step S6041: Based on the target trajectory of each water surface tracer in the current frame, obtain the displacement information of each water surface tracer in the current frame.

[0164] Optionally, in this embodiment of the disclosure, the server calculates the displacement information of each water surface tracer in the current frame based on the coordinate information of the first target detection box and the coordinate information of the second target detection box in the target trajectory of each water surface tracer in the current frame.

[0165] Step S6042: Based on the displacement information and the time interval between the current frame and the next frame, obtain the flow velocity of each water surface tracer in the current frame.

[0166] Optionally, in this embodiment of the disclosure, the server calculates the time interval between the current frame and the next frame based on the frame rate of the video data to be detected, and calculates the flow velocity of each water surface tracer in the current frame based on the displacement information of each water surface tracer in the current frame and the time interval between the current frame and the next frame.

[0167] Step S6043: Fuse the flow velocity of each water surface tracer in the current frame to obtain the water surface flow velocity of the image data of the current frame.

[0168] Optionally, in this embodiment of the disclosure, the server performs a fusion calculation (such as average calculation, median calculation, mode calculation, weighted calculation, etc.) on the flow velocity of each water surface tracer in the current frame to obtain the water surface flow velocity of the image data of the current frame.

[0169] In this embodiment, by using a target tracking algorithm to update the historical trajectories of all surface tracers in the current frame, all surface tracers can be continuously and stably tracked, accurate target trajectories can be obtained, and the accuracy of water surface velocity detection can be improved. By obtaining the flow velocity of each surface tracer in the current frame based on its target trajectory, and fusing the flow velocities of each surface tracer in the current frame to obtain the water surface flow velocity of the image data for the current frame, errors in water surface flow velocity can be reduced, ensuring the reliability of water surface flow velocity detection and improving its accuracy.

[0170] In one alternative implementation, the server can train the initial detection network to obtain the target detection network before using the target detection network to extract features, fuse features, and predict the image data of the current frame.

[0171] Specifically, the server acquires water surface tracer image data through web crawling or on-site photography, and further expands the dataset using data augmentation techniques such as random rotation and random cropping. After standardizing the image size, the categories and bounding boxes of the water surface tracers in the images are manually labeled using annotation tools to obtain the water surface tracer dataset. The dataset is then divided into training, validation, and test sets in a certain ratio (e.g., 8:1:1). The initial detection network is trained using the training set, the hyperparameters of the initial detection network are adjusted using the validation set, and finally, the model performance of the initial detection network is tested using the test set.

[0172] It should be noted that when training the initial detection network, the server uses the full intersection-over-union (CIU) loss function and the stochastic gradient descent algorithm for optimization, with a momentum of 0.9, a weight decay of 0.0001, and an initial learning rate of 0.0001. During the warm-up phase of training (i.e., the first 10 iterations), the server linearly increases the learning rate to 0.001, and then decays it exponentially by 0.95 for 100 iterations. Training of the initial detection network stops when the loss on the validation set no longer decreases (i.e., the detection boxes and labeled bounding boxes essentially overlap), resulting in the object detection network.

[0173] This embodiment provides a water surface flow velocity detection device, such as Figure 7 Shown, including:

[0174] The acquisition module 701 is used to acquire multiple frames of image data to be detected, and the image data includes water surface tracers for water surface velocity detection.

[0175] The first obtaining module 702 is used to extract features, fuse features, and predict the image data of the current frame using an object detection network to obtain a first object detection box corresponding to the image data of the current frame, and to extract features, fuse features, and predict the image data of the next frame using the object detection network to obtain a second object detection box corresponding to the image data of the next frame. The object detection network includes a feature extraction layer, an attention mechanism, a feature fusion layer, and a prediction layer, and the attention mechanism is located between the feature extraction layer and the feature fusion layer.

[0176] The second obtaining module 703 is used to update the historical trajectories of all water surface tracers in the current frame based on the first target detection box and the second target detection box using a target tracking algorithm, so as to obtain the target trajectories of all water surface tracers in the current frame.

[0177] The third module 704 is used to obtain the surface flow velocity of the current frame image data based on the target trajectory.

[0178] In this embodiment, multiple frames of image data to be detected are acquired. These image data include surface tracers used for water surface velocity detection. A target detection network is used to extract, fuse, and predict features from the current frame's image data to obtain a first target detection box. The same network is then used to extract, fuse, and predict features from the next frame's image data to obtain a second target detection box. The target detection network includes a feature extraction layer, an attention mechanism, a feature fusion layer, and a prediction layer, with the attention mechanism located between the feature extraction and feature fusion layers. A target tracking algorithm is used to update the historical trajectories of all surface tracers in the current frame based on the first and second target detection boxes, obtaining the target trajectories of all surface tracers in the current frame. The water surface velocity of the current frame's image data is then obtained based on the target trajectories. Because this embodiment improves water surface velocity detection accuracy by fusing an attention mechanism in the target detection network and using a target tracking algorithm to track surface tracers, this method achieves higher accuracy.

[0179] In some alternative implementations, the first obtaining module 702 includes:

[0180] The first submodule is used to extract features from the image data of the current frame based on the feature extraction layer to obtain the first feature, and to extract features from the image data of the next frame to obtain the second feature;

[0181] The second submodule is used to perform convolution operation on the image data of the current frame based on the convolution layer to obtain the third feature, and select the first network feature from the preset layer from the third feature, perform convolution operation on the image data of the next frame to obtain the fourth feature, and select the second network feature from the preset layer from the fourth feature.

[0182] The third submodule is used to weight the first feature based on the attention mechanism to obtain the weighted first feature, and to weight the second feature to obtain the weighted second feature.

[0183] The fourth submodule is used to perform feature fusion on the first network features and the weighted first features based on the feature fusion layer to obtain the first fused features, and to perform feature fusion on the second network features and the weighted second features to obtain the second fused features.

[0184] The output submodule is used to detect the first fusion feature of the input based on the prediction layer, output the first target detection box corresponding to the image data of the current frame, detect the second fusion feature of the input, and output the second target detection box corresponding to the image data of the next frame.

[0185] In some alternative implementations, the third submodule includes:

[0186] The first obtaining unit is used to perform a pooling operation on the first feature based on the average pooling layer to obtain the first tensor, and to perform a pooling operation on the second feature to obtain the second tensor.

[0187] The second obtaining unit is used to perform one-dimensional convolution on the first tensor based on the one-dimensional convolution layer to obtain the third tensor, and to perform one-dimensional convolution on the second tensor to obtain the fourth tensor.

[0188] The third unit is used to transform the third tensor to a preset interval using an activation function to obtain the first weight, and to transform the fourth tensor to a preset interval to obtain the second weight.

[0189] The fourth unit is used to fuse the first feature and the first weight to obtain the weighted first feature, and to fuse the second feature and the second weight to obtain the weighted second feature.

[0190] In some alternative implementations, the second obtaining module 703 includes:

[0191] The first acquisition submodule is used to acquire the status information of the first target detection box;

[0192] The fifth submodule is used to predict the trajectory of all water surface tracers in the current frame for the next frame based on the state information, and to obtain the tracking box of all water surface tracers in the current frame for the next frame.

[0193] The sixth submodule is used to match the second target detection box with the tracking box of the next frame to obtain the matching result;

[0194] The seventh submodule is used to update the historical trajectories of all water surface tracers in the current frame based on the matching results, and obtain the target trajectory.

[0195] In some alternative implementations, the sixth submodule includes:

[0196] The fifth unit is used to determine the state of the tracking box in the next frame based on the number of historical unrelated frames of all surface tracers in the current frame, and to obtain the first tracking box with a determined state and the second tracking box with an undetermined state.

[0197] The sixth obtaining unit is used to perform a first match between the first tracking box and the second target detection box to obtain a first matching result. The first matching result includes a third tracking box and a fourth tracking box. The third tracking box is a tracking box that does not match the second target detection box, and the fourth tracking box is a tracking box that does not match the second target detection box when a new water surface tracer is detected.

[0198] The seventh unit is used to perform a second matching of the second tracking box, the third tracking box, and the fourth tracking box to obtain the second matching result.

[0199] In some alternative implementations, the seventh submodule includes:

[0200] The eighth unit is used to update the historical trajectories of all water surface tracers in the current frame using the fifth tracking frame to obtain the first target trajectory;

[0201] The ninth unit is used to determine the state of the third tracking frame based on the number of historical unrelated frames of all surface tracers in the current frame corresponding to the third tracking frame, and to obtain the first tracking frame and the second tracking frame.

[0202] The tenth obtaining unit is used to compare the number of historical unrelated frames of the first tracking frame with a preset number of frames to obtain the sixth tracking frame and the seventh tracking frame. The sixth tracking frame is the tracking frame with a number of historical unrelated frames greater than the preset number of frames, and the seventh tracking frame is the tracking frame with a number of historical unrelated frames less than the preset number of frames.

[0203] The eleventh unit is used to create new trajectories for all water surface tracers in the current frame using the fourth and seventh tracking boxes in the second matching result, and to obtain the second target trajectory.

[0204] The deletion unit is used to delete the historical trajectories of all water surface tracers in the current frame corresponding to the second and sixth tracking frames;

[0205] The twelfth obtaining unit is used to obtain the target trajectory based on the first target trajectory and the second target trajectory.

[0206] In some alternative implementations, the third obtaining module 704 includes:

[0207] The second acquisition submodule is used to acquire the displacement information of each water surface tracer in the current frame based on the target trajectory of each water surface tracer in the current frame;

[0208] The eighth submodule is used to obtain the flow velocity of each water surface tracer in the current frame based on displacement information and the time interval between the current frame and the next frame.

[0209] The ninth submodule is used to fuse the flow velocity of each water surface tracer in the current frame to obtain the water surface flow velocity of the image data of the current frame.

[0210] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0211] In this embodiment, the water surface velocity detection device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0212] This disclosure also provides a computer device having the above-described features. Figure 7 The water surface velocity detection device shown.

[0213] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of this disclosure, such as... Figure 8 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 Take a processor 10 as an example.

[0214] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0215] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0216] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0217] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0218] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0219] This disclosure also provides a computer-readable storage medium in which the methods described in this disclosure can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium after being downloaded over a network. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium may be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0220] A portion of this disclosure can be applied to computer program products, such as computer program instructions, which, when executed by a computer, can invoke or provide methods and / or technical solutions according to this disclosure through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, and installation package files. Accordingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions; the computer compiling the instructions and then executing the corresponding compiled program; the computer reading and executing the instructions; or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0221] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for detecting water surface velocity, characterized in that, The method includes: Acquire multiple frames of image data to be detected, the image data including water surface tracers for water surface velocity detection; The object detection network is used to extract features, fuse features, and predict the image data of the current frame to obtain a first object detection box corresponding to the image data of the current frame. The object detection network is then used to extract features, fuse features, and predict the image data of the next frame to obtain a second object detection box corresponding to the image data of the next frame. The object detection network includes a feature extraction layer, an attention mechanism, a feature fusion layer, and a prediction layer. The attention mechanism is located between the feature extraction layer and the feature fusion layer. Using a target tracking algorithm based on the first target detection box and the second target detection box, the historical trajectories of all water surface tracers in the current frame are updated to obtain the target trajectories of all water surface tracers in the current frame; The water surface flow velocity of the current frame's image data is obtained based on the target trajectory; The step of using a target tracking algorithm to update the historical trajectories of all water surface tracers in the current frame based on the first target detection box and the second target detection box to obtain the target trajectories of all water surface tracers in the current frame includes: Obtain the status information of the first target detection box; Based on the state information, the trajectories of all surface tracers in the current frame are predicted for the next frame, and the tracking boxes of all surface tracers in the current frame for the next frame are obtained. The second target detection box is matched with the tracking box in the next frame to obtain the matching result; Based on the matching results, the historical trajectories of all water surface tracers in the current frame are updated to obtain the target trajectory; The step of matching the second target detection box with the tracking box of the next frame to obtain a matching result includes: Based on the number of historical unrelated frames of all surface tracers in the current frame, the state of the tracking frame in the next frame is determined, resulting in a first tracking frame with a determined state and a second tracking frame with an undetermined state. The first tracking box is matched with the second target detection box to obtain a first matching result. The first matching result includes a third tracking box, a fourth tracking box, and a fifth tracking box. The third tracking box is a tracking box that does not match the second target detection box. The fourth tracking box is a tracking box that does not match the second target detection box when a new water surface tracer is detected. The fifth tracking box is a tracking box that successfully matches the second target detection box. The second tracking box, the third tracking box, and the fourth tracking box are matched in a second way to obtain a second matching result, which includes the third tracking box, the fourth tracking box, and the fifth tracking box. The step of updating the historical trajectories of all surface tracers in the current frame based on the matching results to obtain the target trajectory includes: The historical trajectories of all water surface tracers in the current frame are updated using the fifth tracking frame to obtain the first target trajectory; Based on the number of historical unrelated frames of all surface tracers in the current frame corresponding to the third tracking frame, the state of the third tracking frame is determined, and the first tracking frame and the second tracking frame are obtained. The number of historical unrelated frames of the first tracking frame is compared with a preset number of frames to obtain a sixth tracking frame and a seventh tracking frame. The sixth tracking frame is the tracking frame whose number of historical unrelated frames is greater than the preset number of frames, and the seventh tracking frame is the tracking frame whose number of historical unrelated frames is less than the preset number of frames. Using the fourth and seventh tracking boxes in the second matching result, new trajectories of all water surface tracers in the current frame are created to obtain the second target trajectory. Delete the historical trajectories of all water surface tracers in the current frame corresponding to the second and sixth tracking frames; The target trajectory is obtained based on the first target trajectory and the second target trajectory.

2. The method according to claim 1, characterized in that, The feature extraction layer comprises multiple convolutional layers. The step of using the object detection network to extract, fuse, and predict features from the image data of the current frame to obtain a first object detection box corresponding to the image data of the current frame, and then using the object detection network to extract, fuse, and predict features from the image data of the next frame to obtain a second object detection box corresponding to the image data of the next frame, includes: Based on the feature extraction layer, feature extraction is performed on the image data of the current frame to obtain a first feature, and feature extraction is performed on the image data of the next frame to obtain a second feature; Based on the convolutional layer, the image data of the current frame is convolved to obtain a third feature, and a first network feature from a preset layer is selected from the third feature. The image data of the next frame is then convolved to obtain a fourth feature, and a second network feature from the preset layer is selected from the fourth feature. The first feature is weighted based on the attention mechanism to obtain a weighted first feature, and the second feature is weighted to obtain a weighted second feature. Based on the feature fusion layer, the first network feature and the weighted first feature are fused to obtain a first fused feature, and the second network feature and the weighted second feature are fused to obtain a second fused feature; Based on the prediction layer, the first fusion feature of the input is detected, and the first target detection box corresponding to the image data of the current frame is output. The second fusion feature of the input is detected, and the second target detection box corresponding to the image data of the next frame is output.

3. The method according to claim 2, characterized in that, The step of weighting the first feature based on the attention mechanism to obtain a weighted first feature, and weighting the second feature to obtain a weighted second feature, includes: The first feature is pooled using an average pooling layer to obtain a first tensor, and the second feature is pooled using an average pooling layer to obtain a second tensor. The first tensor is convolved in one dimension using a one-dimensional convolutional layer to obtain the third tensor, and the second tensor is convolved in one dimension to obtain the fourth tensor. The third tensor is transformed to a preset interval using an activation function to obtain the first weight, and the fourth tensor is transformed to the preset interval to obtain the second weight. The first feature and the first weight are fused to obtain the weighted first feature, and the second feature and the second weight are fused to obtain the weighted second feature.

4. The method according to claim 1, characterized in that, The water surface flow velocity obtained from the image data of the current frame based on the target trajectory includes: Based on the target trajectory of each water surface tracer in the current frame, obtain the displacement information of each water surface tracer in the current frame; Based on the displacement information and the time interval between the current frame and the next frame, the flow velocity of each water surface tracer in the current frame is obtained; The surface flow velocity of the image data of the current frame is obtained by fusing the flow velocities of each surface tracer in the current frame.

5. A water surface velocity detection device, characterized in that, The device includes: The acquisition module is used to acquire multiple frames of image data to be detected, the image data including water surface tracers for water surface velocity detection; The first obtaining module is used to extract features, fuse features, and predict the image data of the current frame using an object detection network to obtain a first object detection box corresponding to the image data of the current frame, and to extract features, fuse features, and predict the image data of the next frame using the object detection network to obtain a second object detection box corresponding to the image data of the next frame. The object detection network includes a feature extraction layer, an attention mechanism, a feature fusion layer, and a prediction layer, wherein the attention mechanism is located between the feature extraction layer and the feature fusion layer. The second obtaining module is used to update the historical trajectories of all water surface tracers in the current frame based on the first target detection box and the second target detection box using a target tracking algorithm, so as to obtain the target trajectories of all water surface tracers in the current frame. The third module is used to obtain the surface flow velocity of the image data of the current frame based on the target trajectory; Specifically, the second obtaining module is used for: Obtain the status information of the first target detection box; Based on the state information, the trajectories of all surface tracers in the current frame are predicted for the next frame, and the tracking boxes of all surface tracers in the current frame for the next frame are obtained. The second target detection box is matched with the tracking box in the next frame to obtain the matching result; Based on the matching results, the historical trajectories of all water surface tracers in the current frame are updated to obtain the target trajectory; The step of matching the second target detection box with the tracking box of the next frame to obtain a matching result includes: Based on the number of historical unrelated frames of all surface tracers in the current frame, the state of the tracking frame in the next frame is determined, resulting in a first tracking frame with a determined state and a second tracking frame with an undetermined state. The first tracking box is matched with the second target detection box to obtain a first matching result. The first matching result includes a third tracking box, a fourth tracking box, and a fifth tracking box. The third tracking box is a tracking box that does not match the second target detection box. The fourth tracking box is a tracking box that does not match the second target detection box when a new water surface tracer is detected. The fifth tracking box is a tracking box that successfully matches the second target detection box. The second tracking box, the third tracking box, and the fourth tracking box are matched in a second way to obtain a second matching result, which includes the third tracking box, the fourth tracking box, and the fifth tracking box. The step of updating the historical trajectories of all surface tracers in the current frame based on the matching results to obtain the target trajectory includes: The historical trajectories of all water surface tracers in the current frame are updated using the fifth tracking frame to obtain the first target trajectory; Based on the number of historical unrelated frames of all surface tracers in the current frame corresponding to the third tracking frame, the state of the third tracking frame is determined, and the first tracking frame and the second tracking frame are obtained. The number of historical unrelated frames of the first tracking frame is compared with a preset number of frames to obtain a sixth tracking frame and a seventh tracking frame. The sixth tracking frame is the tracking frame whose number of historical unrelated frames is greater than the preset number of frames, and the seventh tracking frame is the tracking frame whose number of historical unrelated frames is less than the preset number of frames. Using the fourth and seventh tracking boxes in the second matching result, new trajectories of all water surface tracers in the current frame are created to obtain the second target trajectory. Delete the historical trajectories of all water surface tracers in the current frame corresponding to the second and sixth tracking frames; The target trajectory is obtained based on the first target trajectory and the second target trajectory.

6. A computer device, characterized in that, include: The system includes a memory and a processor, which are interconnected and the memory stores computer instructions. The processor executes the computer instructions to perform the water surface velocity detection method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the water surface velocity detection method according to any one of claims 1 to 4.

Images