Vision-based underwater moving target speed measurement method and system

Abstract

The invention discloses a vision-based underwater moving target speed measurement method and system. The attention feature extraction module is used for building a lightweight image feature extraction network, the parameter quantity and the calculation quantity can be reduced, weights are distributed for feature maps of different channels, the weight of the channel where important features are located is improved, and the detection precision is improved; the improved SIoU is used as a regression loss function, so that the model training speed can be increased, later-stage oscillation of model training is reduced, and the model detection precision is further improved; in order to improve the speed measurement precision, a target position extraction module is adopted to process a target area extracted by a target detection module, and the center-of-mass coordinates of the target area are obtained; in order to reduce the speed measurement error, a speed measurement module is adopted to calculate the multi-area point displacement distance, abnormal values are eliminated, and the target movement speed is reliably calculated.

Description

Technical Field

[0001] The invention belongs to the field of speed measurement and relates to a method for measuring the speed of a moving target, and in particular to a method and system for measuring the speed of an underwater moving target based on vision. Background Art

[0002] The speed information of underwater moving objects plays an important role in the navigation, positioning and control of underwater equipment. At present, large underwater moving equipment mostly uses the Doppler effect to measure the speed of objects. This method has high measurement accuracy, but there are problems such as large equipment size and high cost. In order to get rid of the above equipment constraints and provide a variety of underwater target speed measurement methods, it is necessary to use visual detection to measure the displacement speed of underwater moving equipment.

[0003] In recent years, underwater optical technology has been widely used in underwater biological detection, underwater robot research and other research fields due to its own characteristics. The current research on underwater visual measurement methods focuses on detecting underwater targets, and there is little research on the speed measurement of underwater targets.

[0004] The visual speed measurement method for moving targets in ground environments performs poorly in complex natural water bodies with a large number of aquatic organisms and water impurities, and lacks targeted adaptation for underwater environmental targets.

[0005] In summary, in view of the complex underwater water environment, it is necessary to propose a speed measurement method for underwater moving targets based on vision. Summary of the invention

[0006] In order to solve the problems brought about by the existing underwater speed measurement equipment, the present invention provides a vision-based underwater moving target speed measurement method and system, which can alleviate the insufficient performance of ground-based visual speed measurement methods in underwater environments, provide multi-angle speed measurement accuracy support for underwater speed measurement, and provide a variety of underwater target speed measurement methods.

[0007] The technical solution of the present invention is: 1. A method for measuring the speed of underwater moving targets based on vision S1: Acquire underwater target motion image sequence; S2: using the target detection module to perform target detection on each image frame in the underwater target motion image sequence to obtain the target area corresponding to each image frame; S3: according to the target area of ​​each image frame, using the target position extraction module to extract the characteristics and position of the target in each image frame; S4: According to the characteristics and position of the target in each image frame, the speed of the target is detected by using the speed measurement module, thereby completing the speed measurement of the underwater moving target.

[0008] In S2, the target detection module includes an image feature extraction network, a feature fusion network and a prediction network connected in sequence, wherein the image feature extraction network includes a first convolution module and multiple groups of feature extraction modules, the first convolution module is connected to the first group of feature extraction modules, the input of the image feature extraction network is used as the input of the first convolution module, and the multiple groups of feature extraction modules are connected in sequence; except for the first group of feature extraction modules, other groups of feature extraction modules are connected to the prediction network through a feature fusion network.

[0009] Each group of feature extraction modules includes a connected second convolution module and an attention feature extraction module, the input of each group of feature extraction modules is used as the input of the second convolution module, and the output of the attention feature extraction module is used as the output of each group of feature extraction modules; the attention feature extraction module includes a third convolution module, a bottleneck structure module, and a channel feature fusion module based on the ECA attention mechanism; the input of the attention feature extraction module is used as the input of the third convolution module, and the output of the third convolution module is split in the channel dimension to obtain the features after the first convolution and the features after the second convolution, and the features after the second convolution are connected to the second bottleneck structure module after passing through the first bottleneck structure module, and the input of the third convolution module, the features after the first convolution, the output of the first bottleneck structure module, and the output of the second bottleneck structure module are spliced ​​in the channel dimension and then input into the channel feature fusion module based on the ECA attention mechanism, and the channel feature fusion module based on the ECA attention mechanism is used to redistribute the feature map weights of the channel dimension.

[0010] The channel feature fusion module based on the ECA attention mechanism includes a fourth convolution module and an ECA attention mechanism module connected in sequence.

[0011] When the image feature extraction network performs position regression, the anchor frame regression loss adopts the improved SIoU loss function, so that the anchor frame regresses along the diagonal direction. The formula is as follows: L SIoU-改 =1-IoU+(△+Ω) / 2 △=2-e^((Λ-2)×(c w / C w ) 2 ) -e^((Λ-2)×(c h / C h ) 2 ) Ω=(1- e^(-(|ww gt |) / max(w,w gt )) θ + e^(-(|hh gt |) / max(h,h gt )) θ Λ=cos(2×arcsin(x+π / 4)) Among them, L SIoU-改 is the anchor box regression loss, IoU is the intersection over union ratio of the predicted box and the actual box, θ is the degree of attention paid by the anchor box regression loss to the shape loss, △ is the distance loss of the center point, Ω is the anchor box shape loss, Λ is the regression angle loss, w,h,w gt ,h gt are the width and height of the predicted box and the width and height of the actual box, respectively, c w ,c h is the distance between the center of the predicted box and the actual box in the X-axis and Y-axis directions, C w , C h is the width and height of the minimum circumscribed rectangle of the predicted box and the actual box, x is the angle between the center points of the two boxes, | | is the absolute value, and ^ represents the power.

[0012] The target position extraction module includes an edge contour extraction module, an abnormal value elimination module, and a centroid coordinate calculation module which are connected in sequence.

[0013] The edge contour extraction module comprises a grayscale image conversion module, an adaptive binarization module and a stripe edge extraction module based on the Suzuki-Abe method which are connected in sequence.

[0014] In the outlier removal module, firstly, outlier detection is performed on the initial edge contour output by the edge contour extraction module to obtain abnormal edge points; then, the proportion of outliers in each target area is calculated respectively; for target areas where the proportion of outliers is less than a preset threshold, the abnormal edge points are removed and then the target edge is completed using linear interpolation; if the proportion of outliers in the target area is greater than the preset threshold, the entire target edge in the target area is removed; finally, the final target edge contour is sent to the centroid coordinate calculation module.

[0015] The S4 is specifically: S41: sorting the feature position of each target in each image frame to obtain a feature position sequence corresponding to each target; S42: After matching the features of each target, obtain a feature matching result of each target; S43: using a monocular camera calibration method to sequentially convert each position in the feature position sequence of each target from a pixel coordinate system to a world coordinate system, thereby obtaining an actual distance sequence of each target; S44: calculating the displacement between two adjacent positions in the actual distance sequence of each target, and eliminating abnormal actual distances in the actual distance sequence of each target according to the displacement, so as to obtain a processed actual distance sequence; S45: Based on the feature matching result of each target, if the processed actual distance sequence consists of the actual distances of N consecutive frames of the same feature, the displacements of multiple groups of K frames are calculated and the average is taken as the movement speed of the current target; if the processed actual distance sequence contains different features and the frame number interval of the same feature is L frames, the actual distance of the feature beyond the camera area is calculated according to the feature interval of each target, and then the movement speed of the current target is calculated in combination with the multiple actual distances of the feature.

[0016] 2. A vision-based underwater moving target speed measurement system An image acquisition module, used to acquire a sequence of underwater target motion images; The target detection module is used to perform target detection on each image frame in the underwater target motion image sequence to obtain the target area corresponding to each image frame; A target position extraction module is used to extract the characteristics and position of the target in each image frame according to the target area of ​​each image frame; The speed measurement module is used to detect the speed of the target according to the characteristics and position of the target in each image frame, thereby completing the speed measurement of the underwater moving target.

[0017] 3. A computer device The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the vision-based underwater moving target speed measurement method when executing the computer program.

[0018] 4. A computer-readable storage medium The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the vision-based underwater moving target speed measurement method are implemented.

[0019] 5. A computer program product The computer program product includes a computer program / instruction, which implements the steps of the vision-based underwater moving target speed measurement method when executed by a processor.

[0020] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention introduces an improved C2f module based on the attention mechanism and an improved SIoU regression loss function to construct a deep learning underwater detection module, thereby measuring the speed of the actual underwater target moving image data obtained, thereby improving the detection accuracy of underwater targets and the training speed of the model.

[0021] 2. The present invention utilizes image processing methods such as adaptive threshold segmentation to obtain effective feature points in the target area and reduce the interference of abnormal data.

[0022] 3. The present invention constructs a target motion speed measurement module based on actual conditions and sets different speed measurement calculation methods for different measurement scenarios.

[0023] 4. The vision-based underwater moving target speed measurement method proposed in the present invention can effectively obtain the moving speed of a specific underwater moving target and meet the measurement error of 0.5m / s; and it introduces a variety of underwater speed measurement methods, which effectively supplements the shortcomings of underwater speed measurement methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The present invention is a flow chart of the method.

[0025] Figure 2 Schematic diagram of the target detection module.

[0026] Figure 3 Schematic diagram of the structure of the attention feature extraction module.

[0027] Figure 4 Schematic diagram of forcing the anchor box to regress along the diagonal direction to improve the SIoU loss function.

[0028] Figure 5 Schematic diagram of the image results in different processing steps.

[0029] Figure 6 Schematic diagram of the network structure of the feature fusion network.

[0030] Figure 7 Schematic diagram of the continuous calculation mode.

[0031] Figure 8 A schematic diagram of the fuzzy computing model. DETAILED DESCRIPTION

[0032] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0033] The present invention proposes a method for measuring the speed of underwater moving targets based on vision. Figure 1 As shown, the measurement method includes the following steps: S1: Use an underwater camera to collect and obtain underwater target motion image sequences; S2: using the target detection module to perform target detection on each image frame in the underwater target motion image sequence to obtain the target area corresponding to each image frame; like Figure 2As shown, the target detection module includes an image feature extraction network, a feature fusion network and a prediction network connected in sequence, wherein the image feature extraction network includes a first convolution module with a step size of 2 and a convolution kernel size of 3*3 and multiple groups of feature extraction modules, the first convolution module is connected to the first group of feature extraction modules, the input of the image feature extraction network is used as the input of the first convolution module, and the multiple groups of feature extraction modules are connected in sequence; except for the first group of feature extraction modules, the other groups of feature extraction modules are connected to the prediction network through the feature fusion network, that is, the output of the first group of feature extraction networks does not participate in the feature fusion process, and the feature fusion network fuses the outputs of the other groups of feature extraction modules. In this embodiment, four groups of feature extraction modules are set.

[0034] Each set of feature extraction modules includes a second convolution module with a step size of 2 and a convolution kernel size of 3*3 and an attention feature extraction module. The input of each set of feature extraction modules is used as the input of the second convolution module, and the output of the attention feature extraction module is used as the output of each set of feature extraction modules. The attention feature extraction module is constructed based on the C2f module, such as Figure 3 As shown, the attention feature extraction module structurally includes a third convolution module (standard convolution module) with a step size of 1, a bottleneck structure module, and a channel feature fusion module based on the ECA attention mechanism; the input of the attention feature extraction module is used as the input of the third convolution module, and the output of the third convolution module is split in the channel dimension to obtain the features after the first convolution and the features after the second convolution, the feature area after the first convolution is the feature corresponding to the first half channel dimension of the output of the third convolution module, the feature after the second convolution is the feature corresponding to the second half channel dimension of the output of the third convolution module, the feature after the second convolution is connected to the second bottleneck structure module after the first bottleneck structure module, the input of the third convolution module, the features after the first convolution, the output of the first bottleneck structure module and the output of the second bottleneck structure module are spliced ​​in the channel dimension and then input into the channel feature fusion module based on the ECA attention mechanism, and the channel feature fusion module based on the ECA attention mechanism is used to redistribute the weights of the feature maps in the channel dimension.

[0035] The standard convolution module with a stride of 1 is composed of a 2D convolution with a convolution kernel size of 3, a batch normalization module, and an activation function (SiLU) connected in sequence. It is used to integrate the feature maps of the input channels and perform channel dimension reduction to reduce the computational complexity of image feature extraction. Among them, the stride of the 2D convolution determines the stride of the standard convolution module. Several standard convolution modules are used in each layer of the image feature extraction network. Each standard convolution module performs convolution calculations on all input feature maps. That is, for a certain layer of the image feature extraction network, if the layer has n standard convolution modules, its input is m feature maps. Each standard convolution module needs to perform convolution operations on the m input feature maps respectively, and obtain a feature map after adding the convolution operation results in the same position. Since there are n standard convolution modules in this layer of the network, the output of this layer is n feature maps. The standard convolution module can extract the spatial features and channel features of the input image, but the deeper the number of convolution layers, the more likely it is that shallow information will be lost.

[0036] The bottleneck structure module consists of two sequentially connected convolution modules with a stride of 1 and a convolution kernel size of 3*3, which is used to extract the spatial features of the input feature map and keep the number of channels of the output feature map consistent with that of the input feature map.

[0037] The channel feature fusion module based on the ECA attention mechanism includes a fourth convolution module (standard convolution module) with a step size of 1 and a convolution kernel size of 1*1 and an ECA attention mechanism module, which are connected in sequence. It is used to splice and integrate the calculation results of the aforementioned modules in the channel dimension to achieve local cross-channel interaction, extract inter-channel dependencies, and keep the number of channels of the output feature map consistent with the number of channels of the input feature map of the attention feature extraction module.

[0038] In this embodiment, the training data set used by the target detection module is derived from the actual shooting of the underwater camera, with a total of 1,000 underwater pictures. In order to be used for model training, the present invention uses LabelImg to annotate these pictures in YOLO format. In order to enrich the data set, the present invention uses data enhancement methods such as Mosaic to expand the training data set to improve the robustness of the trained model. The expanded data set contains a total of 1,800 pictures.

[0039] When the image feature extraction network performs position regression, the anchor box regression loss uses the improved SIoU loss function, so that the anchor box regresses along the diagonal direction, such as Figure 4 As shown, the specific formula is as follows: L SIoU-改 =1-IoU+(△+Ω) / 2 △=2-e^((Λ-2)×(c w / C w ) 2 ) -e^((Λ-2)×(c h / Ch ) 2 ) Ω=(1- e^(-(|ww gt |) / max(w,w gt )) θ + e^(-(|hh gt |) / max(h,h gt )) θ Λ=cos(2×arcsin(x+π / 4)) Among them, L SIoU-改 is the anchor box regression loss, IoU is the intersection over union ratio of the predicted box and the actual box, θ is the degree of attention paid by the anchor box regression loss to the shape loss, which is generally set to 2. △ is the distance loss of the center point, Ω is the anchor box shape loss, Λ is the regression angle loss, w,h,w gt ,h gt are the width and height of the predicted box and the width and height of the actual box, respectively, c w ,c h is the distance between the center of the predicted box and the actual box in the X-axis and Y-axis directions, C w , C h is the width and height of the minimum circumscribed rectangle of the predicted box and the actual box, x is the angle between the center points of the two boxes, || is the absolute value, and ^ represents the power. By adopting the improved SIoU regression loss function, the model training speed is accelerated, the oscillation in the late stage of model training is reduced, and the model detection accuracy is further improved.

[0040] The input feature map scales of the feature fusion network are 20, 40, and 80, i.e., feature maps at three scales, and the output feature map scale of the feature fusion network is consistent with the input feature map; Figure 6 As shown in Figure 1, the feature fusion network adopts the idea of ​​PAN structure, and its feature fusion process includes two processes: upward fusion and downward fusion. Among them, the fusion of each two adjacent scale feature maps is integrated through a layer of attention feature extraction module.

[0041] In the top-down fusion process, the input is the feature maps of different scales output by the bottom-up fusion process, which are the outputs of the second, third, and fourth groups of feature extraction modules in the image feature extraction network, that is, feature maps with scales of 80, 40, and 20, respectively. Feature maps of different scales are upsampled only once before fusion, and the calculation process is as follows: the feature map with a scale of 80 is downsampled and fused with the feature map with a scale of 40, and the fusion result is input into the above-mentioned attention feature extraction module for channel-dimensional feature integration. Finally, the output of this module is the feature maps with scales of 80, 40, and 20 after feature integration.

[0042] The feature fusion network fuses feature maps at different scales, improves the deep semantic information of shallow feature maps, and improves the shape contour information of deep feature maps, so that feature maps at each scale have both shallow contour features and deep semantic information. The feature maps at different scales after feature fusion are input into the corresponding prediction network to obtain the coordinates of the target area. Then, the coordinates of all target areas in the image frame obtained by each prediction network and the category information corresponding to each target area are input into the target position extraction module. Among them, the prediction network adopts the idea of ​​divide and conquer of target classification and anchor frame regression, decouples classification and regression tasks, and obtains the position and category of the target in the image respectively.

[0043] The configuration for building the experimental platform is: Ubuntu20.04 operating system, CUDA10.1, python3.8, and graphics card is NVIDIA Geforce RTX 3060.

[0044] As shown in Table 1, the target detection module proposed in the present invention has a certain speed in terms of accuracy compared with target detection modules such as SSD and Faster RCNN, with an accuracy of up to 93.54% and a detection speed of 74, which makes the target detection method suitable for underwater environments with limited resources and can monitor the camera screen in real time.

[0045] Table 1 is a comparison table of the target detection module proposed in this invention and the target detection modules such as SSD.

[0046] S3: according to the target area of ​​each image frame, using the target position extraction module to extract the characteristics and position of the target in each image frame; Although the target detection module can reduce the interference of redundant factors such as background, the target area itself still has noise points caused by light reflected from water impurities, such as Figure 5 As shown, the target position extraction module can effectively reduce the interference caused by noise points. The target position extraction module includes an edge contour extraction module, an outlier removal module, and a centroid coordinate calculation module which are connected in sequence.

[0047] The edge contour extraction module is structurally composed of a grayscale image conversion module, an adaptive binarization module, and a stripe edge extraction module based on the Suzuki-Abe method. First, the target area image is converted into a grayscale image, and the color information is converted into grayscale information to reduce the information density. Then, the target area is divided into two parts by adaptive threshold segmentation based on the characteristics of the target area. There are many features of underwater targets. For different features, feature edge extraction and centroid calculation are used to obtain the location information of the feature edge. The following example takes a target with multiple stripe edges on the surface as an example. Figure 5As shown. Since the target area is composed of two stripes of different colors, after it is converted into a grayscale image, the grayscale difference is large, and the grayscale difference within the stripe is small. Therefore, there is almost no noise inside the stripe area of ​​the converted binary image. Then, the Suzuki-Abe method is used to extract the outer boundary contour of the white foreground part of the binary image. This method starts from a certain pixel point in the image, and can move along the boundary direction of the foreground and record the boundary pixels until it returns to the starting point to form a complete boundary contour. Since the boundaries in the other three directions are actually the boundaries of the target area, the remaining boundaries after elimination are the edge contours that may contain outliers, which are recorded as the initial edge contours.

[0048] The outlier rejection module is used to monitor the extracted initial edge contours and reduce the impact of abnormal edge contours on speed measurement accuracy. Based on the target area state statistics, outlier judgment rules are set, and two types of treatments are taken for outliers according to the set thresholds.

[0049] In the outlier removal module, firstly, outlier detection is performed on the initial edge contour output by the edge contour extraction module to obtain abnormal edge points; then, the outlier ratio of each target area is calculated respectively. For target areas where the outlier ratio is less than the preset threshold, the abnormal edge points are removed and the target edge is completed by linear interpolation. If the outlier ratio of the target area is greater than the preset threshold, the entire target edge in the target area is removed; finally, the final target edge contour is sent to the centroid coordinate calculation module. Taking the target with multiple stripe edges on the surface as an example, the normal stripe edge is relatively smooth. If the method model detects a sudden change edge, it can be considered that the target feature is abnormal. For target features with abnormalities, the outlier processing logic is selected according to the comparison between the ratio of the outlier area in the entire feature area and the preset threshold. The outlier processing logic is divided into: outlier interpolation cleaning and feature area removal.

[0050] The calculation method of the outlier ratio is as follows: ratio err =err tol / edge tol Among them, ratio err is the proportion of outliers, err tol is the number of abnormal edge pixels, edge tol is the number of pixels of the extracted stripe edge The centroid coordinate calculation module calculates the centroid coordinates of each target area and uses it as the characteristic position of the target. The characteristic position of each target is input into the speed measurement module to measure the target movement speed and output the final target movement speed measurement result.

[0051] If the proportion of outliers is less than the set threshold, the outliers are removed and the obscured stripe edges are complemented by linear interpolation. If the proportion of outliers is greater than the set threshold, the stripe edges are removed.

[0052] The edge feature, as a curve, cannot intuitively represent the current position of the object. Therefore, the present invention introduces centroid coordinates to represent the current position of the moving object.

[0053] The formula for calculating the centroid coordinates is: X=1 / n×∑ n i=1 x i Y=1 / n×∑ n i=1 y i Among them, X is the X-axis coordinate of the centroid of the stripe edge, Y is the Y-axis coordinate of the centroid of the stripe edge, n is the total number of pixels on the stripe edge, and x i is the X-axis coordinate of the i-th pixel point extracted on the edge of the stripe, y i is the Y-axis coordinate of the i-th pixel point extracted on the edge of the stripe.

[0054] S4: According to the characteristics and position of the target in each image frame, the speed of the target is detected by using the speed measurement module, thereby completing the speed measurement of the underwater moving target.

[0055] S4 is specifically: S41: sorting the feature position of each target in each image frame to obtain a feature position sequence corresponding to each target; S42: After matching the features of each target, obtain a feature matching result of each target; S43: Using a monocular camera calibration method, each position in the feature position sequence of each target is converted from the pixel coordinate system to the world coordinate system in sequence, so as to obtain the actual distance sequence of each target; the monocular camera calibration method is to obtain the actual distance of the target at a fixed distance corresponding to each pixel block of the image screen, and then convert all the centroid coordinate distance changes of the target area into the actual movement distance of the target from the bottom of the camera screen.

[0056] S44: calculating the displacement between two adjacent positions in the actual distance sequence of each target, and eliminating abnormal actual distances in the actual distance sequence of each target according to the displacement, so as to obtain a processed actual distance sequence; S45: Based on the feature matching result of each target, if the processed actual distance sequence consists of the actual distances of N consecutive frames of the same feature, the displacements of multiple groups of K frames are calculated and the average is taken as the movement speed of the current target; if the processed actual distance sequence contains different features and the frame number interval of the same feature is L frames, the actual distance of the feature beyond the camera area is calculated by fuzzy calculation according to the prior feature interval of each target, and then the movement speed of the current target is calculated by combining the multiple actual distances of the feature.

[0057] Since the target can enter the image from multiple directions of the camera, for the convenience of explanation, the following description takes the case where the target only enters from the bottom of the image as an example. In order to improve the speed monitoring accuracy, the present invention calculates the target moving speed by increasing the speed measurement interval. The speed measurement interval is set based on the maximum speed of the speed measurement target and the camera frame rate. Therefore, there are two different calculation modes in the speed measurement analysis module.

[0058] Figure 7 This is a schematic diagram of the operation of the continuous calculation mode. When the centroid coordinates of the same target feature are obtained for a certain number of consecutive frames, the continuous calculation mode is entered. Based on the speed measurement interval frame, the moving distance of the feature target within the time period is obtained, and after eliminating outliers, the average of the moving distances of multiple feature points is taken as the moving distance of the target.

[0059] Figure 8 The diagram is a schematic diagram of the operation of the fuzzy calculation mode. When the same feature point of a moving target is calculated in a preset interval frame, it may exceed the camera area. The fuzzy calculation method is used to obtain the current position of the feature that exceeds the camera area based on the prior target feature interval X, and calculate the moving distance of the target.

[0060] The present invention provides a speed measurement method for underwater moving targets based on vision. The method uses an attention feature extraction module to build a lightweight image feature extraction network, which can reduce the amount of parameters and calculations, and assign weights to feature maps of different channels, increase the weights of channels where important features are located, and improve detection accuracy; using improved SIoU as a regression loss function can speed up model training, reduce oscillation in the late stage of model training, and further improve model detection accuracy; in order to improve the speed measurement accuracy, a target position extraction module is used to process the target area extracted by a target detection module to obtain the centroid coordinates of the target area; in order to reduce the speed measurement error, a speed measurement module is used to calculate the displacement distance of multi-region points, eliminate outliers, and reliably calculate the target movement speed.

[0061] The present invention also proposes a vision-based underwater moving target speed measurement system, the measurement system comprising: An image acquisition module, used to acquire a sequence of underwater target motion images; The target detection module is used to perform target detection on each image frame in the underwater target motion image sequence to obtain the target area corresponding to each image frame; A target position extraction module is used to extract the characteristics and position of the target in each image frame according to the target area of ​​each image frame; The speed measurement module is used to detect the speed of the target according to the characteristics and position of the target in each image frame, thereby completing the speed measurement of the underwater moving target.

[0062] Finally, it should be noted that the above embodiments and explanations are only used to illustrate the technical solution of the present invention rather than to limit it. Those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope disclosed in the technical solution of the present invention, which should be included in the scope of protection of the claims of the present invention.

Claims

1. A method for measuring the speed of an underwater moving target based on vision, characterized in that: The following steps are involved: S1: Acquire underwater target motion image sequence; S2: using the target detection module to perform target detection on each image frame in the underwater target motion image sequence to obtain the target area corresponding to each image frame; S3: according to the target area of ​​each image frame, using the target position extraction module to extract the characteristics and position of the target in each image frame; S4: According to the characteristics and position of the target in each image frame, the speed of the target is detected by using the speed measurement module, thereby completing the speed measurement of the underwater moving target.

2. The method for measuring the speed of an underwater moving target based on vision according to claim 1, characterized in that: In S2, the target detection module includes an image feature extraction network, a feature fusion network and a prediction network connected in sequence, wherein the image feature extraction network includes a first convolution module and multiple groups of feature extraction modules, the first convolution module is connected to the first group of feature extraction modules, the input of the image feature extraction network is used as the input of the first convolution module, and the multiple groups of feature extraction modules are connected in sequence; except for the first group of feature extraction modules, other groups of feature extraction modules are connected to the prediction network through a feature fusion network.

3. The method for measuring the speed of an underwater moving target based on vision according to claim 2, characterized in that: Each group of the feature extraction modules includes a connected second convolution module and an attention feature extraction module, the input of each group of the feature extraction modules is used as the input of the second convolution module, and the output of the attention feature extraction module is used as the output of each group of the feature extraction modules; the attention feature extraction module includes a third convolution module, a bottleneck structure module, and a channel feature fusion module based on the ECA attention mechanism; The input of the attention feature extraction module is used as the input of the third convolution module. The output of the third convolution module is split in the channel dimension to obtain the features after the first convolution and the features after the second convolution. The features after the second convolution are connected to the second bottleneck structure module after passing through the first bottleneck structure module. The input of the third convolution module, the features after the first convolution, the output of the first bottleneck structure module and the output of the second bottleneck structure module are spliced ​​in the channel dimension and then input into the channel feature fusion module based on the ECA attention mechanism. The channel feature fusion module based on the ECA attention mechanism is used to redistribute the feature map weights of the channel dimension.

4. The method for measuring the speed of an underwater moving target based on vision according to claim 3, characterized in that: The channel feature fusion module based on the ECA attention mechanism includes a fourth convolution module and an ECA attention mechanism module connected in sequence.

5. The method for measuring the speed of an underwater moving target based on vision according to claim 2, characterized in that: When the image feature extraction network performs position regression, the anchor frame regression loss adopts the improved SIoU loss function, so that the anchor frame regresses along the diagonal direction. The formula is as follows: L SIoU-改 =1-IoU+(△+Ω) / 2 △=2−e^((Λ−2)×(c w / C w ) 2 ) −e^((Λ−2)×(c h / C h ) 2 ) Ω=(1- e^(-(|w-w gt |) / max(w,w gt )) θ + e^(-(|h-h gt |) / max(h,h gt )) θ Λ=cos(2×arcsin(x+π / 4)) Among them, L SIoU-改 is the anchor box regression loss, IoU is the intersection over union ratio of the predicted box and the actual box, θ is the degree of attention paid by the anchor box regression loss to the shape loss, △ is the distance loss of the center point, Ω is the anchor box shape loss, Λ is the regression angle loss, w,h,w gt ,h gt are the width and height of the predicted box and the width and height of the actual box, respectively, c w ,c h is the distance between the center of the predicted box and the actual box in the X-axis and Y-axis directions, C w , C h is the width and height of the minimum circumscribed rectangle of the predicted box and the actual box, x is the angle between the center points of the two boxes, | | is the absolute value, and ^ represents the power.

6. The method for measuring the speed of an underwater moving target based on vision according to claim 1, characterized in that: The target position extraction module includes an edge contour extraction module, an abnormal value elimination module, and a centroid coordinate calculation module which are connected in sequence.

7. The method for measuring the speed of an underwater moving target based on vision according to claim 6, characterized in that: The edge contour extraction module comprises a grayscale image conversion module, an adaptive binarization module and a stripe edge extraction module based on the Suzuki-Abe method which are connected in sequence.

8. The method for measuring the speed of an underwater moving target based on vision according to claim 6, characterized in that: In the outlier removal module, firstly, outlier detection is performed on the initial edge contour output by the edge contour extraction module to obtain abnormal edge points; then, the proportion of outliers in each target area is calculated respectively; for target areas where the proportion of outliers is less than a preset threshold, the abnormal edge points are removed and then the target edge is completed using linear interpolation; if the proportion of outliers in the target area is greater than the preset threshold, the entire target edge in the target area is removed; finally, the final target edge contour is sent to the centroid coordinate calculation module.

9. The method for measuring the speed of an underwater moving target based on vision according to claim 1, characterized in that: The S4 is specifically: S41: sorting the feature position of each target in each image frame to obtain a feature position sequence corresponding to each target; S42: After matching the features of each target, obtain a feature matching result of each target; S43: using a monocular camera calibration method to sequentially convert each position in the feature position sequence of each target from a pixel coordinate system to a world coordinate system, thereby obtaining an actual distance sequence of each target; S44: calculating the displacement between two adjacent positions in the actual distance sequence of each target, and eliminating abnormal actual distances in the actual distance sequence of each target according to the displacement, so as to obtain a processed actual distance sequence; S45: Based on the feature matching result of each target, if the processed actual distance sequence consists of the actual distances of N consecutive frames of the same feature, the displacements of multiple groups of K frames are calculated and the average is taken as the movement speed of the current target; if the processed actual distance sequence contains different features and the frame number interval of the same feature is L frames, the actual distance of the feature beyond the camera area is calculated according to the feature interval of each target, and then the movement speed of the current target is calculated in combination with the multiple actual distances of the feature.

10. A vision-based underwater moving target speed measurement system, characterized in that: include: An image acquisition module, used to acquire a sequence of underwater target motion images; The target detection module is used to perform target detection on each image frame in the underwater target motion image sequence to obtain the target area corresponding to each image frame; A target position extraction module is used to extract the characteristics and position of the target in each image frame according to the target area of ​​each image frame; The speed measurement module is used to detect the speed of the target according to the characteristics and position of the target in each image frame, thereby completing the speed measurement of the underwater moving target.

Images