Underwater foresight sonar image target detection method and device

Abstract

The invention discloses an underwater forward-looking sonar image target detection method and device, and the method comprises the steps: obtaining underwater forward-looking sonar data, carrying out the target detection of a current frame of forward-looking sonar image of the underwater forward-looking sonar data, and obtaining a first target detection result; based on the current frame of forward-looking sonar image, predicting a next frame of forward-looking sonar image of the current frame of forward-looking sonar image by a Kalman filter to obtain a prediction detection result; acquiring a next frame of forward-looking sonar image of the current frame of forward-looking sonar image as the current frame of forward-looking sonar image, and performing target detection on the current frame of forward-looking sonar image to obtain a second target detection result; and matching the prediction detection result with the second target detection result, and determining the trajectory of the target according to the matching result and the first target detection result. The method can effectively identify a shielded target or distinguish target tracks with similar features.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and underwater sonar data processing, and in particular to an underwater forward-looking sonar image target detection method and device. Background Art

[0002] Underwater target detection technology has a wide range of applications and is an important technology. Accurate real-time detection of underwater targets can provide accurate information, which is conducive to decision-making and arrangements. It can also avoid underwater obstacles, ensure the safe passage of ships, and improve the safety level.

[0003] Sonar imaging is a key technology for underwater target detection. It captures targets through the return of sound waves. Since sound waves travel farther underwater than light waves, sonar images are often used for underwater target detection and positioning. Forward-looking sonar image target detection is one of the important means of underwater target detection. Compared with land and air image acquisition methods, underwater image acquisition is more difficult. The sonar data that can be used for analysis and research are often insufficient in quantity and quality, which brings many challenges to the detection and identification of underwater targets. With the continuous emergence of algorithms based on deep learning, the features extracted by convolutional neural networks have good anti-interference ability and target representation ability, which can greatly improve the performance of target detection and improve its performance in complex environments. The original YOLOv1 pioneered the single-stage detector and achieved high-speed target detection. Subsequently, YOLOv2, YOLOv3 and YOLOv4 have improved in accuracy and speed. YOLOv5 and YOLOv6 pay more attention to the lightweight and engineering implementation of the model, further improving its practicality. YOLOv7 introduced a more complex feature extraction network and multi-scale feature fusion technology, which improved accuracy while maintaining high efficiency. On this basis, YOLOv8 adopted a new architecture design and training strategy to further improve performance. Compared with its predecessor, YOLOv8 uses deep separable convolution to reduce the amount of calculation and improve computational efficiency, and performs well in the current general image detection field. However, the underwater environment is complex, the target features are not obvious, and it is easy to have problems such as missed detection and false detection. Summary of the invention

[0004] In view of this, the present invention provides an underwater forward-looking sonar image target detection method and device, which can solve technical problems such as feature degradation caused by complex underwater conditions, target blur, target occlusion, etc., and the occurrence of missed detections and false detections.

[0005] In order to solve the above technical problems, the present invention is implemented as follows.

[0006] An underwater forward-looking sonar image target detection method, comprising:

[0007] Step S1: Acquire underwater forward-looking sonar data, and use the improved YOLOv8 model to perform target detection on the current frame forward-looking sonar image of the underwater forward-looking sonar data to obtain a first target detection result; Based on the current frame forward-looking sonar image, use the Kalman filter to predict the next frame forward-looking sonar image of the current frame forward-looking sonar image to obtain a predicted detection result;

[0008] Step S2: obtaining a next frame of forward-looking sonar image of the current frame as the forward-looking sonar image of the current frame, and performing target detection on the forward-looking sonar image of the current frame by using the improved YOLOv8 model to obtain a second target detection result;

[0009] Step S3: Match the predicted detection result and the second target detection result, and determine the trajectory of the target according to the matching result and the first target detection result.

[0010] Preferably, the step S1: acquiring underwater forward-looking sonar data, performing target detection on the current frame forward-looking sonar image of the underwater forward-looking sonar video by an improved YOLOv8 model, and obtaining a first target detection result, wherein:

[0011] The improved YOLOv8 model improves the feature extraction module in the YOLOv8 model; the feature extraction module in the improved YOLOv8 model includes multiple convolution blocks, each two convolution blocks form a group, there are NUM groups of convolution groups in total, and the two convolution blocks in each group are connected in sequence; the first convolution block in the first group of convolution groups receives the input of the C2f module, the second convolution block in the first group of convolution groups takes the output of the first convolution block in the first group of convolution groups as input, and fuses the output of the first convolution block in the first group of convolution groups and the output of the second convolution block in the first group of convolution groups to obtain the output of the first group of convolution groups; the second group of convolution groups to the N-1th group of convolution groups all take the output of the previous group of convolution groups as the input of the first convolution block in the group of convolution groups, and the second convolution block in the group of convolution groups takes The output of the first convolution block in the convolution group is taken as input, and the output of the first convolution block in the convolution group, the output of the second convolution block in the convolution group, and the output of the convolution group in the previous convolution group are fused to obtain the output of the convolution group; the NUM-th convolution group takes the output of the NUM-1-th convolution group as the input of the first convolution block in the convolution group, and the second convolution block in the convolution group takes the output of the first convolution block in the convolution group as input, and the output of the first convolution block in the convolution group, the output of the second convolution block in the convolution group, and the output of the convolution group in the previous convolution group are fused as the output of the C2f module in the improved YOLOv8 model, that is, the feature vector obtained by extracting features from the forward-looking sonar image of the current frame of the underwater forward-looking sonar video.

[0012] Preferably, in step S1, based on the current frame forward-looking sonar image, a Kalman filter is used to predict the next frame forward-looking sonar image of the current frame forward-looking sonar image to obtain a predicted detection result, wherein:

[0013] The state transfer equation of the target motion used by the Kalman filter is:

[0014] {x t ,y t} = f trans ({x t-1 ,y t-1},θ)

[0015] x t =d x (x t-1 ) 2 +e x (x t-1 )+f x

[0016] y t =d x (y t-1 ) 2 +e x (y t-1 )+f y

[0017]

[0018] Among them, x t is the target horizontal coordinate at time t, y t is the target ordinate at time t, x t-1 is the target horizontal coordinate at time t-1, y t-1 is the target ordinate at time t-1, θ is the external control input, f trans The function is the state transfer equation, e x d x 、f x 、f y d y 、e y Function is the parameter of the equation to be solved.

[0019] Preferably, the step S3: matching the predicted detection result and the second target detection result includes:

[0020] Step S31: record the second target detection result as R t ={r1,r2,...,r n}, where r i =(b i ,s i ,ci ), R t represents the detection result corresponding to the tth frame, r1,r2,...,r n is the n targets included in the detection result; 1≤i≤n, b i Represents the bounding box corresponding to the i-th target (x1 i ,y1 i ,w1 i ,h1 i ) where x1 i is the horizontal coordinate of the upper left corner of the bounding box corresponding to the i-th target, y1 i is the ordinate of the upper left corner of the bounding box corresponding to the i-th target, w1 i is the width of the bounding box corresponding to the i-th target, h1 i is the height of the bounding box corresponding to the i-th target, s i is the detection score corresponding to the i-th target, c i is the target category corresponding to the i-th target;

[0021] The predicted detection result is recorded as {o1, o2, ..., o n},o m To predict the mth target in the detection result;

[0022] The target r in the second target detection result i And predict the target o in the detection result m Form a pair to be matched;

[0023] Step S32: For each to-be-matched pair, a cost function is calculated. The cost function is as follows:

[0024]

[0025] Among them, I i is the matching value of the pair to be matched, B i is the target r to be matched i The corresponding prediction box, G m is the target o to be matched m The corresponding target frame, b is the target r to be matched i The center point of the corresponding prediction box, b m is the target o to be matched m The corresponding target box center point, d is B i and G m The diagonal distance of the minimum enclosing rectangle, s i is the target r to be matched i The corresponding detection score;

[0026] If the I calculated by the to-be-matched pair iexceeds the preset threshold, then the target r in the second target detection result i And predict the target o in the detection result m Matching success;

[0027] Step S33: If there is a target that matches the second target detection result and the predicted detection result, the target is determined to be a valid target.

[0028] Preferably, in step S3, determining the trajectory of the target according to the matching result and the first target detection result comprises:

[0029] Step S31: setting a valid target set and an invalid target set, wherein the valid target set records the identifiers of each valid target, the number of successful matches of the valid target, and the frame numbers of the current frame and the previous frame corresponding to each successful match of the valid target; and the invalid target set records the identifiers of the targets that do not appear in the valid target set;

[0030] Step S32: For each valid target in the valid target set, the following operations are performed:

[0031] Obtain the maximum frame number among all frame numbers corresponding to all successful matches of the valid target;

[0032] If the difference between the frame number of the current frame and the maximum frame number is greater than or equal to 3, the valid target set clears all information of the valid target and records the cleared valid target into the invalid target set; the full information includes the identifier of the valid target, the number of successful matches of the valid target, and the frame numbers of the current frame and the previous frame corresponding to the successful matches of the valid target; if there is a formal trajectory corresponding to the valid target in the trajectory list, the formal trajectory is deleted;

[0033] If the difference between the frame number of the current frame and the maximum frame number is less than 3, when the valid target has no corresponding formal track:

[0034] When the number of successful matches of the valid target is 1, a preliminary trajectory is generated according to the trajectory of the valid target, and the parameters of the preliminary trajectory are updated;

[0035] When the number of successful matches of the valid target is 2, generating a formal trajectory based on the preliminary trajectory, assigning a trajectory number to the formal trajectory, storing the formal trajectory in a trajectory list, and outputting the formal trajectory;

[0036] If the difference between the frame number of the current frame and the maximum frame number is less than 3, when the valid target has a corresponding formal track:

[0037] Based on the official trajectory, the predicted detection result corresponding to the official trajectory, and the target in the second target detection result, the official trajectory is updated and the official trajectory is output;

[0038] Step S33: Obtain all successfully matched targets in the predicted detection result and the second target detection result, and obtain each formal track corresponding to all the successfully matched targets; for each formal track, perform the following operations:

[0039] Based on the official trajectory, the predicted detection result corresponding to the official trajectory, and the target in the second target detection result, target tracking parameters are updated, where the target tracking parameters are used to guide tracking of the target.

[0040] An underwater forward-looking sonar image target detection device, comprising:

[0041] Prediction module: configured to obtain underwater forward-looking sonar data, perform target detection on a current frame forward-looking sonar image of the underwater forward-looking sonar data by an improved YOLOv8 model, and obtain a first target detection result; based on the current frame forward-looking sonar image, perform prediction on a next frame forward-looking sonar image of the current frame forward-looking sonar image by a Kalman filter, and obtain a predicted detection result;

[0042] A detection module is configured to obtain a next frame of forward-looking sonar image of a current frame as the forward-looking sonar image of the current frame, and perform target detection on the forward-looking sonar image of the current frame by the improved YOLOv8 model to obtain a second target detection result;

[0043] Matching module: configured to match the predicted detection result and the second target detection result, and determine the trajectory of the target according to the matching result and the first target detection result.

[0044] The present invention provides a computer-readable storage medium, wherein a plurality of instructions are stored in the storage medium; the plurality of instructions are used for a processor to load and execute the method as described above.

[0045] An electronic device provided by the present invention is characterized in that the electronic device comprises:

[0046] A processor, which is used to execute multiple instructions;

[0047] A memory for storing a plurality of instructions;

[0048] The plurality of instructions are used to be stored by the memory and loaded and executed by the processor to implement the method as described above.

[0049] Beneficial effects:

[0050] (1) The present invention utilizes the characteristics of forward-looking sonar images and adds Kalman filtering to perform target tracking. This does not increase excessive calculations to affect real-time performance, but can predict the target position of the next frame in real time, effectively predict the position of obscured targets, and distinguish targets.

[0051] (3) The present invention can effectively enhance the target feature extraction capability and reduce missed detection and false detection phenomena.

[0052] (3) The present invention extracts target features through hierarchical connection and module connection, connects the lower layer to the higher layer through jumps, and integrates multi-scale information across layers, which has higher detection accuracy and can better detect underwater targets that are difficult to identify.

[0053] (4) The present invention uses a Kalman filter to predict trajectories, combines neural network prediction information, and uses a cost function to comprehensively consider the trajectory prediction position and the detection prediction position, which can effectively identify occluded targets or distinguish target trajectories with similar features. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic diagram of the process flow of the underwater forward-looking sonar image target detection method of the present invention;

[0055] Figure 2 It is a schematic diagram of the architecture of the underwater forward-looking sonar image target detection method of the present invention;

[0056] Figure 3 Schematic diagram of the C2f module structure in YOLOv8 of the present invention;

[0057] Figure 4 It is a schematic diagram of hierarchical iterative connection of the present invention;

[0058] Figure 5 This is a schematic diagram of the module aggregation connection of the present invention;

[0059] Figure 6 Schematic diagram of the cost function of the present invention;

[0060] Figure 7 It is a structural schematic diagram of the underwater forward-looking sonar image target detection device of the present invention. DETAILED DESCRIPTION

[0061] The present invention is described in detail below in conjunction with the accompanying drawings and embodiments.

[0062] like Figure 1-Figure 2 As shown, the present invention proposes an underwater forward-looking sonar image target detection method, the method comprising:

[0063] Step S1: Acquire underwater forward-looking sonar data, and use the improved YOLOv8 model to perform target detection on the current frame forward-looking sonar image of the underwater forward-looking sonar data to obtain a first target detection result; Based on the current frame forward-looking sonar image, use the Kalman filter to predict the next frame forward-looking sonar image of the current frame forward-looking sonar image to obtain a predicted detection result;

[0064] Step S2: obtaining a next frame of forward-looking sonar image of the current frame as the forward-looking sonar image of the current frame, and performing target detection on the forward-looking sonar image of the current frame by using the improved YOLOv8 model to obtain a second target detection result;

[0065] Step S3: Match the predicted detection result and the second target detection result, and determine the trajectory of the target according to the matching result and the first target detection result.

[0066] In the present invention, the forward-looking sonar utilizes the principle of sound wave reflection to convert the sound wave signal reflected by the target into an image, thereby obtaining the position and shape of the target object.

[0067] Furthermore, if Figure 3-Figure 5 As shown, the step S1: acquiring underwater forward-looking sonar data, performing target detection on the current frame forward-looking sonar image of the underwater forward-looking sonar video by the improved YOLOv8 model, and obtaining a first target detection result, wherein:

[0068] The improved YOLOv8 model improves the feature extraction module in the YOLOv8 model; the feature extraction module in the improved YOLOv8 model includes multiple convolution blocks, each two convolution blocks form a group, there are NUM groups of convolution groups in total, and the two convolution blocks in each group are connected in sequence; the first convolution block in the first group of convolution groups receives the input of the C2f module, the second convolution block in the first group of convolution groups takes the output of the first convolution block in the first group of convolution groups as input, and fuses the output of the first convolution block in the first group of convolution groups and the output of the second convolution block in the first group of convolution groups to obtain the output of the first group of convolution groups; the second group of convolution groups to the N-1th group of convolution groups all take the output of the previous group of convolution groups as the input of the first convolution block in the group of convolution groups, and the second convolution block in the group of convolution groups takes The output of the first convolution block in the convolution group is taken as input, and the output of the first convolution block in the convolution group, the output of the second convolution block in the convolution group, and the output of the convolution group in the previous convolution group are fused to obtain the output of the convolution group; the NUM-th convolution group takes the output of the NUM-1-th convolution group as the input of the first convolution block in the convolution group, and the second convolution block in the convolution group takes the output of the first convolution block in the convolution group as input, and the output of the first convolution block in the convolution group, the output of the second convolution block in the convolution group, and the output of the convolution group in the previous convolution group are fused as the output of the C2f module in the improved YOLOv8 model, that is, the feature vector obtained by extracting features from the forward-looking sonar image of the current frame of the underwater forward-looking sonar video.

[0069] The convolution block includes a first convolution layer, a segmentation layer, a bottleneck layer module, a connection layer, and a second convolution layer; the bottleneck layer module includes a plurality of sequentially connected bottleneck layers; the first convolution layer convolves the input of the first convolution layer, the segmentation layer divides the output of the first convolution layer into a first part and a second part, the second part is sent to the bottleneck layer module, the connection layer splices the first part with the output of the bottleneck layer module, the spliced ​​result is convolved by the second convolution layer, and the convolution result of the second convolution layer is used as the output of the convolution block.

[0070] YOLOv8 proposed the C2f module. The input data is divided into two parts after passing through the first convolution layer. One part is directly passed to the output, and the other part is processed by multiple residual modules. Finally, the results of the two parts are spliced ​​in the channel dimension and passed through the second convolution layer to obtain the final output. Compared with the C3 module in the previous version of YOLO, C2f adds a residual module to obtain richer gradient flow information residual modules, such as Figure 3As shown in the figure. Although the semantic information of the deeper layer is richer, the detailed information is less. The present invention makes improvements by connecting the lower layer to the higher layer through jumps and connecting the context information across layers. The lowest layer features are aggregated and the next layer is merged iteratively. The aggregation is continuously iterated, and the low-level features are refined, as shown in the figure. Figure 4 As shown, the role of the convolution block can be expressed as:

[0071]

[0072] Among them, I is the level iteration function, s1;…; sn is each level, and N is the aggregation node.

[0073] Then the convolution blocks at different layers in the network are merged, the combined feature channels are retained, and the output of N is returned to the backbone as the input of the next layer, as Figure 5 As shown, the function of each convolution block can be expressed as:

[0074]

[0075] Where n is the depth and Conv represents a convolutional block.

[0076] The improved YOLOv8 model obtains five-level feature maps {P1, P2, P3, P4, P5} of decreasing size based on the feature vector extracted by the feature extraction module, and performs feature fusion through bottom-up and top-down paths, that is, feature maps P1, feature maps P2 and feature maps P3 are fused to obtain three-level feature maps {P3', P4', P5'}; the three-level feature maps are fused, and then the fused three-level feature maps are decoupled to obtain detection targets, each of which has three sizes.

[0077] In the present invention, the targets of the three sizes are input into the classification branch and the regression branch for calculation. The classification branch and the regression branch are the same as those in YOLOv8.

[0078] In step S1, based on the current frame forward-looking sonar image, a Kalman filter is used to predict the next frame forward-looking sonar image of the current frame forward-looking sonar image to obtain a predicted detection result, wherein:

[0079] The state transfer equation of the target motion used by the Kalman filter is:

[0080] {x t ,y t} = f trans ({x t-1 ,y t-1},θ)

[0081] x t =d x (xt-1 ) 2 +e x (x t-1 )+f x

[0082] y t =d x (y t-1 ) 2 +e x (y t-1 )+f y

[0083]

[0084] Among them, x t is the target horizontal coordinate at time t, y t is the target ordinate at time t, x t-1 is the target horizontal coordinate at time t-1, y t-1 is the target ordinate at time t-1, θ is the external control input, f trans The function is the state transfer equation, e x d x 、f x 、f y d y 、e y Function is the parameter of the equation to be solved.

[0085] In the present invention, each moment corresponds to a frame of forward-looking sonar image.

[0086] Underwater targets are unclear and easily obscured, which may cause the same target to be identified as different targets at different times. Therefore, in addition to paying attention to the appearance characteristics of the target, it is particularly important to add the temporal motion information of the target. The present invention adds an unscented Kalmar filter to predict the target trajectory for rapid estimation of the target motion trend, and uses an unscented transformation to process the Sigma point set so that the mean and covariance of the Sigma point set match the statistical characteristics of the original system, and then uses the Sigma point set for nonlinear mapping to obtain an approximate state probability density function. When using the Kalman filter, it is necessary to give the state transfer equation of the target, and establish a separate state transfer equation for each historical information of the target during motion.

[0087] The step S3: matching the predicted detection result and the second target detection result, comprises:

[0088] Step S31: record the second target detection result as R t ={r1,r2,...,r n}, where r i =(b i ,s i ,ci ), R t represents the detection result corresponding to the tth frame, r1,r2,...,r n is the n targets included in the detection result; 1≤i≤n, b i Represents the bounding box corresponding to the i-th target (x1 i ,y1 i ,w1 i ,h1 i ) where x1 i is the horizontal coordinate of the upper left corner of the bounding box corresponding to the i-th target, y1 i is the ordinate of the upper left corner of the bounding box corresponding to the i-th target, w1 i is the width of the bounding box corresponding to the i-th target, h1 i is the height of the bounding box corresponding to the i-th target, s i is the detection score corresponding to the i-th target, c i is the target category corresponding to the i-th target;

[0089] The predicted detection result is recorded as {o1, o2, ..., o n},o m To predict the mth target in the detection result;

[0090] The target r in the second target detection result i And predict the target o in the detection result m Form a pair to be matched;

[0091] Step S32: For each to-be-matched pair, a cost function is calculated. The cost function is as follows:

[0092]

[0093] Among them, I i is the matching value of the pair to be matched, B i is the target r to be matched i The corresponding prediction box, G m is the target o to be matched m The corresponding target frame, b is the target r to be matched i The center point of the corresponding prediction box, b m is the target o to be matched m The corresponding target box center point, d is B i and G m The diagonal distance of the minimum enclosing rectangle, s i is the target to be matched i The corresponding detection score;

[0094] If the I calculated by the to-be-matched pair iexceeds the preset threshold, then the target r in the second target detection result i And predict the target o in the detection result m Matching success;

[0095] Step S33: If there is a target that matches the second target detection result and the predicted detection result, the target is determined to be a valid target.

[0096] This cost function is not affected by the distance and the near view and has scale invariance. It can effectively improve the accuracy of target detection and effectively solve problems such as target loss and predicted position offset. The experimental results are shown in Table 1.

[0097] Table 1 Improved effectiveness experiment

[0098]

[0099] Furthermore, in step S3, determining the trajectory of the target according to the matching result and the first target detection result includes:

[0100] Step S31: setting a valid target set and an invalid target set, wherein the valid target set records the identifiers of each valid target, the number of successful matches of the valid target, and the frame numbers of the current frame and the previous frame corresponding to each successful match of the valid target; and the invalid target set records the identifiers of the targets that do not appear in the valid target set;

[0101] Step S32: For each valid target in the valid target set, the following operations are performed:

[0102] Obtain the maximum frame number among all frame numbers corresponding to all successful matches of the valid target;

[0103] If the difference between the frame number of the current frame and the maximum frame number is greater than or equal to 3, the valid target set clears all information of the valid target and records the cleared valid target into the invalid target set; the full information includes the identifier of the valid target, the number of successful matches of the valid target, and the frame numbers of the current frame and the previous frame corresponding to the successful matches of the valid target; if there is a formal trajectory corresponding to the valid target in the trajectory list, the formal trajectory is deleted;

[0104] If the difference between the frame number of the current frame and the maximum frame number is less than 3, when the valid target has no corresponding formal track:

[0105] When the number of successful matches of the valid target is 1, a preliminary trajectory is generated according to the trajectory of the valid target, and the parameters of the preliminary trajectory are updated;

[0106] When the number of successful matches of the valid target is 2, generating a formal trajectory based on the preliminary trajectory, assigning a trajectory number to the formal trajectory, storing the formal trajectory in a trajectory list, and outputting the formal trajectory;

[0107] If the difference between the frame number of the current frame and the maximum frame number is less than 3, when the valid target has a corresponding formal track:

[0108] Based on the official trajectory, the predicted detection result corresponding to the official trajectory, and the target in the second target detection result, the official trajectory is updated and the official trajectory is output;

[0109] Step S33: Obtain all successfully matched targets in the predicted detection result and the second target detection result, and obtain each formal track corresponding to all the successfully matched targets; for each formal track, perform the following operations:

[0110] Based on the official trajectory, the predicted detection result corresponding to the official trajectory, and the target in the second target detection result, target tracking parameters are updated, where the target tracking parameters are used to guide tracking of the target.

[0111] Furthermore, the present invention is deployed on a domestic platform. That is, it is deployed on a domestic server, converted to a domestic chip general model, and the MNNConvert tool converts the ".onnx" format file into a ".mnn" format file, and then calls the detection model in C++, and realizes trajectory prediction and position calibration according to the above steps, effectively solving the problem of occluded target loss.

[0112] The present invention also provides an underwater forward-looking sonar image target detection device, the device comprising:

[0113] Prediction module: configured to obtain underwater forward-looking sonar data, perform target detection on a current frame forward-looking sonar image of the underwater forward-looking sonar data by an improved YOLOv8 model, and obtain a first target detection result; based on the current frame forward-looking sonar image, perform prediction on a next frame forward-looking sonar image of the current frame forward-looking sonar image by a Kalman filter, and obtain a predicted detection result;

[0114] The detection module is configured to obtain a next frame of forward-looking sonar image of the current frame as the forward-looking sonar image of the current frame, and perform target detection on the forward-looking sonar image of the current frame by the improved YOLOv8 model to obtain a second target detection result;

[0115] Matching module: configured to match the predicted detection result and the second target detection result, and determine the trajectory of the target according to the matching result and the first target detection result.

[0116] The above specific embodiments only describe the design principle of the present invention. The shapes and names of the components in the description may be different and are not limited. Therefore, those skilled in the art in the field of the present invention may modify or replace the technical solutions recorded in the above embodiments; and these modifications and replacements do not deviate from the creative purpose and technical solutions of the present invention and should all fall within the protection scope of the present invention.

Claims

1. A method for underwater forward-looking sonar image target detection, characterized in that: The method comprises: Step S1: Acquire underwater forward-looking sonar data, and use the improved YOLOv8 model to perform target detection on the current frame forward-looking sonar image of the underwater forward-looking sonar data to obtain a first target detection result; Based on the current frame forward-looking sonar image, use the Kalman filter to predict the next frame forward-looking sonar image of the current frame forward-looking sonar image to obtain a predicted detection result; Step S2: obtaining a next frame of forward-looking sonar image of the current frame as the forward-looking sonar image of the current frame, and performing target detection on the forward-looking sonar image of the current frame by using the improved YOLOv8 model to obtain a second target detection result; Step S3: Match the predicted detection result and the second target detection result, and determine the trajectory of the target according to the matching result and the first target detection result.

2. The method according to claim 1, characterized in that The step S1: acquiring underwater forward-looking sonar data, performing target detection on the current frame forward-looking sonar image of the underwater forward-looking sonar video by using the improved YOLOv8 model, and obtaining a first target detection result, wherein: The improved YOLOv8 model improves the feature extraction module in the YOLOv8 model; the feature extraction module in the improved YOLOv8 model includes multiple convolution blocks, each two convolution blocks form a group, there are NUM groups of convolution groups in total, and the two convolution blocks in each group are connected in sequence; the first convolution block in the first group of convolution groups receives the input of the C2f module, the second convolution block in the first group of convolution groups takes the output of the first convolution block in the first group of convolution groups as input, and fuses the output of the first convolution block in the first group of convolution groups and the output of the second convolution block in the first group of convolution groups to obtain the output of the first group of convolution groups; the second group of convolution groups to the N-1th group of convolution groups all take the output of the previous group of convolution groups as the input of the first convolution block in the group of convolution groups, and the second convolution block in the group of convolution groups takes The output of the first convolution block in the convolution group is taken as input, and the output of the first convolution block in the convolution group, the output of the second convolution block in the convolution group, and the output of the convolution group in the previous convolution group are fused to obtain the output of the convolution group; the NUM-th convolution group takes the output of the NUM-1-th convolution group as the input of the first convolution block in the convolution group, and the second convolution block in the convolution group takes the output of the first convolution block in the convolution group as input, and the output of the first convolution block in the convolution group, the output of the second convolution block in the convolution group, and the output of the convolution group in the previous convolution group are fused as the output of the C2f module in the improved YOLOv8 model, that is, the feature vector obtained by extracting features from the forward-looking sonar image of the current frame of the underwater forward-looking sonar video.

3. The method according to claim 1, characterized in that In step S1, based on the current frame forward-looking sonar image, a Kalman filter is used to predict the next frame forward-looking sonar image of the current frame forward-looking sonar image to obtain a predicted detection result, wherein: The state transfer equation of the target motion used by the Kalman filter is: {x t ,y t }=f trans ({x t-1 ,y t-1 },θ) x t =d x (x t-1 ) 2 +e x (x t-1 )+f x y t =d x (y t-1 ) 2 +e x (y t-1 )+f y Among them, x t is the target horizontal coordinate at time t, y t is the target ordinate at time t, x t-1 is the target horizontal coordinate at time t-1, y t-1 is the target ordinate at time t-1, θ is the external control input, f trans The function is the state transfer equation, e x ,d x 、f x 、f y ,d y 、e y Function is the parameter of the equation to be solved.

4. The method according to any one of claims 1 to 3, characterized in that The step S3: matching the predicted detection result and the second target detection result, comprises: Step S31: record the second target detection result as R t ={r1,r2,...,r n }, where r i =(b i ,s i ,c i ), R t represents the detection result corresponding to the tth frame, r1,r2,...,r n is the n targets included in the detection result; 1≤i≤n, b i Represents the bounding box corresponding to the i-th target (x1 i ,y1 i ,w1 i ,h1 i ) where x1 i is the horizontal coordinate of the upper left corner of the bounding box corresponding to the i-th target, y1 i is the ordinate of the upper left corner of the bounding box corresponding to the i-th target, w1 i is the width of the bounding box corresponding to the i-th target, h1 i is the height of the bounding box corresponding to the i-th target, s i is the detection score corresponding to the i-th target, c i is the target category corresponding to the i-th target; The predicted detection result is recorded as {o1, o2, ..., o n },o m To predict the mth target in the detection result; The target r in the second target detection result i And predict the target o in the detection result m Form a pair to be matched; Step S32: For each to-be-matched pair, a cost function is calculated. The cost function is as follows: Among them, I i is the matching value of the pair to be matched, B i is the target to be matched i The corresponding prediction box, G m is the target o to be matched m The corresponding target frame, b is the target r to be matched i The center point of the corresponding prediction box, b m is the target o to be matched m The corresponding target box center point, d is B i and G m The diagonal distance of the minimum enclosing rectangle, s i is the target to be matched i The corresponding detection score; If the I calculated by the to-be-matched pair i exceeds the preset threshold, then the target r in the second target detection result i And predict the target o in the detection result m Matching success; Step S33: If there is a target that matches the second target detection result and the predicted detection result, the target is determined to be a valid target.

5. The method according to claim 4, characterized in that The step S3, determining the trajectory of the target according to the matching result and the first target detection result, comprises: Step S31: setting a valid target set and an invalid target set, wherein the valid target set records the identifiers of each valid target, the number of successful matches of the valid target, and the frame numbers of the current frame and the previous frame corresponding to each successful match of the valid target; and the invalid target set records the identifiers of the targets that do not appear in the valid target set; Step S32: For each valid target in the valid target set, the following operations are performed: Obtain the maximum frame number among all frame numbers corresponding to all successful matches of the valid target; If the difference between the frame number of the current frame and the maximum frame number is greater than or equal to 3, the valid target set clears all information of the valid target and records the cleared valid target into the invalid target set; the full information includes the identifier of the valid target, the number of successful matches of the valid target, and the frame numbers of the current frame and the previous frame corresponding to the successful matches of the valid target; if there is a formal trajectory corresponding to the valid target in the trajectory list, the formal trajectory is deleted; If the difference between the frame number of the current frame and the maximum frame number is less than 3, when the valid target has no corresponding formal track: When the number of successful matches of the valid target is 1, a preliminary trajectory is generated according to the trajectory of the valid target, and the parameters of the preliminary trajectory are updated; When the number of successful matches of the valid target is 2, generating a formal trajectory based on the preliminary trajectory, assigning a trajectory number to the formal trajectory, storing the formal trajectory in a trajectory list, and outputting the formal trajectory; If the difference between the frame number of the current frame and the maximum frame number is less than 3, when the valid target has a corresponding formal track: Based on the official trajectory, the predicted detection result corresponding to the official trajectory, and the target in the second target detection result, the official trajectory is updated and the official trajectory is output; Step S33: Obtain all successfully matched targets in the predicted detection result and the second target detection result, and obtain each formal track corresponding to all the successfully matched targets; for each formal track, perform the following operations: Based on the official trajectory, the predicted detection result corresponding to the official trajectory, and the target in the second target detection result, target tracking parameters are updated, where the target tracking parameters are used to guide tracking of the target.

6. An underwater forward-looking sonar image target detection device, characterized in that: The device comprises: Prediction module: configured to obtain underwater forward-looking sonar data, perform target detection on a current frame forward-looking sonar image of the underwater forward-looking sonar data by an improved YOLOv8 model, and obtain a first target detection result; based on the current frame forward-looking sonar image, perform prediction on a next frame forward-looking sonar image of the current frame forward-looking sonar image by a Kalman filter, and obtain a predicted detection result; A detection module is configured to obtain a next frame of forward-looking sonar image of a current frame as the forward-looking sonar image of the current frame, and perform target detection on the forward-looking sonar image of the current frame by the improved YOLOv8 model to obtain a second target detection result; Matching module: configured to match the predicted detection result and the second target detection result, and determine the trajectory of the target according to the matching result and the first target detection result.

7. A computer-readable storage medium, wherein a plurality of instructions are stored in the storage medium; the plurality of instructions are used for a processor to load and execute the method as claimed in any one of claims 1 to 5.

8. An electronic device, characterized in that: The electronic device comprises: A processor, which is used to execute multiple instructions; A memory for storing a plurality of instructions; The plurality of instructions are used to be stored in the memory and loaded and executed by the processor according to any one of claims 1 to 5.

Images