Target identification method and device based on sonar image, equipment and storage medium

Abstract

The invention belongs to the field of artificial intelligence and financial science and technology, and relates to a target recognition method based on a sonar image, which comprises the steps of performing frame extraction processing on a collected to-be-detected sonar image, inputting an obtained frame extraction image into a trained target detection model, the target detection model comprising a backbone network, a neck network and a head network, and carrying out feature extraction, feature fusion and prediction on the frame extraction image through the backbone network, the neck network and the head network to obtain a target identification result, and carrying out statistics on the types and the number of targets according to the target identification result. The invention further provides a target recognition device and equipment based on the sonar image and a storage medium. The method can improve the comprehensiveness and accuracy of target detection, further improves the efficiency and accuracy of quantity statistics, and is simple, convenient and objective to operate.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and financial technology, and in particular to a target recognition method, device, equipment and storage medium based on sonar images. Background Art

[0002] In the field of agricultural insurance, it has always been a difficult problem to accurately inspect and determine the loss of aquatic products. Currently, there are four common methods for inspecting and determining the loss of aquatic products:

[0003] 1) Pond cleaning method: drain all the water in the breeding pond, and the surveyor will count the aquatic products in the pond one by one and determine the loss of dead aquatic products.

[0004] 2) Fishing method: Generally, dead fish will float to the water surface naturally. The floating fish will be caught and brought ashore and counted at one time.

[0005] 3) Sampling loss assessment method: This method is often used in large-scale aquaculture in lakes, reservoirs, rice fields and tidal flats.

[0006] 4) Empirical damage measurement method: The policyholder and the insurer estimate the damage to aquatic products based on past experience.

[0007] Among the above four methods, the pond clearing method is limited to a local breeding area and is difficult to operate, time-consuming, and labor-intensive. In addition, during the pumping process, the oxygen content in the water decreases due to the reduction in water volume, and some aquatic products will die due to lack of water, increasing the number of aquatic product deaths and causing secondary damage to aquatic products. The fishing method requires aquatic products to float, but it is not applicable to aquatic products such as shellfish, shrimp, and crabs. The sampling loss assessment error is large, and the accuracy of the loss assessment cannot be guaranteed. The empirical loss measurement method has a large space for human manipulation, the results are not objective, and the error is large.

[0008] Therefore, there is an urgent need for an objective and practical target recognition method with high estimation accuracy, saving a lot of manpower and material resources to estimate the quantity of aquatic products. Summary of the invention

[0009] The purpose of the embodiments of the present application is to propose a target recognition method, device, computer equipment and storage medium based on sonar images to solve the technical problems that the methods of aquatic product detection and recognition in the prior art have low accuracy and efficiency, consume a lot of manpower and material resources, and are not practical enough.

[0010] In order to solve the above technical problems, the embodiment of the present application provides a target recognition method based on sonar images, which adopts the following technical solutions:

[0011] Collecting the sonar image to be detected in the target area, and extracting frames of the sonar image to be detected at preset time intervals to obtain an extracted frame image;

[0012] Inputting the extracted frame image into a trained target detection model, wherein the target detection model includes a backbone network, a neck network, and a head network;

[0013] Extract features of different scales from the frame-sampling image through the backbone network to obtain a multi-scale feature map, and input the multi-scale feature map into the neck network;

[0014] Performing feature fusion attention enhancement on the multi-scale feature map through the neck network to obtain a fused attention feature map;

[0015] Performing target detection on the fused attention feature map through the head network, and outputting a target recognition result, wherein the target recognition result includes a target type, a target position, and a target rotation angle;

[0016] According to the target recognition result, the target types and the number corresponding to each target type are counted.

[0017] In order to solve the above technical problems, the embodiment of the present application further provides a target recognition device based on sonar images, which adopts the following technical solution:

[0018] A frame extraction module is used to collect the sonar image to be detected in the target area, and extract frames of the sonar image to be detected at intervals according to a preset time interval to obtain a frame extracted image;

[0019] An input module, used to input the frame-sampling image into a trained target detection model, wherein the target detection model includes a backbone network, a neck network and a head network;

[0020] A feature extraction module, used to extract features of different scales on the frame-sampling image through the backbone network to obtain a multi-scale feature map, and input the multi-scale feature map into the neck network;

[0021] A feature fusion module, used for performing feature fusion attention enhancement on the multi-scale feature map through the neck network to obtain a fused attention feature map;

[0022] A detection and recognition module, configured to perform target detection on the fused attention feature map through the head network and output a target recognition result, wherein the target recognition result includes a target type, a target position, and a target rotation angle;

[0023] The statistical module is used to count the target types and the number corresponding to each target type according to the target recognition result.

[0024] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0025] The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the target recognition method based on sonar images when executing the computer-readable instructions.

[0026] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0027] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the target recognition method based on sonar images as described above are implemented.

[0028] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0029] The present application provides a target recognition method based on sonar images. The collected sonar images to be detected are subjected to frame extraction processing, and the obtained frame extraction images are input into a trained target detection model. The target detection model includes a backbone network, a neck network and a head network. The frame extraction images are subjected to feature extraction, feature fusion and prediction through the backbone network, the neck network and the head network to obtain target recognition results, and the types and quantities of the targets are counted according to the target recognition results. The frame extraction images are subjected to multi-scale feature extraction through the backbone network, so that richer and more distinguishable features can be extracted from the images, which is helpful to improve the accuracy of target detection. Feature maps of different scales are fused through the neck network, so that the dependency between features can be enhanced to balance the semantic information of the image, thereby improving the detection capability of the model for targets of different sizes. Targets of different scales are predicted through the head network, so that the comprehensiveness and accuracy of target detection are further improved, and at the same time, the efficiency and accuracy of quantitative statistics can be improved. The operation is simple, convenient and objective, and a lot of manpower and material resources are saved. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0032] Figure 2 is a flow chart of an embodiment of a target recognition method based on sonar images according to the present application;

[0033] Figure 3is a schematic diagram of the structure of the target detection model provided in the embodiment of the present application;

[0034] Figure 4 is a schematic structural diagram of an embodiment of a target recognition device based on sonar images according to the present application;

[0035] Figure 5 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0037] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0038] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0039] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0040] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0041] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0042] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0043] It should be noted that the target recognition method based on sonar images provided in the embodiments of the present application is generally executed by a server / terminal device, and accordingly, the target recognition device based on sonar images is generally arranged in the server / terminal device.

[0044] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0045] Continue to refer Figure 2 , shows a flow chart of an embodiment of a target recognition method based on sonar images according to the present application, comprising the following steps:

[0046] Step S201, collecting the sonar image to be detected in the target area, and extracting frames of the sonar image to be detected at preset time intervals to obtain an extracted frame image.

[0047] In this embodiment, the target recognition method based on sonar images is executed on the electronic device (for example Figure 1 The server / terminal device shown in the figure can obtain the sonar image to be detected through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0048] The target area is an aquaculture area where the quantity of aquatic products needs to be estimated. The sonar image can be obtained by mapping the target aquaculture area with a sonar detector. Specifically, four sonar detectors placed diagonally in the target aquaculture area can be used to map the target aquaculture area, thereby obtaining the sonar image to be detected.

[0049] The sonar image to be detected is subjected to interval frame extraction at a preset time interval, wherein interval frame extraction (Interval Frame Extraction) refers to extracting specific frames (i.e., frame-extracted images) from a continuous video stream at a certain time interval. For example, the preset time interval can be selected as 2s, 5s, etc., and can be set according to actual conditions.

[0050] In some optional implementations of the present embodiment, the head image, middle image and tail image are respectively extracted at preset time intervals for the sonar image to be detected, and the head frame, middle frame and tail frame are obtained accordingly; the head frame, middle frame and tail frame are combined to obtain an extracted frame image.

[0051] Among them, head image frame extraction refers to extracting frames from the beginning of the image to obtain the key information of the head of the image; middle image frame extraction refers to extracting frames from the middle part of the image to obtain the key information in the middle of the image; tail image frame extraction refers to extracting frames from the end of the image to obtain the key information of the end of the image.

[0052] By extracting the head image, the middle image and the tail image, the demand for storage and processing resources can be reduced, resource allocation can be optimized, and the frame extraction efficiency can be improved. At the same time, it is helpful to observe and analyze the target's behavioral trends.

[0053] Step S202, inputting the frame-sampling image into the trained target detection model, wherein the target detection model includes a backbone network, a neck network and a head network.

[0054] In this embodiment, the target detection model is an improved YOLOv9 network model, see Figure 3As shown in the figure, it includes a backbone network, a neck network, a head network and auxiliary branches. Among them, the backbone network (Backbone) is responsible for feature extraction of the input image, and the feature map output by the backbone network will be passed to the neck network for further processing; the neck network (Neck) is mainly used to fuse the feature map output by the backbone network, and fuse feature maps of different scales through upsampling, downsampling and other operations to improve the model's detection ability for targets of different sizes; the head network (Head) is used to predict target types and regress bounding boxes through multi-scale detection; the auxiliary branch helps the network to better learn and update weights during training by providing additional gradient paths, which helps to retain more information in the deep layers of the network and generate more reliable gradients, thereby ensuring a more accurate association between targets and inputs during training. It should be noted that the auxiliary branches are beneficial in the training stage, but are usually abandoned in the reasoning and testing stages, leaving only the standard backbone network, neck network, and head network structure.

[0055] Step S203, extracting features of different scales from the frame-sampling image through the backbone network to obtain a multi-scale feature map, and inputting the multi-scale feature map into the neck network.

[0056] In this embodiment, the EMA attention mechanism is introduced at the end of the backbone network. EMA is an efficient multi-scale attention mechanism based on cross-space learning. It rearranges some channels to the batch dimension and divides the channel dimension into multiple subsets to reduce the computational burden while maintaining the information of each channel. The EMA attention mechanism readjusts the channel weights in each parallel branch by encoding global information and captures the interaction between pixels through the interaction between different dimensions.

[0057] Specifically, the backbone network includes a first CBS convolutional layer, a second CBS convolutional layer, a first RepNCSPELAN4 layer, a first downsampling module, a second RepNCSPELAN4 layer, a second downsampling module, a third RepNCSPELAN4 layer, a third downsampling module, a fourth RepNCSPELAN4 layer, an EMA attention layer and a SPPELAN layer.

[0058] In some optional implementations of this embodiment, the step of extracting features of different scales on the frame-sampling image through the backbone network to obtain a multi-scale feature map includes:

[0059] The first CBS convolutional layer is used to convolve the frame image to obtain the first-level convolutional feature map;

[0060] The first-level convolutional feature map is convolved through the second CBS convolutional layer to obtain a second-level convolutional feature map;

[0061] The second-level convolution feature map is subjected to feature extraction and fusion through the first RepNCSPELAN4 layer to obtain the first convolution fusion feature map;

[0062] Performing a downsampling operation on the first convolution fusion feature map through a first downsampling module to obtain a first downsampled convolution feature map;

[0063] Perform feature extraction and fusion on the first down-sampling convolution feature map through the second RepNCSPELAN4 layer to obtain a second convolution fusion feature map, and input the second convolution fusion feature map into the neck network and the second down-sampling module;

[0064] Performing a downsampling operation on the second convolution fusion feature map through a second downsampling module to obtain a second downsampled convolution feature map;

[0065] Perform feature extraction and fusion on the second down-sampling convolution feature map through the third RepNCSPELAN4 layer to obtain a third convolution fusion feature map, and input the third convolution fusion feature map into the neck network and the third down-sampling module;

[0066] Performing a downsampling operation on the third convolution fusion feature map through a third downsampling module to obtain a third downsampling convolution feature map;

[0067] Perform feature extraction and fusion on the third down-sampled convolution feature map through the fourth RepNCSPELAN4 layer to obtain a fourth convolution fusion feature map;

[0068] The EMA attention layer is used to calculate the attention of the fourth convolution fusion feature map to obtain a multi-scale attention feature map.

[0069] The multi-scale attention feature map is extracted through the SPPELAN layer to obtain a multi-scale feature map.

[0070] Among them, the CBS convolution layer mainly includes three parts: the convolution layer (Conv Layer), the batch normalization layer (BatchNormalization Layer, referred to as BN layer) and the activation function layer (SiLU activation function). The CBS convolution layer is responsible for transforming the number of channels of the input image, feature extraction, and transforming the size of the feature map. The size transformation of the feature map is achieved by adjusting the stride step size parameter.

[0071] The RepNCSPELAN4 layer is a cross-level fusion layer. After a series of convolution operations, the features of the front part are connected with the features of the back part to achieve feature fusion at different stages. Specifically, the RepNCSPELAN4 layer is mainly composed of Conv (convolution layer) and RepNCSP (reparameterized CSP module). The RepNCSP module contains multiple RepNBottleneck (reparameterized bottleneck modules), which enhance the reuse and expression capabilities of features through stacking and residual connections (shortcut). The input feature map (including the second-level convolution feature map, the first down-sampled convolution feature map, the second down-sampled convolution feature map, and the third down-sampled convolution feature map) is first preliminarily processed by the Conv layer to adjust the number of channels and size of the feature map to obtain a preliminary processed feature map. The processed feature map is divided into two parts, one of which is input into the RepNCSP module, and the features are extracted and fused by stacking multiple internal RepNBottleneck modules, and the output feature map is spliced ​​with the other part.

[0072] The downsampling module (ADown) process includes average pooling with a step size of 1 for the input convolution fusion feature map (including the first convolution fusion feature map, the second convolution fusion feature map and the third convolution fusion feature map), and then dividing them into two groups on average. One group undergoes a convolution with a step size of 2; the other group undergoes a maximum pooling and then a convolution; finally, the two groups of output feature maps are spliced ​​and output to obtain the corresponding downsampled convolution feature map.

[0073] The attention calculation is performed on the fourth convolution fusion feature map through the EMA attention layer. Specifically, the fourth convolution fusion feature map is input into two different processing processes: the first process uses one-dimensional global pooling to extract information, and the second process obtains features through a 3x3 convolution layer. The outputs of these two processes are then input into the sigmoid activation function and normalization steps for adjustment. These adjusted features are then fused through a cross-dimensional interaction module, whose goal is to capture the interaction between pixels. After this series of processing, the feature map will be subjected to the final sigmoid function adjustment, with the aim of capturing more critical features, thereby generating the final multi-scale attention feature map.

[0074] In this embodiment, the SPPELAN layer combines the characteristics of spatial pyramid pooling (SPP) and efficient local aggregation network (ELAN). SPP is first used to extract features at different scales, and then these features are locally aggregated through ELAN. It can more effectively extract feature information at different scales and maintain sensitivity to important local features in the image.

[0075] By introducing the backbone network of the EMA attention mechanism to perform feature extraction on the frame-by-frame images, it is possible to more effectively extract feature information of different scales, enhance the ability to extract key features of small targets, and help improve the model's ability to detect targets of different sizes.

[0076] Step S204, performing feature fusion attention enhancement on the multi-scale feature map through the neck network to obtain a fused attention feature map.

[0077] In this embodiment, a hybrid attention mechanism is introduced into the neck network, including a channel attention mechanism (ECANet) and a spatial attention mechanism (Spatial Attention) connected in series, wherein the channel attention mechanism ECANet performs global average pooling on the input feature map, and the feature map is changed from a matrix of [h, w, c] to a vector of [1, 1, c], and then an adaptive one-dimensional convolution kernel size kernel_size is calculated according to the number of channels of the feature map, and then kernel_size is used in the one-dimensional convolution to obtain the weight for each channel of the feature map, and finally the normalized weight is multiplied by the original input feature map channel by channel to generate a weighted feature map; the spatial attention mechanism Spatial Attention obtains spatial description information from the input feature map, and then uses convolution operation and activation function to map the spatial description information to spatial attention weights, and then multiplies the spatial attention weights by the input feature map to achieve weighted combination of different spatial positions, and finally obtains a feature map enhanced by the spatial attention mechanism.

[0078] In some optional implementations, the neck network includes a first neck branch and a second neck branch, the first neck branch includes a first upsampling layer (Upsample), a first connection layer (Concat), a fifth RepNCSPELAN4 layer, a second upsampling layer, a second connection layer, a sixth RepNCSPELAN4 layer, a fourth downsampling module, a third connection layer, a seventh RepNCSPELAN4 layer, a fifth downsampling module, a fourth connection layer and an eighth RepNCSPELAN4 layer, wherein the sixth RepNCSPELAN4 layer, the seventh RepNCSPELAN4 layer and the eighth RepNCSPELAN4 layer are each connected to a mixed attention layer, which are the first mixed attention layer, the second mixed attention layer and the third mixed attention layer, respectively.

[0079] Among them, the first connection layer is connected to the third RepNCSPELAN4 layer, the second connection layer is connected to the second RepNCSPELAN4 layer, the third connection layer is connected to the fifth RepNCSPELAN4 layer, and the fourth connection layer is connected to the SPPELAN layer.

[0080] In this embodiment, the step of fusing the multi-scale feature map with the attention feature map through the neck network to obtain the fused enhanced feature map includes:

[0081] Performing an upsampling operation on the multi-scale feature map through a first upsampling layer to obtain a first upsampling feature map;

[0082] The third convolution fusion feature and the first up-sampled feature map are spliced ​​through the first connection layer to obtain a first spliced ​​feature map;

[0083] The first concatenated feature map is fused and enhanced by the fifth RepNCSPELAN4 layer to obtain a first enhanced fused feature map, and the first enhanced fused feature map is input into the second upsampling layer and the third connection layer;

[0084] Performing an upsampling operation on the first enhanced fusion feature map through a second upsampling layer to obtain a second upsampling feature map;

[0085] The second convolution fusion feature and the second up-sampled feature map are concatenated through a second connection layer to obtain a second concatenated feature map;

[0086] The second concatenated feature map is fused and enhanced by the sixth RepNCSPELAN4 layer to obtain a second enhanced fused feature map, and the second enhanced fused feature map is input into the fourth downsampling module and the first mixed attention layer;

[0087] Perform attention calculation on the second enhanced fused feature map through the first mixed attention layer, and obtain the first fused attention feature map to input into the head network;

[0088] Downsampling the second enhanced fused feature map by a fourth downsampling module to obtain a first downsampled fused feature map;

[0089] The first down-sampled fused feature map and the first enhanced fused feature map are spliced ​​through a third connection layer to obtain a third spliced ​​feature map;

[0090] The third concatenated feature map is fused and enhanced through the seventh RepNCSPELAN4 layer to obtain a third enhanced fused feature map, and the third enhanced fused feature map is input into the fifth downsampling module and the second mixed attention layer;

[0091] Perform attention calculation on the third enhanced fused feature map through the second mixed attention layer, and obtain the second fused attention feature map to input into the head network;

[0092] Downsampling the third enhanced fused feature map through a fifth downsampling module to obtain a second downsampled fused feature map;

[0093] The second down-sampled fusion feature map and the multi-scale feature map are spliced ​​through a fourth connection layer to obtain a fourth spliced ​​feature map;

[0094] The fourth concatenated feature map is fused and enhanced by the eighth RepNCSPELAN4 layer to obtain a fourth enhanced fused feature map;

[0095] The fourth enhanced fused feature map is subjected to attention calculation through the third mixed attention layer, and the third fused attention feature map is obtained and input into the head network.

[0096] In this example, the first mixed attention layer sequentially performs channel attention calculation and spatial attention calculation on the second enhanced fused feature map to obtain the first fused attention feature map; the second mixed attention layer sequentially performs channel attention calculation and spatial attention calculation on the third enhanced fused feature map to obtain the second fused attention feature map; the third mixed attention layer sequentially performs channel attention calculation and spatial attention calculation on the fourth enhanced fused feature map to obtain the third fused attention feature map.

[0097] Among them, the first fused attention feature map, the second fused attention feature map and the third fused attention feature map are feature maps of different scales. For example, the size of the first fused attention feature map is 80×80×256, the size of the second fused attention feature map is 40×40×512, and the size of the third fused attention feature map is 20×20×512. Feature maps of different scales are input into the head network to recognize objects of different sizes.

[0098] By extracting more complex features in the image through the neck network and fusing them with the already extracted feature information, the feature representation ability and the richness of semantic information can be increased to improve the accuracy of target detection.

[0099] Step S205, performing target detection on the fused attention feature map through the head network, and outputting a target recognition result, which includes the target type, target position, and target rotation angle.

[0100] In this embodiment, the LSK attention mechanism is introduced into the head network to capture small target aquatic products. The head network includes a first detection head, a second detection head, a third detection head and a classification prediction layer.

[0101] In some optional implementations of this embodiment, the step of performing target detection on the fused attention feature map through the head network and outputting the target recognition result includes:

[0102] By using the first detection head, performing target category prediction and bounding box prediction according to the first fused attention feature map, a first target feature map is obtained;

[0103] By using the second detection head, performing target category prediction and bounding box prediction according to the second fused attention feature map, a second target feature map is obtained;

[0104] By using the third detection head, performing target category prediction and bounding box prediction according to the third fused attention feature map, a third target feature map is obtained;

[0105] The first target feature map, the second target feature map and the third target feature map are input into the classification prediction layer for classification and regression to obtain the target recognition result.

[0106] Three detection heads are used to detect targets of different scales: the first detection head is responsible for detecting large-sized targets, which can increase the model's ability to detect large targets; the second detection head is responsible for detecting medium-sized targets, which helps the model locate medium-sized targets more accurately; the third detection head is responsible for detecting small-sized targets, which can improve the model's detection sensitivity to small targets.

[0107] Specifically, the first detection head, the second detection head and the third detection head independently include a bounding box prediction branch and a target category prediction branch, wherein the bounding box prediction branch includes a first convolution block, a first LSK attention layer and a Bbox Loss layer connected end to end in sequence, and the target category prediction branch includes a second convolution block, a second LSK attention layer and a Cls Loss layer connected end to end in sequence.

[0108] Taking the first detection head as an example, the step of performing target category prediction and bounding box prediction according to the first fused attention feature map through the first detection head to obtain the first target feature map includes:

[0109] Input the first fused attention feature map into the bounding box prediction branch and the target category prediction branch of the first detection head respectively;

[0110] The first fused attention feature map is predicted through the first convolution block, the first LSK attention layer and the Bbox Loss layer to obtain a bounding box feature map;

[0111] The first fusion attention feature map is predicted through the second convolution block, the second LSK attention layer and the Cls Loss layer to obtain the target category feature map;

[0112] The bounding box feature map and the target category feature map are output as the first target feature map.

[0113] Among them, the first convolution block and the second convolution block respectively include a convolution layer and a 2D convolution layer connected end to end in sequence, and the LSK attention layer is connected after the corresponding 2D convolution layer.

[0114] The convolution layer and 2D convolution layer are used to extract and integrate the information on the feature map to generate the final detection result. The bounding box prediction branch outputs the regression value of the bounding box, including the center point coordinates, width, height and possible rotation angle (for the oriented bounding box OBB) of the bounding box; the target category prediction branch outputs the category prediction of the target in each bounding box.

[0115] In this embodiment, the working principles of the first LSK attention layer and the second LSK attention layer are as follows:

[0116] The feature map obtained after the first fusion attention feature map is processed by the convolution layer and the 2D convolution layer is input into the LSK attention layer. The depth-separable convolution output of the LSK attention layer is the attention map attn1. The attention map attn1 is output as the attention map attn2 through the dilated convolution. The number of channels of the attention map attn1 and the attention map attn2 is halved through convolution. Then, the attention map attn1 and the attention map attn2 are connected in the channel dimension through feature fusion to obtain the attention map attn3. The average attention (avg_attn) and maximum attention (m_attn) of the attention map attn3 are calculated. ax_attn), connect them in the channel dimension to get the output spliced ​​attention map, convolve the output spliced ​​attention map, and pass the sigmoid activation function to get the channel attention weight sig, the attention map attn1 and the attention map attn2 are multiplied by the channel attention weight sig, and weighted summation is performed in the channel dimension to get the weighted feature map attn, the weighted feature map attn is convolved to restore the original number of channels, the first fused attention feature map is multiplied by the weighted feature map attn to get the output of the LSK attention mechanism, that is, the corresponding bounding box feature map and target category feature map.

[0117] By introducing the LSK attention mechanism in the detection head, the detection accuracy is improved, complex backgrounds can be processed, feature expression is enhanced, and the ability to capture contextual information is improved.

[0118] Finally, the target feature map output by each detection head is passed through the classification prediction layer to perform bounding box regression and target classification to obtain the final target recognition result.

[0119] In this embodiment, the target recognition result is output in the form of a bounding box, each of which contains the target category, confidence score, and precise location and direction information, including the type of aquatic products and the number of aquatic products corresponding to each type.

[0120] By predicting the location and type information of the target through the head network, the target can be detected accurately and efficiently, improving the efficiency and accuracy of detection.

[0121] Step S206, counting the target types and the number of each target type according to the target recognition result.

[0122] In this embodiment, the target identification result is output in the form of a bounding box. By counting the number of bounding boxes and the target type corresponding to each bounding box, the target type and the number of each target type in the sonar image to be detected can be obtained.

[0123] Among them, the target species are aquatic product varieties, which can be major categories, such as fish, shrimp, crabs, etc., or subcategories within major categories, such as fish including but not limited to cod, hairtail, mackerel, pomfret, sardines, etc.

[0124] In this embodiment, after identifying the types and quantities of aquatic products in the area, the momentum change of each aquatic product is calculated through the interval image, and the momentum change of each aquatic product is used as the activity mark of the aquatic product. The size of this value is used as the basis for the damage of the aquatic product, and is used as an important data reference for aquatic insurance loss assessment, which can greatly save the manpower and material resources invested by surveyors.

[0125] This application uses a backbone network to perform multi-scale feature extraction on the frame-sampling image, which can extract richer and more distinctive features from the image, helping to improve the accuracy of target detection; by fusing feature maps of different scales through the neck network, the dependency between features can be enhanced to balance the image semantic information, thereby improving the model's detection capability for targets of different sizes; by predicting targets of different scales through the head network, the comprehensiveness and accuracy of target detection are further improved, and at the same time, the efficiency and accuracy of quantitative statistics can be improved, the operation is simple, convenient and objective, and a lot of manpower and material resources are saved.

[0126] In some optional implementations, before the step of inputting the extracted frame image into the trained target detection model, the following steps are further included:

[0127] A sonar detection image data set is obtained, and frame extraction is performed on the sonar detection image data set to obtain frame extraction sample images;

[0128] Preprocessing the frame sample images to obtain a preprocessed sonar image set;

[0129] The pre-built initial target detection model is trained based on the pre-processed sonar image set to obtain a final target detection model, wherein the initial target detection model also includes an auxiliary branch.

[0130] The sonar detection image data of the insured aquatic product breeding areas in different places are obtained as the sonar detection image data set. After the frame is extracted in intervals, the sample images are preprocessed. The preprocessing includes: denoising, enhancement, image expansion, image annotation and other processing methods. In this embodiment, the sample images are denoised, enhanced, expanded, and labeled in sequence.

[0131] Image augmentation is a data enhancement technology that can generate more training samples by performing image transformations such as rotation, flipping, translation, scaling, and noise injection on images to achieve image augmentation, which helps improve the generalization ability of the model. Rotation is to randomly rotate the image to a certain angle; flipping is to flip the image horizontally or vertically; translation is to apply a random translation transformation on the image; scaling is to change the size of the image; and noise injection is to add random noise to the image.

[0132] The enhancement methods include mosaic data enhancement (mosic), image mixing (mixup), etc. Mosaic data enhancement is an effective data enhancement technology. By randomly cropping, scaling, and splicing images, the diversity and complexity of training data are increased, and the generalization ability of the target detection model and the detection ability of small targets are improved; image mixing (mixup) data enhancement is an effective technology that generates new training samples by linearly mixing images and labels, thereby increasing the diversity of training data and improving the generalization ability, robustness, and smooth decision boundaries of the model. This method has been widely used in tasks such as image classification and target detection, and has significantly improved the performance of the model.

[0133] Image annotation is to use a bounding box to mark the types of aquatic products on the expanded image. Common image annotation tools such as LabelImg annotation tool, ImgLab annotation tool, etc. can be used for annotation.

[0134] The annotated images are grouped into a sonar image set, and the sonar image set is randomly divided into a training set and a validation set according to a preset ratio. For example, the preset ratio is 8:2, that is, the ratio of the training set to the validation set is 8:2. The pre-built initial target detection model is trained with the training set and the validation set. The training is assisted by auxiliary branches during the training process, and finally the trained target detection model is obtained. The structure of the auxiliary branch is shown in Figure 3 shown.

[0135] This application can improve image quality by preprocessing the sample images, while enhancing the diversity of image data and improving the generalization ability of the model; by introducing an efficient multi-scale attention mechanism EMA based on cross-space learning at the end of the backbone network and an LSK attention mechanism in the head network, the model's ability to extract key features of small targets is enhanced, the probability of the model missing small-target aquatic products is reduced, and the bounding box regression is faster and more accurate, thereby improving the accuracy and efficiency of model detection.

[0136] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0137] The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning. A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium may be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0138] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0139] Further references Figure 4 , as a response to the above Figure 2 The present application provides an embodiment of a target recognition device based on sonar images. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0140] like Figure 4As shown, the target recognition device 400 based on sonar images described in this embodiment includes: a frame extraction module 401, an input module 402, a feature extraction module 403, a feature fusion module 404, a detection and recognition module 405 and a statistical module 406. Among them:

[0141] The frame extraction module 401 is used to collect the sonar image to be detected in the target area, and extract frames for the sonar image to be detected at preset time intervals to obtain a frame extracted image;

[0142] The input module 402 is used to input the frame-sampling image into the trained target detection model, wherein the target detection model includes a backbone network, a neck network and a head network;

[0143] The feature extraction module 403 is used to extract features of different scales on the frame-sampling image through the backbone network to obtain a multi-scale feature map, and input the multi-scale feature map into the neck network;

[0144] The feature fusion module 404 is used to perform feature fusion attention enhancement on the multi-scale feature map through the neck network to obtain a fused attention feature map;

[0145] The detection and recognition module 405 is used to perform target detection on the fused attention feature map through the head network and output a target recognition result, wherein the target recognition result includes a target type, a target position, and a target rotation angle;

[0146] The statistics module 406 is used to count the target types and the number corresponding to each target type according to the target recognition result.

[0147] Based on the above-mentioned sonar image-based target recognition device 400, multi-scale feature extraction is performed on the frame-by-frame image through the backbone network, so that richer and more distinctive features can be extracted from the image, which helps to improve the accuracy of target detection; the feature maps of different scales are fused through the neck network, which can enhance the dependency between features to balance the image semantic information, thereby improving the model's detection ability for targets of different sizes; targets of different scales are predicted through the head network, which further improves the comprehensiveness and accuracy of target detection. At the same time, it can improve the efficiency and accuracy of quantitative statistics, and the operation is simple, convenient and objective, saving a lot of manpower and material resources.

[0148] In some optional implementations, the frame extraction module 401 is further used for:

[0149] Extracting the head image, the middle image and the tail image of the sonar image to be detected at preset time intervals to obtain the head frame, the middle frame and the tail frame respectively;

[0150] The head frame, the middle frame and the tail frame are combined to obtain a frame extraction image.

[0151] In some optional implementations, the backbone network includes a first CBS convolution layer, a second CBS convolution layer, a first RepNCSPELAN4 layer, a first downsampling module, a second RepNCSPELAN4 layer, a second downsampling module, a third RepNCSPELAN4 layer, a third downsampling module, a fourth RepNCSPELAN4 layer, an EMA attention layer and a SPPELAN layer, and the feature extraction module 403 is further used to:

[0152] Performing a convolution operation on the frame-sampling image through the first CBS convolution layer to obtain a first-level convolution feature map;

[0153] Performing a convolution operation on the first-level convolutional feature map through the second CBS convolutional layer to obtain a second-level convolutional feature map;

[0154] Performing feature extraction and fusion on the second-level convolutional feature map through the first RepNCSPELAN4 layer to obtain a first convolutional fusion feature map;

[0155] Performing a downsampling operation on the first convolution fusion feature map through the first downsampling module to obtain a first downsampled convolution feature map;

[0156] Performing feature extraction and fusion on the first down-sampling convolution feature map through the second RepNCSPELAN4 layer to obtain a second convolution fusion feature map, and inputting the second convolution fusion feature map into the neck network and the second down-sampling module;

[0157] Performing a downsampling operation on the second convolution fusion feature map through the second downsampling module to obtain a second downsampled convolution feature map;

[0158] Performing feature extraction and fusion on the second down-sampling convolution feature map through the third RepNCSPELAN4 layer to obtain a third convolution fusion feature map, and inputting the third convolution fusion feature map into the neck network and the third down-sampling module;

[0159] Performing a downsampling operation on the third convolution fusion feature map through the third downsampling module to obtain a third downsampling convolution feature map;

[0160] Performing feature extraction and fusion on the third down-sampled convolutional feature map through the fourth RepNCSPELAN4 layer to obtain a fourth convolutional fusion feature map;

[0161] Performing attention calculation on the fourth convolutional fusion feature map through the EMA attention layer to obtain a multi-scale attention feature map;

[0162] The multi-scale attention feature map is subjected to feature extraction through the SPPELAN layer to obtain a multi-scale feature map.

[0163] By introducing the backbone network of the EMA attention mechanism to perform feature extraction on the frame-by-frame images, it is possible to more effectively extract feature information of different scales, enhance the ability to extract key features of small targets, and help improve the model's ability to detect targets of different sizes.

[0164] In some optional implementations, the neck network includes a first neck branch and a second neck branch, the first neck branch includes a first upsampling layer, a first connection layer, a fifth RepNCSPELAN4 layer, a second upsampling layer, a second connection layer, a sixth RepNCSPELAN4 layer, a fourth downsampling module, a third connection layer, a seventh RepNCSPELAN4 layer, a fifth downsampling module, a fourth connection layer and an eighth RepNCSPELAN4 layer, wherein the sixth RepNCSPELAN4 layer, the seventh RepNCSPELAN4 layer and the eighth RepNCSPELAN4 layer are each connected to a hybrid attention layer, which are the first hybrid attention layer, the second hybrid attention layer and the third hybrid attention layer, respectively, and the feature fusion module 404 is further used to:

[0165] Performing an upsampling operation on the multi-scale feature map through the first upsampling layer to obtain a first upsampling feature map;

[0166] Splicing the third convolution fusion feature and the first up-sampled feature map through the first connection layer to obtain a first spliced ​​feature map;

[0167] The first concatenated feature map is fused and enhanced by the fifth RepNCSPELAN4 layer to obtain a first enhanced fused feature map, and the first enhanced fused feature map is input into the second upsampling layer and the third connection layer;

[0168] Performing an upsampling operation on the first enhanced fusion feature map through the second upsampling layer to obtain a second upsampling feature map;

[0169] Splicing the second convolution fusion feature and the second up-sampled feature map through the second connection layer to obtain a second spliced ​​feature map;

[0170] The second concatenated feature map is fused and enhanced by the sixth RepNCSPELAN4 layer to obtain a second enhanced fused feature map, and the second enhanced fused feature map is input into the fourth downsampling module and the first hybrid attention layer;

[0171] Performing attention calculation on the second enhanced fused feature map through the first mixed attention layer, obtaining a first fused attention feature map and inputting it into the head network;

[0172] Downsampling the second enhanced fused feature map by the fourth downsampling module to obtain a first downsampled fused feature map;

[0173] Splicing the first down-sampled fused feature map and the first enhanced fused feature map through the third connection layer to obtain a third spliced ​​feature map;

[0174] The third concatenated feature map is fused and enhanced by the seventh RepNCSPELAN4 layer to obtain a third enhanced fused feature map, and the third enhanced fused feature map is input into the fifth downsampling module and the second mixed attention layer;

[0175] Performing attention calculation on the third enhanced fused feature map through the second mixed attention layer, obtaining a second fused attention feature map and inputting it into the head network;

[0176] Downsampling the third enhanced fused feature map by the fifth downsampling module to obtain a second downsampled fused feature map;

[0177] Splicing the second down-sampled fused feature map and the multi-scale feature map through the fourth connection layer to obtain a fourth spliced ​​feature map;

[0178] The fourth concatenated feature map is fused and enhanced by the eighth RepNCSPELAN4 layer to obtain a fourth enhanced fused feature map;

[0179] The fourth enhanced fused feature map is subjected to attention calculation through the third mixed attention layer to obtain a third fused attention feature map which is input into the head network.

[0180] By extracting more complex features in the image through the neck network and fusing them with the already extracted feature information, the feature representation ability and the richness of semantic information can be increased to improve the accuracy of target detection.

[0181] In some optional implementations, the head network includes a first detection head, a second detection head, a third detection head and a classification prediction layer, and the detection recognition module 405 is further used to:

[0182] By using the first detection head, performing target category prediction and bounding box prediction according to the first fused attention feature map, obtaining a first target feature map;

[0183] By using the second detection head, performing target category prediction and bounding box prediction according to the second fused attention feature map, obtaining a second target feature map;

[0184] By means of the third detection head, target category prediction and bounding box prediction are performed according to the third fused attention feature map to obtain a third target feature map;

[0185] The first target feature map, the second target feature map and the third target feature map are input into the classification prediction layer for classification and regression to obtain a target recognition result.

[0186] By predicting the location and type information of the target through the head network, the target can be detected accurately and efficiently, improving the efficiency and accuracy of detection.

[0187] In this embodiment, the first detection head, the second detection head and the third detection head respectively independently include a bounding box prediction branch and a target category prediction branch, wherein the bounding box prediction branch includes a first convolution block, a first LSK attention layer and a Bbox Loss layer connected end to end in sequence, and the target category prediction branch includes a second convolution block, a second LSK attention layer and a Cls Loss layer connected end to end in sequence, and the detection and recognition module 405 is further used to:

[0188] Inputting the first fused attention feature map into the bounding box prediction branch and the target category prediction branch of the first detection head respectively;

[0189] Predicting the first fused attention feature map through the first convolution block, the first LSK attention layer, and the Bbox Loss layer to obtain a bounding box feature map;

[0190] Predicting the first fused attention feature map through the second convolution block, the second LSK attention layer and the Cls Loss layer to obtain a target category feature map;

[0191] The bounding box feature map and the object category feature map are output as a first object feature map.

[0192] By introducing the LSK attention mechanism in the detection head, the detection accuracy is improved, complex backgrounds can be processed, feature expression is enhanced, and the ability to capture contextual information is improved.

[0193] In some optional implementations, the target recognition device 300 based on sonar images further includes a training module, including:

[0194] A frame extraction submodule is used to obtain a sonar detection image data set, perform interval frame extraction on the sonar detection image data set, and obtain a frame extraction sample image;

[0195] A preprocessing submodule, used for preprocessing the frame sample images to obtain a preprocessed sonar image set;

[0196] The training submodule is used to train the pre-constructed initial target detection model based on the pre-processed sonar image set to obtain a final target detection model, wherein the initial target detection model also includes an auxiliary branch.

[0197] By preprocessing the sample images, the image quality can be improved, the diversity of image data can be enhanced, and the generalization ability of the model can be improved; by introducing an efficient multi-scale attention mechanism EMA based on cross-space learning at the end of the backbone network and an LSK attention mechanism in the head network, the model's ability to extract key features of small targets is enhanced, the probability of the model missing small target aquatic products is reduced, and the bounding box regression is faster and more accurate, thereby improving the accuracy and efficiency of model detection.

[0198] To solve the above technical problems, the present application also provides a computer device. Figure 5 , Figure 5 This is a basic structural block diagram of the computer device in this embodiment.

[0199] The computer device 5 includes a memory 51, a processor 52, and a network interface 53 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 5 with a memory 51, a processor 52, and a network interface 53, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0200] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0201] The memory 51 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 51 can be an internal storage unit of the computer device 5, such as a hard disk or memory of the computer device 5. In other embodiments, the memory 51 can also be an external storage device of the computer device 5, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 5. Of course, the memory 51 can also include both the internal storage unit of the computer device 5 and its external storage device. In this embodiment, the memory 51 is generally used to store the operating system and various application software installed on the computer device 5, such as computer-readable instructions of a target recognition method based on sonar images, etc. In addition, the memory 51 can also be used to temporarily store various types of data that have been output or are to be output.

[0202] The processor 52 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 52 is generally used to control the overall operation of the computer device 5. In this embodiment, the processor 52 is used to run the computer-readable instructions stored in the memory 51 or process data, such as running the computer-readable instructions of the target recognition method based on sonar images.

[0203] The network interface 53 may include a wireless network interface or a wired network interface. The network interface 53 is generally used to establish a communication connection between the computer device 5 and other electronic devices.

[0204] By performing multi-scale feature extraction on the frame-by-frame images through the backbone network, richer and more distinctive features can be extracted from the images, which helps to improve the accuracy of target detection; by fusing feature maps of different scales through the neck network, the dependency between features can be enhanced to balance the image semantic information, thereby improving the model's detection ability for targets of different sizes; by predicting targets of different scales through the head network, the comprehensiveness and accuracy of target detection are further improved. At the same time, the efficiency and accuracy of quantitative statistics can be improved. The operation is simple, convenient and objective, saving a lot of manpower and material resources.

[0205] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the target recognition method based on sonar images as described above.

[0206] By performing multi-scale feature extraction on the frame-by-frame images through the backbone network, richer and more distinctive features can be extracted from the images, which helps to improve the accuracy of target detection; by fusing feature maps of different scales through the neck network, the dependency between features can be enhanced to balance the image semantic information, thereby improving the model's detection ability for targets of different sizes; by predicting targets of different scales through the head network, the comprehensiveness and accuracy of target detection are further improved. At the same time, the efficiency and accuracy of quantitative statistics can be improved. The operation is simple, convenient and objective, saving a lot of manpower and material resources.

[0207] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0208] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A target recognition method based on sonar images, characterized in that: The steps include: Collecting the sonar image to be detected in the target area, and extracting frames of the sonar image to be detected at preset time intervals to obtain an extracted frame image; Inputting the extracted frame image into a trained target detection model, wherein the target detection model includes a backbone network, a neck network, and a head network; Extract features of different scales from the frame-sampling image through the backbone network to obtain a multi-scale feature map, and input the multi-scale feature map into the neck network; Performing feature fusion attention enhancement on the multi-scale feature map through the neck network to obtain a fused attention feature map; Performing target detection on the fused attention feature map through the head network, and outputting a target recognition result, wherein the target recognition result includes a target type, a target position, and a target rotation angle; According to the target recognition result, the target types and the number corresponding to each target type are counted.

2. The target recognition method based on sonar images according to claim 1 is characterized in that: The step of performing interval frame extraction on the sonar image to be detected at a preset time interval to obtain a frame extracted image comprises: Extracting the head image, the middle image and the tail image of the sonar image to be detected at preset time intervals to obtain the head frame, the middle frame and the tail frame respectively; The head frame, the middle frame and the tail frame are combined to obtain a frame extraction image.

3. The target recognition method based on sonar images according to claim 1 is characterized in that: The backbone network includes a first CBS convolution layer, a second CBS convolution layer, a first RepNCSPELAN4 layer, a first downsampling module, a second RepNCSPELAN4 layer, a second downsampling module, a third RepNCSPELAN4 layer, a third downsampling module, a fourth RepNCSPELAN4 layer, an EMA attention layer and an SPPELAN layer. The step of extracting features of different scales from the frame-sampling image through the backbone network to obtain a multi-scale feature map includes: Performing a convolution operation on the frame-sampling image through the first CBS convolution layer to obtain a first-level convolution feature map; Performing a convolution operation on the first-level convolutional feature map through the second CBS convolutional layer to obtain a second-level convolutional feature map; Performing feature extraction and fusion on the second-level convolutional feature map through the first RepNCSPELAN4 layer to obtain a first convolutional fusion feature map; Performing a downsampling operation on the first convolution fusion feature map through the first downsampling module to obtain a first downsampled convolution feature map; Performing feature extraction and fusion on the first down-sampling convolution feature map through the second RepNCSPELAN4 layer to obtain a second convolution fusion feature map, and inputting the second convolution fusion feature map into the neck network and the second down-sampling module; Performing a downsampling operation on the second convolution fusion feature map through the second downsampling module to obtain a second downsampled convolution feature map; Performing feature extraction and fusion on the second down-sampling convolution feature map through the third RepNCSPELAN4 layer to obtain a third convolution fusion feature map, and inputting the third convolution fusion feature map into the neck network and the third down-sampling module; Performing a downsampling operation on the third convolution fusion feature map through the third downsampling module to obtain a third downsampling convolution feature map; Performing feature extraction and fusion on the third down-sampled convolutional feature map through the fourth RepNCSPELAN4 layer to obtain a fourth convolutional fusion feature map; Performing attention calculation on the fourth convolution fusion feature map through the EMA attention layer to obtain a multi-scale attention feature map; The multi-scale attention feature map is subjected to feature extraction through the SPPELAN layer to obtain a multi-scale feature map.

4. The target recognition method based on sonar images according to claim 3 is characterized in that: The neck network includes a first neck branch and a second neck branch, the first neck branch includes a first upsampling layer, a first connection layer, a fifth RepNCSPELAN4 layer, a second upsampling layer, a second connection layer, a sixth RepNCSPELAN4 layer, a fourth downsampling module, a third connection layer, a seventh RepNCSPELAN4 layer, a fifth downsampling module, a fourth connection layer and an eighth RepNCSPELAN4 layer, wherein the sixth RepNCSPELAN4 layer, the seventh RepNCSPELAN4 layer and the eighth RepNCSPELAN4 layer are each connected to a hybrid attention layer, which are the first hybrid attention layer, the second hybrid attention layer and the third hybrid attention layer respectively, and the step of fusing the multi-scale feature map through the neck network to obtain a fused enhanced feature map includes: Performing an upsampling operation on the multi-scale feature map through the first upsampling layer to obtain a first upsampling feature map; Splicing the third convolution fusion feature and the first up-sampled feature map through the first connection layer to obtain a first spliced ​​feature map; The first concatenated feature map is fused and enhanced by the fifth RepNCSPELAN4 layer to obtain a first enhanced fused feature map, and the first enhanced fused feature map is input into the second upsampling layer and the third connection layer; Performing an upsampling operation on the first enhanced fusion feature map through the second upsampling layer to obtain a second upsampling feature map; Splicing the second convolution fusion feature and the second up-sampled feature map through the second connection layer to obtain a second spliced ​​feature map; The second concatenated feature map is fused and enhanced by the sixth RepNCSPELAN4 layer to obtain a second enhanced fused feature map, and the second enhanced fused feature map is input into the fourth downsampling module and the first hybrid attention layer; Performing attention calculation on the second enhanced fused feature map through the first mixed attention layer, obtaining a first fused attention feature map and inputting it into the head network; Downsampling the second enhanced fused feature map by the fourth downsampling module to obtain a first downsampled fused feature map; Splicing the first down-sampled fused feature map and the first enhanced fused feature map through the third connection layer to obtain a third spliced ​​feature map; The third concatenated feature map is fused and enhanced by the seventh RepNCSPELAN4 layer to obtain a third enhanced fused feature map, and the third enhanced fused feature map is input into the fifth downsampling module and the second mixed attention layer; Performing attention calculation on the third enhanced fused feature map through the second mixed attention layer, obtaining a second fused attention feature map and inputting it into the head network; Downsampling the third enhanced fused feature map by the fifth downsampling module to obtain a second downsampled fused feature map; Splicing the second down-sampled fused feature map and the multi-scale feature map through the fourth connection layer to obtain a fourth spliced ​​feature map; The fourth concatenated feature map is fused and enhanced by the eighth RepNCSPELAN4 layer to obtain a fourth enhanced fused feature map; The fourth enhanced fused feature map is subjected to attention calculation through the third mixed attention layer to obtain a third fused attention feature map which is input into the head network.

5. The target recognition method based on sonar images according to claim 4 is characterized in that: The head network includes a first detection head, a second detection head, a third detection head and a classification prediction layer, and the step of performing target detection on the fused attention feature map through the head network and outputting a target recognition result includes: By using the first detection head, performing target category prediction and bounding box prediction according to the first fused attention feature map, obtaining a first target feature map; By using the second detection head, performing target category prediction and bounding box prediction according to the second fused attention feature map, obtaining a second target feature map; By means of the third detection head, target category prediction and bounding box prediction are performed according to the third fused attention feature map to obtain a third target feature map; The first target feature map, the second target feature map and the third target feature map are input into the classification prediction layer for classification and regression to obtain a target recognition result.

6. The target recognition method based on sonar images according to claim 5 is characterized in that: The first detection head, the second detection head, and the third detection head independently include a bounding box prediction branch and a target category prediction branch, respectively, wherein the bounding box prediction branch includes a first convolution block, a first LSK attention layer, and a BboxLoss layer connected end to end in sequence, and the target category prediction branch includes a second convolution block, a second LSK attention layer, and a ClsLoss layer connected end to end in sequence, and the step of performing target category prediction and bounding box prediction according to the first fused attention feature map through the first detection head to obtain the first target feature map includes: Inputting the first fused attention feature map into the bounding box prediction branch and the target category prediction branch of the first detection head respectively; Predicting the first fused attention feature map through the first convolution block, the first LSK attention layer, and the Bbox Loss layer to obtain a bounding box feature map; Predicting the first fused attention feature map through the second convolution block, the second LSK attention layer and the Cls Loss layer to obtain a target category feature map; The bounding box feature map and the object category feature map are output as a first object feature map.

7. The target recognition method based on sonar images according to claim 1 is characterized in that: Before the step of inputting the extracted frame image into the trained target detection model, the method further includes: Acquire a sonar detection image data set, perform interval frame extraction on the sonar detection image data set, and obtain frame extraction sample images; Preprocessing the frame sample images to obtain a preprocessed sonar image set; The pre-constructed initial target detection model is trained based on the pre-processed sonar image set to obtain a final target detection model, wherein the initial target detection model also includes an auxiliary branch.

8. A target recognition device based on sonar images, characterized in that: include: A frame extraction module is used to collect the sonar image to be detected in the target area, and extract frames of the sonar image to be detected at intervals according to a preset time interval to obtain a frame extracted image; An input module, used to input the frame-sampling image into a trained target detection model, wherein the target detection model includes a backbone network, a neck network and a head network; A feature extraction module, used to extract features of different scales on the frame-sampling image through the backbone network to obtain a multi-scale feature map, and input the multi-scale feature map into the neck network; A feature fusion module, used for performing feature fusion attention enhancement on the multi-scale feature map through the neck network to obtain a fused attention feature map; A detection and recognition module, configured to perform target detection on the fused attention feature map through the head network and output a target recognition result, wherein the target recognition result includes a target type, a target position, and a target rotation angle; The statistical module is used to count the target types and the number corresponding to each target type according to the target recognition result.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the target recognition method based on sonar images as described in any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the target recognition method based on sonar images as described in any one of claims 1 to 7 are implemented.

Images