Sonar few-sample target detection method based on prototype distillation and scale perception

Abstract

The invention relates to the technical field of sonar detection, in particular to a sonar few-sample target detection method based on prototype distillation and scale perception, and the method comprises the steps: obtaining a sonar image data set, and constructing a query set and a support set from the sonar image data set; performing feature extraction on the query set and the support set to respectively obtain a query feature map and a support feature map; inputting the support feature map into an integrated fine-grained prototype distillation module, and extracting to obtain an enhanced fine-grained category prototype; inputting the enhanced fine-grained category prototype and the query feature map into a scale perception prototype distribution module to obtain an enhanced query feature map; performing feature extraction on the support feature map and the enhanced query feature map to respectively obtain a class prototype and RoI features; enabling the class prototype and the RoI features to pass through a self-adaptive weighted fusion module to obtain a fused feature map; and inputting the fused feature map into a detection head to obtain a target detection result. According to the invention, the discrimination of target features can be improved and noise interference can be effectively suppressed.

Description

Technical Field

[0001] The present invention relates to the field of sonar detection technology, and in particular to a sonar few-sample target detection method based on prototype distillation and scale perception. Background Art

[0002] Sonar imaging is a key technology that uses the principles of sound wave propagation to detect and image underwater environments. It is widely used in fields such as marine exploration, military reconnaissance, and underwater archaeology. Compared to optical images, sonar images have low resolution, high noise, and blurred object edges. This poses significant challenges to traditional object detection algorithms, especially when sample availability is scarce. Few-shot learning aims to learn models with generalization capabilities from a limited number of labeled examples to address the difficulty of acquiring such examples. By introducing strategies such as metric learning, meta-learning, and data augmentation, it improves the adaptability of models in data-scarce situations and has become a research hotspot in computer vision. Few-shot object detection is an extension of the few-shot learning approach to object detection tasks. Its goal is to accurately identify the location and category of objects in an image with only a few examples of the target category. Few-shot object detection technology is particularly important for sonar imaging scenarios, as underwater target samples are expensive to acquire and difficult to label. Detection models with few-shot learning capabilities are urgently needed to improve detection performance and practical application value.

[0003] In existing technologies, sonar images are characterized by low resolution, high noise, and blurred target edges, which pose difficulties for traditional target detection algorithms, especially when samples are scarce. There are few labeled samples, and underwater target samples are expensive to obtain and difficult to label, making traditional methods difficult to effectively detect with a small number of samples. Existing methods lack precise prototype representation, and feature representation stability is insufficient in complex noisy environments. Underwater targets vary in scale, and traditional methods lack the ability to match and express features at multiple scales. When RoI features are aggregated with prototypes, the importance of operations is not distinguished, making it difficult to highlight key features and susceptible to noise interference. The ability to distinguish target features needs to be improved.

[0004] Based on the above problems, the present invention proposes a sonar few-shot target detection method based on prototype distillation and scale perception. Summary of the Invention

[0005] To address the aforementioned technical issues of sonar imagery's low resolution, high noise, scarcity of annotated samples, variable object scales, imprecise prototype representation, and inadequate feature aggregation strategies, a sonar few-shot object detection method based on prototype distillation and scale awareness is provided. This method primarily integrates a fine-grained prototype distillation module (which uses a multi-head sparse attention mechanism to focus different feature queries on independent local regions of supported features and fuses global residual information), a scale-aware prototype allocation module (which employs a multi-head inflated attention mechanism to achieve multi-scale matching of fine-grained prototypes with query feature maps), and an adaptive weighted fusion module (which dynamically aggregates RoI features and prototypes using adaptive weights). This method improves feature representation stability in complex noisy environments, enhances multi-scale object detection capabilities, highlights key features, and suppresses noise interference, significantly improving the accuracy and robustness of object detection in sonar few-shot scenarios.

[0006] The technical means adopted in the present invention are as follows: The sonar few-shot target detection method based on prototype distillation and scale perception includes the following steps: A sonar image dataset is obtained, and a query set and a support set are constructed from the sonar image dataset; the query set and the support set are subjected to feature extraction using the first, second, and third stages of a 101-layer residual network to obtain a query feature map and a support feature map, respectively; the support feature map is input into an integrated fine-grained prototype distillation module to extract an enhanced fine-grained category prototype; the enhanced fine-grained category prototype and the query feature map are input into a scale-aware prototype assignment module to implement multi-scale prototype assignment between the fine-grained category prototype and the query feature map, thereby obtaining an enhanced query feature map; the support feature map and the enhanced query feature map are subjected to feature extraction using the fourth stage of a 101-layer residual network to obtain class prototypes and region of interest features, respectively; the class prototype and region of interest features are fused through an adaptive weighted fusion module to obtain a fused feature map; the fused feature map is input into a detection head to obtain a target detection result.

[0007] Furthermore, the integrated fine-grained prototype distillation module includes a global support feature processing branch and a fine-grained local support feature processing branch. The workflow of the integrated fine-grained prototype distillation module is as follows: The support feature map is input into the global support feature processing branch to obtain the global support feature; the support feature map and the feature query vector are input into the fine-grained local support feature processing branch, and the fine-grained local semantic fusion representation is obtained by calculating the similarity matrix. The calculation formula of the similarity matrix is:

[0008] in, is the similarity matrix, is the feature query vector, W is the linear transformation matrix, and c' is The channel feature dimension, To support the feature map, Softmax is the normalized activation function; the global support features and fine-grained local semantics are fused through residual connections to obtain an enhanced fine-grained category prototype. The calculation formula of the enhanced fine-grained category prototype is:

[0009] in, P For enhanced fine-grained category prototypes, A i is the similarity matrix, To support feature maps, concat is a concatenation operation, and α is a learning parameter that balances local and global information. It is a global support feature.

[0010] Furthermore, the workflow of the global support feature processing branch is as follows: The support feature map is used as input, and the support feature map is subjected to global average pooling to obtain the global support feature.

[0011] Furthermore, the workflow of the fine-grained local support feature processing branch is as follows: Taking the support feature map as input, the support feature map is divided into blocks, and then the divided support feature map is sampled to obtain local support feature blocks; for each local support feature block and the feature query vector corresponding to the local support feature block, a scaled dot product attention mechanism is introduced, and combined with the similarity matrix to obtain each local fine-grained prototype; each local fine-grained prototype is spliced ​​to obtain a fine-grained local semantic fusion representation.

[0012] Furthermore, the workflow of the scale-aware prototype allocation module is as follows: The query feature map is subjected to dilated convolution feature extraction and then linear projection processing to obtain the scale query feature of each head:

[0013] in, For the i The scale query features output by each head, is the linear projection matrix, For the i The dilated convolution of the head, The query feature map is generated; the enhanced fine-grained category prototype and background information are spliced ​​to obtain the spliced ​​category prototype; the scale query feature of each head is matched with the spliced ​​category prototype, and then the affinity matrix is ​​generated through Softmax normalization:

[0014] in, is the affinity matrix, For the i The scale query features output by each head, is the prototype of the spliced ​​class, is the linear projection matrix, Softmax is the activation function, and c'' is The channel feature dimension of the concatenated class prototype and affinity matrix is ​​used to perform weighted fusion. The weighted fusion feature is added to the query feature map through residual connection to obtain an enhanced query feature map:

[0015] in, For the enhanced query feature graph, To query the feature map, is the scaling parameter, is the number of heads, is the affinity matrix, is the class prototype after splicing, is the learnable fusion weight associated with the i-th head.

[0016] Furthermore, the workflow of the adaptive weighted fusion module is as follows: The RoI features and the class prototype are subjected to nonlinear transformation, wherein the nonlinear transformation includes element-by-element multiplication, subtraction, and concatenation to obtain transformed features; the transformed features are weightedly fused with the RoI features according to learnable weights to obtain a fused feature map:

[0017] in, is the fused feature map, are the first learnable weight, the second learnable weight, the third learnable weight and the fourth learnable weight, They are the first fully connected layer, the second fully connected layer, the third fully connected layer and the fourth fully connected layer, is the feature after transformation, is the class-level prototype obtained after the fourth stage of the 101-layer residual network. It is the element-wise product operation.

[0018] Furthermore, the first fully connected layer, the second fully connected layer, the third fully connected layer and the fourth fully connected layer all have a ReLU activation function.

[0019] Compared with the prior art, the present invention has the following advantages: To address the challenges of sonar target detection, such as limited labeled samples, strong image noise, low contrast, and large object scale variation, a new meta-learning-based sonar few-shot target detection model is proposed. Furthermore, to address the common problem of inaccurate class prototype representation in existing methods, a fine-grained class prototype distillation module is designed to enable different feature queries to focus on local regions supporting independent features. Global residual information is introduced to balance local detail with global context, resulting in more stable feature representation in complex noisy environments. Furthermore, to address the scale variability of underwater targets, a scale-aware prototype assignment module is proposed. This module utilizes a multi-headed inflated attention mechanism to perform multi-scale matching between fine-grained class prototypes and query feature maps to enhance the expressive power of query features. Finally, to address the problem that traditional methods fail to distinguish the importance of each operation when aggregating RoI features with class prototypes, making it difficult to highlight key features, an adaptive weighted fusion module is proposed. This module dynamically aggregates RoI features and class prototypes through an adaptive weighted fusion mechanism, thereby improving the discriminability of target features and effectively suppressing noise interference.

[0020] Based on the above reasons, the present invention can be widely promoted in fields such as sonar detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] Figure 1 It is a schematic diagram of the overall framework of the model involved in the present invention.

[0023] Figure 2 Schematic diagram of the fine-grained prototype distillation module of the present invention.

[0024] Figure 3 Schematic diagram of the scale-aware prototype allocation module of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] Existing sonar imaging utilizes the reflection properties of sound waves propagating through water to reconstruct underwater scenes in two or three dimensions. Its core technologies include various signal processing methods, such as pulse compression, beamforming, and time-of-flight ranging (TDF), to obtain target echo information and generate images. Sonar images have low resolution and are susceptible to multipath interference and scattering noise. Traditional convolutional neural networks often struggle to suppress noise and preserve target edge information during feature extraction, resulting in insufficient detection accuracy and robustness. Few-shot learning, through techniques such as metric learning, meta-learning, and data augmentation, learns feature representations with good generalization capabilities even with only a few labeled examples, thereby alleviating the overfitting problem caused by sample scarcity. Few-shot object detection further incorporates these concepts into the detection framework, typically employing class prototype construction, feature interaction modules, and contextual information fusion strategies to achieve accurate classification and regression of candidate boxes when only a few positive examples are available. In view of the characteristics of sonar scenes such as large target scale variations, complex noise environment and high annotation cost, current methods mostly introduce multi-scale feature pyramids, attention mechanisms and noise suppression modules, but there is still room for improvement in prototype expression accuracy, multi-scale matching strategies and noise robustness.

[0028] like Figure 1As shown in the figure, we first use the first three stages (Res1, Res2, and Res3) of the ResNet-101 backbone network to extract mid-level features from the query image and support images, respectively. The extracted support feature maps are then fed into the Integrated Fine-Grained Prototype Distillation (IFPD) module to extract fine-grained class prototypes. The Scale-Aware Prototype Assignment (SAPA) module then implements multi-scale prototype assignment between the fine-grained class prototypes and the query feature maps. Next, we use the fourth stage (Res4) of the ResNet-101 backbone network to extract high-level features from both branches, further obtaining RoI features and class prototypes. Finally, the high-level features are fused using the newly proposed Adaptive Weighted Fusion (AWF) module and fed into the detection head to generate the final prediction results.

[0029] like Figure 1-3 As shown, the present invention provides a sonar few-shot target detection method based on prototype distillation and scale perception, the method is specifically as follows: S1. Obtain a sonar image dataset and construct a query set and a support set from the sonar image dataset; S2. Use the first, second, and third stages of the 101-layer residual network (ResNet-101) to extract features from the query set and support set, and obtain the query feature map and support feature map respectively.

[0030] Specifically, the sonar image sample set is divided into base classes and new classes, and a meta-learning-based two-branch twin network architecture is constructed to perform feature extraction on the query image and support image respectively to obtain the corresponding query feature map and support feature map.

[0031] S3. Input the support feature map into the integrated fine-grained prototype distillation module to extract the enhanced fine-grained category prototype.

[0032] Specifically, such as Figure 2 As shown in Figure 2, the integrated fine-grained prototype distillation module includes a parallel global support feature processing branch and a fine-grained local support feature processing branch. Given a support feature map F s ∈R hw×c and a set of learnable embeddings q∈R n×c’ As a feature query, hw represents the height and width of the support feature map, c and c' are feature dimensions, c is the channel feature dimension of the support feature map and the query feature map, and c' is the feature query vector q i and multi-scale query features Q i The channel feature dimension of n is the number of feature queries.

[0033] The overall workflow of integrating the fine-grained prototype distillation module is: S31. Input the support feature map into the global support feature processing branch to obtain the global support feature.

[0034] Specifically, the workflow of the global support feature processing branch is to take the support feature map as input, perform global average pooling on the support feature map, and obtain the global support feature.

[0035] S32. Input the support feature map and the feature query vector into the fine-grained local support feature processing branch, and obtain the fine-grained local semantic fusion representation by calculating the similarity matrix. The calculation formula of the similarity matrix is:

[0036] in, is the similarity matrix, is the feature query vector, W is the linear transformation matrix, and c' is The channel feature dimension, To support feature maps, Softmax is the normalized activation function.

[0037] Specifically, the workflow of the fine-grained local support feature processing branch is as follows: S321. Taking the support feature map as input, performing a block operation on the support feature map, and then performing a sampling operation on the block-based support feature map to obtain a local support feature block.

[0038] S322. For each local support feature block and the feature query vector corresponding to the local support feature block, a scaled dot product attention mechanism is introduced and combined with the similarity matrix to obtain each local fine-grained prototype.

[0039] S323. Perform a splicing operation on each local fine-grained prototype to obtain a fine-grained local semantic fusion representation.

[0040] In the fine-grained local support feature processing branch, such as Figure 2 As shown, we will F s The spatial dimension is divided into an equal number of regions for feature query. For each region, we start sampling from its starting position to obtain the sampling features within the region to reduce redundant noise information and focus on the most informative key points.

[0041] S33. To fully integrate local information with global information, we introduce a learnable residual term to fuse global support features and fine-grained local semantics through residual connections to obtain enhanced fine-grained category prototypes. The calculation formula of the enhanced fine-grained category prototype is:

[0042] in, PFor enhanced fine-grained category prototypes, A i is the similarity matrix, To support feature maps, concat is a concatenation operation, and α is a learning parameter that balances local and global information. It is a global support feature.

[0043] S4. Input the enhanced fine-grained category prototype and query feature map into the scale-aware prototype assignment module to achieve multi-scale prototype assignment between the fine-grained category prototype and the query feature map, and obtain the enhanced query feature map.

[0044] Specifically, such as Figure 3 As shown in Figure 2, the workflow of the scale-aware prototype allocation module is as follows: S41. Given a query feature graph F q ∈R hw×c First, a set of dilated convolution kernels with different dilation rates are used to extract local context information. Each dilated convolution kernel corresponds to a specific receptive field and can cover feature spaces of different scales. Then, linear projection processing is performed to obtain the scale query features of each head:

[0045] in, For the i The scale query features output by each head, is the linear projection matrix, For the i The dilated convolution of the head, is the query feature map. Different dilation kernels correspond to different dilation rates, allowing each head to perceive query semantic features of different scales and generate multi-scale query feature representations.

[0046] S42. Concatenate the enhanced fine-grained category prototype and background information to obtain a concatenated category prototype.

[0047] S43. Match the scale query features of each head with the spliced ​​class prototypes, and then generate the affinity matrix through Softmax normalization:

[0048] in, is the affinity matrix, For the i The scale query features output by each head, is the class prototype after splicing, is the linear projection matrix, Softmax is the normalized activation function, and c'' is The channel feature dimension of .

[0049] S44. Based on the concatenated class prototypes and affinity matrix, a method of introducing learnable fusion weights is used for weighted fusion. The weighted fusion features are added to the query feature map through residual connections to achieve prototype assignment and obtain an enhanced query feature map:

[0050] in, For the enhanced query feature graph, To query the feature map, is a scaling parameter, which is used to adjust the impact of enhanced features on the final output. is the number of heads, is the affinity matrix, is the class prototype after splicing, is the learnable fusion weight associated with the i-th head. , the contribution of each scale attention branch to the output result can be dynamically adjusted, thereby improving the accuracy and robustness of prototype allocation.

[0051] S5. Use the fourth stage of the 101-layer residual network (ResNet-101) to extract features from the support feature map and the enhanced query feature map to obtain class prototype and region of interest (RoI) features, respectively.

[0052] S6. The class prototype and the region of interest features are fused through an adaptive weighted fusion module to obtain a fused feature map.

[0053] Specifically, the workflow of the adaptive weighted fusion module is as follows: S61. Perform nonlinear transformation on the RoI features and class prototypes, where the nonlinear transformation includes element-wise multiplication, subtraction, and concatenation to obtain the transformed features.

[0054] S62. The converted features are weightedly fused with the RoI features according to the learnable weights to achieve dynamic regulation of the contribution of each branch and obtain the fused feature map:

[0055] in, is the fused feature map, are the first learnable weight, the second learnable weight, the third learnable weight and the fourth learnable weight, They are the first fully connected layer, the second fully connected layer, the third fully connected layer and the fourth fully connected layer, is the feature after transformation, is the class-level prototype obtained after the fourth stage of the 101-layer residual network. It is an element-wise product operation. Through dynamically adjustable weighting coefficients , the network can adaptively balance the relative contributions of RoI features and class prototypes, and finally generate more discriminative fusion features to input the detection head.

[0056] As a preferred embodiment of the present invention, the first fully connected layer, the second fully connected layer, the third fully connected layer and the fourth fully connected layer all have a ReLU activation function.

[0057] S7. Input the fused feature map into the detection head to obtain the target detection result.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A sonar few-shot target detection method based on prototype distillation and scale perception, characterized by: The following steps are involved: Obtain a sonar image dataset and construct a query set and support set from the sonar image dataset; Performing feature extraction on the query set and the support set using the first, second, and third stages of a 101-layer residual network to obtain a query feature map and a support feature map, respectively; Inputting the support feature map into the integrated fine-grained prototype distillation module to extract enhanced fine-grained category prototypes; Inputting the enhanced fine-grained category prototype and the query feature map into a scale-aware prototype allocation module to implement multi-scale prototype allocation between the fine-grained category prototype and the query feature map, thereby obtaining an enhanced query feature map; Using the fourth stage of the 101-layer residual network to extract features from the support feature map and the enhanced query feature map, respectively obtaining class prototypes and region of interest features; The class prototype and the region of interest features are fused through an adaptive weighted fusion module to obtain a fused feature map; The fused feature map is input into the detection head to obtain the target detection result.

2. The sonar few-shot target detection method based on prototype distillation and scale perception according to claim 1, characterized in that: The integrated fine-grained prototype distillation module includes a global support feature processing branch and a fine-grained local support feature processing branch. The workflow of the integrated fine-grained prototype distillation module is as follows: Inputting the support feature map into a global support feature processing branch to obtain a global support feature; The support feature map and feature query vector are input into the fine-grained local support feature processing branch, and a fine-grained local semantic fusion representation is obtained by calculating the similarity matrix. The calculation formula of the similarity matrix is: in, is the similarity matrix, is the feature query vector, W is the linear transformation matrix, and c' is The channel feature dimension, To support feature maps, Softmax is the normalized activation function; The global support features and fine-grained local semantics are fused through residual connections to obtain an enhanced fine-grained category prototype. The calculation formula of the enhanced fine-grained category prototype is: in, P For enhanced fine-grained category prototypes, A i is the similarity matrix, To support feature maps, concat is a concatenation operation, and α is a learning parameter that balances local and global information. It is a global support feature.

3. The sonar few-shot target detection method based on prototype distillation and scale perception according to claim 2, characterized in that: The workflow of the global support feature processing branch is as follows: The support feature map is used as input, and the support feature map is subjected to global average pooling to obtain the global support feature.

4. The sonar few-shot target detection method based on prototype distillation and scale perception according to claim 2, characterized in that: The workflow of the fine-grained local support feature processing branch is as follows: Taking the support feature map as input, performing a block operation on the support feature map, and then performing a sampling operation on the block-divided support feature map to obtain a local support feature block; For each local support feature block and the feature query vector corresponding to the local support feature block, a scaled dot product attention mechanism is introduced and combined with the similarity matrix to obtain each local fine-grained prototype; Each local fine-grained prototype is spliced ​​together to obtain a fine-grained local semantic fusion representation.

5. The sonar few-shot target detection method based on prototype distillation and scale perception according to claim 1, characterized in that: The workflow of the scale-aware prototype allocation module is as follows: The query feature map is subjected to dilated convolution feature extraction and then linear projection processing to obtain the scale query feature of each head: in, For the i The scale query features output by each head, is the linear projection matrix, For the i The dilated convolution of the head, is the query feature map; The enhanced fine-grained category prototype and background information are spliced ​​together to obtain the spliced ​​category prototype; The scale query features of each head are matched with the spliced ​​class prototypes, and then the affinity matrix is ​​generated through Softmax normalization: in, is the affinity matrix, For the i The scale query features output by each head, is the prototype of the spliced ​​class, is the linear projection matrix, Softmax is the activation function, and c'' is The channel feature dimension of Based on the concatenated class prototype and affinity matrix, a method of introducing learnable fusion weights is used for weighted fusion. The weighted fusion features are added to the query feature map through residual connection to obtain an enhanced query feature map: in, For the enhanced query feature graph, To query the feature map, is the scaling parameter, is the number of heads, is the affinity matrix, is the prototype of the spliced ​​class, is the learnable fusion weight associated with the i-th head.

6. The sonar few-shot target detection method based on prototype distillation and scale perception according to claim 1, characterized in that: The workflow of the adaptive weighted fusion module is as follows: Performing a nonlinear transformation on the RoI feature and the class prototype, wherein the nonlinear transformation includes element-by-element multiplication, subtraction, and concatenation to obtain a transformed feature; The converted features are weightedly fused with the RoI features according to the learnable weights to obtain the fused feature map: in, is the fused feature map, are the first learnable weight, the second learnable weight, the third learnable weight and the fourth learnable weight, They are the first fully connected layer, the second fully connected layer, the third fully connected layer and the fourth fully connected layer, is the feature after transformation, is the class-level prototype obtained after the fourth stage of the 101-layer residual network. It is the element-wise product operation.

7. The sonar few-shot target detection method based on prototype distillation and scale perception according to claim 6, characterized in that: The first fully connected layer, the second fully connected layer, the third fully connected layer and the fourth fully connected layer all have a ReLU activation function.

Images