Forward-looking sonar image segmentation method and device based on meta-transfer learning

Abstract

A forward-looking sonar image segmentation method and apparatus based on meta-transfer learning is disclosed. The method includes: S1, constructing a meta-transfer-based sonar image segmentation model, wherein the feature extractor in the model is a convolutional neural network, and an attention mechanism is employed in the convolutional neural network; S2, pre-training the network model parameters using a pre-training dataset to obtain initialization parameters; S3, training the network model using the pre-trained model with a meta-learning strategy, and using an adaptive factor for model convergence during training; S4, inputting the image to be segmented into the trained network model to obtain the segmentation result. The attention mechanism of this invention can merge feature maps of different dimensions, improving the segmentation accuracy of the model; the meta-learning strategy can support small-sample sonar image segmentation, quickly adapt to and learn small-sample segmentation tasks, and maintain high segmentation accuracy, solving the problem of difficulty in obtaining small sonar image samples and the problem of poor segmentation accuracy on small-sample sonar image datasets.

Description

Technical Field

[0001] This invention belongs to the field of image processing and underwater detection, and relates to a forward-looking sonar image segmentation method and apparatus based on meta-transfer learning. Background Technology

[0002] Sonar is an acoustic imaging technology used for seabed surveys and topographic exploration. Because light waves are easily absorbed by seawater and have difficulty propagating in it, while sound waves attenuate slowly in water and travel long distances, sonar equipment is primarily used for underwater target imaging. It utilizes the principle of sound waves propagating in water and interacting with the seabed or underwater objects, generating images of the seabed topography or underwater objects by receiving the returned echo signals. However, due to changes in attitude and velocity caused by seawater, the data acquired by sonar often does not accurately reflect the true seabed topography echo intensity. This data distortion after analysis severely affects the subsequent processing of sonar images.

[0003] Image segmentation plays a crucial role in sonar image analysis. It effectively distinguishes target objects from the background in sonar images and accurately determines their location and shape. This provides the necessary foundation for subsequent tasks such as target recognition, classification, shape analysis, and motion analysis. However, the acquisition of sonar image data is relatively complex and limited by environmental conditions, making data acquisition often challenging. Furthermore, the acquisition and sharing of sonar image data are restricted because it may contain sensitive information. This data scarcity problem raises another widespread challenge: few-shot learning. Faced with limited sonar image data, it is necessary to learn an effective model from only a small subset of labeled samples to adapt to new tasks and samples.

[0004] Meta-learning is a machine learning method designed to enable machine learning algorithms to learn how to learn. Its goal is to allow learning algorithms to adapt to new tasks more efficiently and flexibly by learning from experience and knowledge gained from multiple tasks. In meta-learning, the task is treated as the input to the learning algorithm, not just training data. By learning on multiple tasks, meta-learning algorithms can learn generalized patterns, policies, and parameter initializations to quickly adapt and learn on new tasks. Summary of the Invention

[0005] The purpose of this invention is to reduce the impact of small sample size due to difficulties in sonar image acquisition on neural network segmentation of sonar images, and to provide a forward-looking sonar image segmentation method and apparatus based on meta-transfer learning.

[0006] This invention can automatically learn the optimal parameter configuration and model selection strategy on a given sonar task and dataset, thereby improving the performance of sonar image data segmentation.

[0007] To solve the above-mentioned technical problems, the technical solution provided by this invention is: a forward-looking sonar image segmentation method based on meta-transfer learning, the specific process of which is as follows:

[0008] S1. Construct a sonar image segmentation model based on meta-transfer learning. The feature extractor in the model is a convolutional neural network. The model employs an attention mechanism in the upsampling and downsampling of the convolutional neural network. The attention mechanism is set as follows:

[0009] S11. In downsampling, serialization generates feature map information in both channel and spatial dimensions. Then, the two feature maps are adaptively corrected with the input feature map to generate the final feature map.

[0010] S12, For the input feature map F∈I C×H×W The feature map F′ is obtained by inputting it into the channel attention mechanism module, and then the feature map F″ is obtained by inputting it into the spatial attention mechanism module to complete the downsampling operation. The calculation formula is as follows:

[0011]

[0012] Where M channel and M space These represent channel attention and spatial attention, respectively.

[0013] S13. The feature map generated by the channel is F′ obtained from the input feature map F:

[0014] M channel (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))) (2)

[0015] Where σ represents the Sigmon function, AvgPool represents average pooling, MaxPool represents max pooling, and MLP represents a multilayer perceptron.

[0016] S14. The spatially generated feature map is obtained from the input feature map F′ to obtain F″:

[0017] M space (F′)=σConvNd((cat(AvgPool(F′);MaxPool(F′)))) (3)

[0018] Where σ represents the Sigmon function, ConvNd represents the convolutional layer, cat represents the concatenation function, AvgPool represents average pooling, and MaxPool represents max pooling.

[0019] S15. In upsampling, the output is obtained by combining the upper layer x and the corresponding downsampled x′. The calculation formula is as follows:

[0020]

[0021] in denoted as Hadamard product, σ as Sigmon function, δ as convolution operation, and R as ReLU function.

[0022] S2. Pre-train the network model parameters to obtain the initial segmentation model.

[0023] S21. Ensure segmentation performance through large-scale pre-training. Training is performed on a large-scale dataset, and the calculation formula is as follows:

[0024]

[0025] Where Θ is the feature extractor and θ is the classifier. Let α be the loss function and α be the optimization learning rate.

[0026] S3. Train the network model using a meta-learning strategy. During training, small sample sonar images are used as the dataset. An adaptive factor module is employed to facilitate model convergence. The meta-learning strategy and adaptive factor are set as follows:

[0027] S31. Meta-learning is divided into two stages: the outer loop and the inner loop.

[0028] S32, The outer loop phase sets the learnable parameter ΦS. 1，2 A baseline classifier θ′:

[0029]

[0030] Where θ′ is the baseline classifier, θ is the currently trained classifier, and β represents the learning rate of the outer loop.

[0031] S33、ΦS 1，2 This represents the network model parameters in the second stage. The baseline classifier θ′ is optimized using the training task loss from the dataset, and the baseline classifier θ′ updates the network model parameters. and adaptive factor μ {1，2} :

[0032]

[0033]

[0034] in Here, γ is the learnable parameter in the convolution or bias, and γ is the inner loop learning rate. This represents the standard cross-entropy loss calculated using classifier θ′ on the test task. This represents the Focal Loss function.

[0035] S34, Utilizing in the inner loop The convolution kernel and bias are then adjusted by performing multiplication and addition operations with the adaptive factor, and the original convolution kernel is multiplied by ΦS1, while the original bias is added to ΦS2.

[0036]

[0037] Where X is the input feature map, W is the convolution kernel, and b is the bias.

[0038] S35. Obtain the final classifier θ:

[0039]

[0040] S4. Input the image to be segmented into the trained network model to obtain the segmentation result.

[0041] A second aspect of the present invention relates to a forward-looking sonar image segmentation apparatus based on meta-transfer learning, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors, when executing the executable code, implement the forward-looking sonar image segmentation method based on meta-transfer learning of the present invention.

[0042] A third aspect of the present invention relates to a computer-readable storage medium, characterized in that it stores a program thereon, which, when executed by a processor, implements the forward-looking sonar image segmentation method based on meta-transfer learning of the present invention.

[0043] The working principle of this invention is:

[0044] In the pre-training phase, the model is not considered in the target domain dataset; it is only pre-trained on readily available data to ensure good feature extraction capabilities. In the meta-training phase, considering that existing segmentation models lack accurate segmentation performance for sonar images, the feature extractor's parameters are adjusted through scaling and shifting to enhance its feature extraction capabilities for the target domain, thus enabling rapid adaptation to sonar image segmentation tasks. The meta-testing phase evaluates the model's performance on few-shot learning problems, verifying its ability to quickly adapt to new tasks and make accurate predictions.

[0045] The innovation of this invention is:

[0046] We propose a meta-learning method for segmenting forward-looking sonar images; and we propose an adaptive factor method for training network models for segmenting small-sample sonar images.

[0047] The present invention has the following beneficial effects:

[0048] 1. To address the problem of limited sonar image samples, this invention proposes a forward-looking sonar image segmentation method based on meta-transfer learning. The method adaptively learns network parameters according to different tasks and objectives, resulting in a more reasonable learning strategy. By allowing the network to learn on its own, the performance of image segmentation is significantly improved under the premise of small sample datasets, thus solving the problem of difficult sonar image acquisition.

[0049] 2. To address the issue of low accuracy in sonar image recognition due to the unique characteristics of sonar images, this invention proposes a forward-looking sonar image segmentation method based on meta-transfer learning. The attention mechanism, by stitching together the output feature maps of multiple convolutional layers, enables the model to focus on the location of the target, reducing the impact of large-scale background on target segmentation and solving the problem of low accuracy in sonar image segmentation caused by noise and other issues. Attached Figure Description

[0050] Figure 1 This is a flowchart of a forward-looking sonar image segmentation method based on meta-transfer learning.

[0051] Figure 2 This is a flowchart of the network parameter learning process based on adaptive factors. Detailed Implementation

[0052] To better understand the above technical solution, the technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0053] Example 1

[0054] like Figure 1 As shown, a forward-looking sonar image segmentation method based on meta-transfer learning is implemented by the following steps:

[0055] S1. Construct a sonar image segmentation model based on meta-transfer learning. The feature extractor in the model is a convolutional neural network. The model employs an attention mechanism in the upsampling and downsampling of the convolutional neural network. The attention mechanism is set as follows:

[0056] S11. In downsampling, attention feature map information can be generated sequentially in both channel and spatial dimensions. Then, the two feature maps are multiplied with the input feature map to adaptively correct the features and generate the final feature map.

[0057] S12, For the input feature map F∈I C×H×W The feature map is obtained by inputting it into the channel attention mechanism module, and then the feature map is obtained by inputting it into the spatial attention mechanism module to complete the downsampling operation. The calculation formula is as follows:

[0058]

[0059] Where Mchannel M is the feature map generated for the channel. space Feature maps generated for space.

[0060] S13. The feature map generated by the channel is obtained from the input feature map through average pooling and max pooling:

[0061] M channel (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))) (2)

[0062] Where σ represents the Sigmon function, AvgPool represents average pooling, MaxPool represents max pooling, and MLP represents a multilayer perceptron.

[0063] S14. The spatially generated feature map is obtained from the input feature map through average pooling and max pooling:

[0064] M space (F′)=σConvNd((cat(AvgPool(F′);MaxPool(F′)))) (3)

[0065] Where σ represents the Sigmon function, ConvNd represents the convolutional layer, cat represents the concatenation function, AvgPool represents average pooling, and MaxPool represents max pooling.

[0066] S15. In upsampling, the output x and x' are connected to the output x and x' of the previous layer of the model via a jump connection in the i-th layer of the network structure. i Get the output x u :

[0067]

[0068] Where σ represents the Sigmon function, δ represents the convolution operation, and R represents the ReLU function. It represents the Hadamardi (or Hadama) stack.

[0069] S2. Pre-train the network model parameters using a pre-training dataset to obtain initialization parameters. The pre-training settings are as follows:

[0070] S21. Ensure segmentation performance through large-scale pre-training, and train on a large-scale dataset:

[0071]

[0072] Where Θ is the feature extractor and θ is the classifier. Let α be the loss function and α be the optimization learning rate.

[0073] S22, The loss function is the cross-entropy loss function:

[0074]

[0075] Where x represents a given image, y represents the label of the corresponding image, and f [Θ；θ] Let be the segmentation function, and let L be the segmentation result of the input image x. Let L be the cross-entropy loss calculation function.

[0076] S3. Train the network model using a meta-learning strategy. During training, small sample sonar images are used as the dataset. An adaptive factor is used for model convergence. The meta-learning strategy and adaptive factor are set as follows:

[0077] S31. Meta-learning is divided into two stages: the outer loop fine-tunes the classifier, and the inner loop learns the shifting and scaling of hyperparameters.

[0078] S32, The outer loop phase sets the learnable parameter ΦS. 1，2 A basic classifier θ′:

[0079]

[0080] Where θ′ is the base classifier, θ is the currently trained classifier, and β represents the learning rate of the outer loop. The expression is as follows:

[0081]

[0082] like Figure 2 As shown, the implementation process of a learning method based on adaptive scaling specifically includes the following steps, where Standard CE Loss represents the standard cross-entropy loss function, and Focal CE Loss represents the composite cross-entropy loss function:

[0083] S33、ΦS 1，2 This represents the hyperparameters of the inner loop, which optimize the current classifier θ′ using the loss from the training task, and update the hyperparameters using the base classifier θ′. and adaptive factor μ {1，2} :

[0084]

[0085]

[0086] in Here, γ is the learnable parameter in the convolution or bias, and γ is the inner loop learning rate. This represents the standard cross-entropy loss calculated using classifier θ′ on the test task. The Focal Loss function is represented as follows:

[0087]

[0088] Where N is the number of samples, p is the probability that the model predicts accurately, and γ > 0 is an adjustable factor.

[0089] S34, Utilizing in the inner loop The convolution kernel and bias are then adjusted by performing multiplication and addition operations with the adaptive factor, and the original convolution kernel is multiplied by ΦS1, while the original bias is added to ΦS2.

[0090]

[0091] Where X is the input feature map, W is the convolution kernel, and b is the bias.

[0092] S35. Obtain the final classifier θ:

[0093]

[0094] S4. Input the image to be segmented into the trained network model to obtain the segmentation result.

[0095] The attention mechanism of this invention can merge feature maps of different dimensions to improve the segmentation accuracy of the model; the meta-learning strategy can support small sample sonar image segmentation, can quickly adapt to and learn small sample segmentation tasks, and maintain high segmentation accuracy, solving the problem of small sonar image samples being difficult to obtain and the problem of poor segmentation accuracy of traditional neural networks on small sample sonar images as datasets.

[0096] Example 2

[0097] This embodiment relates to a forward-looking sonar image segmentation device based on meta-transfer learning, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the forward-looking sonar image segmentation method based on meta-transfer learning in Embodiment 1.

[0098] Example 3

[0099] This embodiment relates to a computer-readable storage medium storing a program that, when executed by a processor, implements the forward-looking sonar image segmentation method based on meta-transfer learning of the present invention.

[0100] Finally, it should be noted that the specific solutions and embodiments described above in conjunction with the figures are only a preferred embodiment of the present invention. Those skilled in the art can make some changes and modifications to the described technical content without departing from the technical scope of the present invention. However, any modifications and variations made without departing from the spirit and principle of the present invention should fall within the scope of the technical solution of the present invention.

Claims

1. A forward-looking sonar image segmentation method based on meta-transfer learning, characterized in that: Includes the following steps: S1. Construct a sonar image segmentation model based on meta-transfer learning. The feature extractor in the sonar image segmentation model is a convolutional neural network. The sonar image segmentation model employs an attention mechanism in the upsampling and downsampling of the convolutional neural network. The attention mechanism is set as follows: S11. In downsampling, serialization generates feature map information in both channel and spatial dimensions. Then, the two feature maps are adaptively corrected with the input feature map to generate the final feature map. S12. For the input feature map F, obtain the feature map F from its input channel module. ′ Then, input it into the spatial module to obtain the feature map F″ to complete the downsampling operation; S13. The feature map generated by the channel is obtained from the input feature map F. ′ ; S14. The spatially generated feature map is derived from the input feature map F. ′ We get F″; S15, In upsampling, x is obtained from the upper layer x and the corresponding downsampling x. ′ Get the output; S2. Pre-train the network model parameters to obtain an initialized segmentation model. The pre-training settings are as follows: S21. Pre-training with optical datasets ensures segmentation performance; S22, The loss function is the cross-entropy loss function; S3. Train the network model using a meta-learning strategy. During training, small sample sonar images are used as the dataset. An adaptive factor module is employed to facilitate model convergence. The meta-learning strategy and adaptive factor are set as follows: S31. Meta-learning is divided into two stages. The first stage performs gradient descent on the classifier, and the second stage learns the network model movement and scaling. S32, The outer loop phase sets the learnable parameter ΦS. 1,2 A baseline classifier θ ′ ; S33、ΦS 1,2 This represents the network model parameters in the second stage, used to optimize the baseline classifier θ using the training task loss from the dataset. ′ baseline classifier θ ′ Update network model parameters and adaptive factor μ {1,2} ; S34, Utilizing in the inner loop With μ i A weighted operation is performed, and the result is weighted with the convolution kernel and bias in the network model. S35. Obtain the final classifier θ; S4. Input the image to be segmented into the trained network model to obtain the segmentation result.

2. The forward-looking sonar image segmentation method based on meta-transfer learning as described in claim 1, characterized in that: The initial network model trained using the meta-learning strategy in step S3 is obtained through pre-training in step S2.

3. The forward-looking sonar image segmentation method based on meta-transfer learning as described in claim 1, characterized in that: Step S33's meta-learning has two stages, characterized by: the first stage performing gradient descent to converge the network model, and the second stage updating the network model parameters. and adaptive factor μ {1,2} convergence.

4. The forward-looking sonar image segmentation method based on meta-transfer learning as described in claim 1, characterized in that: In step S33, for hyperparameters Update its value using cross-entropy loss, for the adaptive factor μ. {1,2} Update its value using Focal loss.

5. The forward-looking sonar image segmentation method based on meta-transfer learning as described in claim 1, characterized in that: In step S4, the requirement for obtaining the segmentation result is that the meta-trained model is tested on new, unseen tasks with only a small number of samples available, and the model makes predictions using previously learned meta-learning strategies and general features.

6. A forward-looking sonar image segmentation device based on meta-transfer learning, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the forward-looking sonar image segmentation method based on meta-transfer learning as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the forward-looking sonar image segmentation method based on meta-transfer learning as described in any one of claims 1-5.

Images