A SAR ship classification method based on contrastive learning pre-training

Abstract

The present invention discloses a SAR ship classification method based on contrastive learning pre-training, comprising: constructing a feature fusion network, the feature fusion network comprising a feature extraction module, a first feature fusion module, and a second feature fusion module connected in sequence; constructing a SimCLR contrastive learning network framework with the feature fusion network as the feature extraction network; obtaining multiple sets of mat format training data sets and multiple sets of JPG format image training data sets, and inputting the multiple sets of JPG image training data sets into the SimCLR network framework for training to obtain multiple sets of pre-trained models; loading the parameters of the multiple sets of pre-trained models into the feature fusion network, and further training using the multiple sets of mat format training data sets to obtain a trained feature fusion network; and inputting the original SAR images to be classified into the trained feature fusion network to obtain a classification result. The present invention uses unsupervised contrastive learning to obtain the pre-trained models and combines them with the feature fusion network to improve ship classification performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of image target classification, and in particular relates to a SAR ship classification method based on contrastive learning pre-training. Background Art

[0002] Since the advent of SAR (Synthetic Aperture Radar) in the 1950s, SAR image target classification technology has developed rapidly. In machine learning-based ship target classification in SAR images, Ji et al. extracted the statistical characteristics of ship RCS (Radar Cross Section), including positional characteristic parameters (mean, variance, and range), distribution characteristic parameters (skewness coefficient and kurtosis coefficient), distribution analysis (probability density function and cumulative distribution function), and percentage probability distribution (10%, 50%, and 90% probability values). They then used a BP neural network to classify ship images. In deep learning-based SAR image ship target classification, convolutional neural networks are mainly used to extract SAR image features for classification. Convolutional neural networks began with the LeNet-5 network in 1998, which realized the function of handwritten digit recognition. It emerged with the AlexNet network proposed by Alex in the ImageNet image classification competition in 2012. Later, new classic networks (such as VGG, GoogLeNet and ResNet) have been applied to SAR image ship target classification.

[0003] Patent application number CN201911238758.8, titled "SAR Ship Target Recognition Method Based on Deep Dense Connections and Metric Learning," proposes a SAR ship target recognition method based on a deep dense connection network and metric learning. This method, based on metric learning, uses a triplet network to extract deep features of similar and heterogeneous samples. By optimizing a loss function, it aims to bring similar samples closer together and dissimilar samples further apart in feature space. The feature extraction network in the triplet network is a modified DenseNet, and the loss function is a joint loss function that combines a weighted cross-entropy loss function, a triplet loss function, and a Fisher discriminant regularization term. The improvements to the DenseNet feature extraction network and the loss function in this invention patent have improved classification results to some extent. However, due to the large number of network parameters and the time-consuming training, this method also uses only deep features, which are relatively abstract and ignore features such as texture information in the image. Therefore, there is still room for improvement.

[0004] Current methods for classifying ship targets in SAR images primarily rely on deep features extracted from convolutional neural networks (CNNs), typically trained from scratch. Deep features contain stronger semantic information, but their resolution is low, resulting in poor detail perception. Shallow features, on the other hand, have higher resolution and include more positional and detailed information. Using only deep features for classification fails to fully utilize shallow features. Summary of the Invention

[0005] To address the above-mentioned problems in the prior art, the present invention provides a SAR ship classification method based on contrastive learning pre-training. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] The present invention provides a SAR ship classification method based on contrastive learning pre-training, comprising:

[0007] S1: Constructing a feature fusion network, the feature fusion network comprising a feature extraction module, a first feature fusion module, and a second feature fusion module connected in sequence, wherein the feature extraction module is used to perform preliminary feature extraction on the input SAR image to obtain shallow features and deep features of the SAR image; the first feature fusion module is used to realize fusion from the deep features to the shallow features to obtain fused shallow features; the second feature fusion module is used to realize continued fusion from the fused shallow features to the deep features and output image category probability;

[0008] S2: constructing a SimCLR contrastive learning network framework using the feature fusion network as a feature extraction network, wherein the SimCLR contrastive learning network framework includes a cascaded feature extraction network and a feature mapping network;

[0009] S3: Obtain multiple sets of mat format training data sets and multiple sets of JPG format image training data sets, and input the multiple sets of JPG image training data sets into the SimCLR contrastive learning network framework for training, to obtain multiple sets of pre-trained models;

[0010] S4: loading the parameters of the multiple sets of pre-trained models into the feature fusion network, and further training the feature fusion network using the multiple sets of mat format training data sets to obtain a trained feature fusion network;

[0011] S5: Input the original SAR image to be classified into the trained feature fusion network to obtain the classification result.

[0012] In one embodiment of the present invention, the feature extraction module includes a first convolution layer CO1, a first batch of normalization layers B1, a first activation function layer R1, a second convolution layer CO2, a second batch of normalization layers B2, a second activation function layer R2, a first dense connection and transition module M1, a second dense connection and transition module M2, a third dense connection and transition module M3, and a fourth dense connection and transition module M4, which are connected in sequence.

[0013] In one embodiment of the present invention, the first dense connection and transition module M1, the second dense connection and transition module M2, the third dense connection and transition module M3, and the fourth dense connection and transition module M4 all have the same structure, and all include cascaded dense connection units and transition units, wherein:

[0014] The densely connected unit includes four subunits connected in series, each subunit includes a batch normalization layer, an activation function layer, a convolution layer, and a splicing layer, and the input and output of the previous subunit are spliced ​​as the input of the next subunit, and the input and output of the last subunit are spliced ​​as the output of the densely connected unit;

[0015] The transition unit includes a batch normalization layer, an activation function layer, a convolution layer and a pooling layer connected in sequence.

[0016] In one embodiment of the present invention, the first feature fusion module includes a first deconvolution layer T1, a first splicing layer CA1, a third convolution layer CO3, a second deconvolution layer T2, a second splicing layer CA2, a fourth convolution layer CO4, a third deconvolution layer T3, a third splicing layer CA3, a fifth convolution layer CO5, a fourth deconvolution layer T4, a fourth splicing layer CA4 and a sixth convolution layer CO6, wherein,

[0017] The input of the first deconvolution layer T1 is connected to the output of the fourth dense connection and transition module M4, the two inputs of the first splicing layer CA1 are respectively connected to the output of the third dense connection and transition module M3 and the output of the first deconvolution layer T1, and the output of the first splicing layer CA1 is connected to the input of the third convolution layer CO3; the input of the second deconvolution layer T2 is connected to the output of the third convolution layer CO3, the two inputs of the second splicing layer CA2 are respectively connected to the output of the second dense connection and transition module M2 and the output of the second deconvolution layer T2, and the output of the second splicing layer CA2 is connected to the fourth convolution layer CO 4; the input of the third deconvolution layer T3 is connected to the output of the fourth convolution layer CO4, the two inputs of the third splicing layer CA3 are respectively connected to the output of the first dense connection and transition module M1 and the output of the third deconvolution layer T3, and the output of the third splicing layer CA3 is connected to the input of the fifth convolution layer CO5; the input of the fourth deconvolution layer T4 is connected to the output of the fifth convolution layer CO5, the two inputs of the fourth splicing layer CA4 are respectively connected to the output of the second activation function layer R2 and the output of the fourth deconvolution layer T4, and the output of the fourth splicing layer CA4 is connected to the input of the sixth convolution layer CO6.

[0018] In one embodiment of the present invention, the second feature fusion module includes a seventh convolutional layer CO7, a third batch normalization layer B3, a third activation function layer R3, a first pooling layer P1, a fifth splicing layer CA5, an eighth convolutional layer CO8, a fourth batch normalization layer B4, a fourth activation function layer R4, a second pooling layer P2, a sixth splicing layer CA6, a ninth convolutional layer CO9, a fifth batch normalization layer B5, a fifth activation function layer R5, a third pooling layer P3, a seventh splicing layer CA7, a tenth ... 10 , the sixth batch normalization layer B6, the sixth activation function layer R6, the fourth pooling layer P4, the eighth splicing layer CA8, the eleventh convolution layer CO 11 , seventh batch normalization layer B7, seventh activation function layer R7, fifth pooling layer P5, eighth batch normalization layer B8, eighth activation function layer R8, sixth pooling layer P6, flattening layer FL, first fully connected layer FC1 and second fully connected layer FC2, where,

[0019] The seventh convolutional layer CO7, the third batch normalization layer B3, the third activation function layer R3 and the first pooling layer P1 are cascaded in sequence, and the input of the seventh convolutional layer CO7 is connected to the output of the sixth convolutional layer CO6, and the output of the first pooling layer P1 and the output of the fifth convolutional layer CO5 are connected to the input of the fifth splicing layer CA5; the eighth convolutional layer CO8, the fourth batch normalization layer B4, the fourth activation function layer R4 and the second pooling layer P2 are cascaded in sequence, and the input of the eighth convolutional layer CO8 is connected to the output of the fifth splicing layer CA5, and the output of the second pooling layer P2 and the output of the fourth convolutional layer CO4 are connected to the input of the sixth splicing layer CA6;

[0020] The ninth convolutional layer CO9, the fifth batch normalization layer B5, the fifth activation function layer R5 and the third pooling layer P3 are cascaded in sequence, and the input of the ninth convolutional layer CO9 is connected to the output of the sixth splicing layer CA6, the output of the third pooling layer P3 and the output of the third convolutional layer CO3 are connected to the input of the seventh splicing layer CA7; the tenth convolutional layer CO 10 , the sixth batch normalization layer B6, the sixth activation function layer R6 and the fourth pooling layer P4 are cascaded in sequence, and the tenth convolutional layer CO 10 The input of the eleventh convolutional layer CO is connected to the output of the seventh concatenation layer CA7, the output of the fourth pooling layer P4 and the output of the fourth dense connection and transition module M4 are connected to the input of the eighth concatenation layer CA8; the eleventh convolutional layer CO 11 , the seventh batch normalization layer B7, the seventh activation function layer R7 and the fifth pooling layer P5 are connected in sequence, and the eleventh convolutional layer CO 11 The input is connected to the output of the eighth concatenation layer CA8, and the output of the fifth pooling layer P5 is sequentially connected to the eighth batch normalization layer B8, the eighth activation function layer R8, the sixth pooling layer P6, the flattening layer FL, the first fully connected layer FC1 and the second fully connected layer FC2.

[0021] In one embodiment of the present invention, the feature mapping network is a multi-layer perceptron, comprising a third fully connected layer FC3, a ninth activation function layer R9 and a fourth fully connected layer FC4 cascaded in sequence, and the input of the third fully connected layer FC3 is connected to the output of the second fully connected layer FC2.

[0022] In one embodiment of the present invention, the S3 includes:

[0023] S3.1: The OpenSARShip dataset is divided into five training sets and test sets in a ratio of 8:2 to obtain five training sets {φ1, φ2, φ3, φ4, φ5} and five corresponding test sets {t1, t2, t3, t4, t5}. The training set {φ1, φ2, φ3, φ4, φ5} is expanded and cropped to obtain the training set {Φ1, Φ2, Φ3, Φ4, Φ5}. The test set {t1, t2, t3, t4, t5} is cropped to obtain the test set {T1, T2, T3, T4, T5}. The mat format training set {Φ1, Φ2, Φ3, Φ4, Φ5} is converted to the JPG format image training set {Φ1', Φ2', Φ3', Φ4', Φ5'}.

[0024] S3.2: Input the JPG image dataset {Φ1', Φ2', Φ3', Φ4', Φ5'} into the SimCLR contrastive learning framework for pre-training to obtain a pre-trained model {ψ'1, ψ'2, ψ'3, ψ'4, ψ'5}. The loss function used is contrastive loss:

[0025]

[0026]

[0027]

[0028] Among them, z i It represents the feature vector obtained by feature extraction and feature mapping of the i-th training data in the training data set, z j It represents the feature vector obtained by feature extraction and feature mapping of the j-th training data in the training data set, s i,j represents the similarity between the feature vectors of the i-th training data and the j-th training data, l(i,j) represents the similarity of the feature vectors after the function transformation, τ represents the temperature coefficient, and L represents the average similarity of all sample pairs obtained by data augmentation for each sample in a data set containing N samples. During the calculation process, for the k-th sample in the N-sample data, two data augmentation methods, random cropping and scaling and Gaussian blurring, are used to obtain the 2k-1 and 2k-th samples in the 2N samples after data augmentation.

[0029] In one embodiment of the present invention, the S4 includes:

[0030] The parameters of each pre-trained model {ψ'1, ψ'2, ψ'3, ψ'4, ψ'5} are loaded into the feature fusion network ψ, and the mat format data set {Φ1, Φ2, Φ3, Φ4, Φ5} corresponding to the JPG format image data set {Φ1', Φ2', Φ3', Φ4', Φ5'} is input into the feature fusion network ψ for fine-tuning training to obtain the trained feature fusion network.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. The SAR ship classification method based on contrastive learning pre-training of the present invention uses unsupervised contrastive learning to obtain a pre-training model. By using a simple SimCLR framework to implement unsupervised contrastive learning, the network can obtain better feature extraction capabilities without label information; the method of loading the pre-training model for fine-tuning can obtain better network parameters than direct training, the network convergence speed becomes faster, and the classification accuracy becomes higher.

[0033] 2. The present invention uses a convolutional neural network with a bidirectional feature fusion structure combined with a path aggregation network (PANet), which combines shallow features with deep features, thereby improving the image feature extraction capability and classification accuracy of the convolutional neural network.

[0034] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of a SAR ship classification method based on contrastive learning pre-training provided by an embodiment of the present invention;

[0036] Figure 2 This is a network framework diagram of a feature fusion network provided by an embodiment of the present invention;

[0037] Figure 3 is a structural diagram of a densely connected unit provided by an embodiment of the present invention;

[0038] Figure 4 This is a schematic structural diagram of a transition unit provided by an embodiment of the present invention;

[0039] Figure 5 Schematic diagram of the structure of a SimCLR contrastive learning network framework provided by an embodiment of the present invention;

[0040] Figure 6 These are three types of SAR ship images used in the embodiments of the present invention;

[0041] Figure 7 Schematic diagram of the structure of a densely connected network based on a triplet network and Fisher's discriminant criterion used in comparative experiments of an embodiment of the present invention;

[0042] Figure 8 It is a learning curve diagram of TriDenseNet and improved network loading pre-training model fine-tuning on three types of SAR ship images according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the SAR ship classification method based on contrastive learning pre-training proposed in accordance with the present invention is described in detail below with reference to the accompanying drawings and specific implementation methods.

[0044] The aforementioned and other technical contents, features, and effects of the present invention are clearly presented in the following detailed description of the specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a deeper and more specific understanding of the technical means and effects adopted by the present invention to achieve the intended purpose can be obtained. However, the accompanying drawings are provided for reference and illustration purposes only and are not intended to limit the technical solutions of the present invention.

[0045] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations are intended to cover non-exclusive inclusions, such that an article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the article or device comprising the element.

[0046] See Figure 1 , Figure 1 1 is a flow chart of a SAR ship classification method based on contrastive learning pre-training provided by an embodiment of the present invention. The SAR ship classification method includes the following steps:

[0047] S1: Construct a feature fusion network ψ that combines PANet and DenseNet. The feature fusion network ψ includes a feature extraction module, a first feature fusion module, and a second feature fusion module that are connected in sequence. The feature extraction module is used to perform preliminary feature extraction on the input SAR image to obtain shallow features and deep features of the SAR image; the first feature fusion module is used to realize the fusion from the deep features to the shallow features to obtain the fused shallow features; the second feature fusion module is used to realize the continued fusion from the fused shallow features to the deep features and output the image category probability.

[0048] See Figure 2 , Figure 2 : This is a network framework structure diagram of a feature fusion network provided by an embodiment of the present invention. The feature extraction module of this embodiment includes two convolutional layers, two batch normalization layers, two activation function layers, and four dense connection modules. Specifically, the feature extraction module includes a first convolutional layer CO1, a first batch normalization layer B1, a first activation function layer R1, a second convolutional layer CO2, a second batch normalization layer B2, a second activation function layer R2, a first dense connection and transition module M1, a second dense connection and transition module M2, a third dense connection and transition module M3, and a fourth dense connection and transition module M4, which are connected in sequence.

[0049] Furthermore, the structures of the first dense connection and transition module M1, the second dense connection and transition module M2, the third dense connection and transition module M3 and the fourth dense connection and transition module M4 are all the same, and all include cascaded dense connection units and transition module units, see Figure 3 and Figure 4 , Figure 3 is a structural diagram of a densely connected unit provided by an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a transition unit provided by an embodiment of the present invention. The densely connected unit includes four subunits connected in series, each subunit includes a batch normalization layer, an activation function layer, a convolution layer, and a splicing layer, and the input and output of the previous subunit are spliced ​​together as the input of the next subunit, and the input and output of the last subunit are spliced ​​together as the output of the densely connected unit; the transition unit includes a batch normalization layer, an activation function layer, a convolution layer, and a pooling window connected in sequence.

[0050] In this embodiment, the input of the feature extraction module is That is, the SAR image to be classified, the input, output and parameter settings and relationships of each layer are as follows:

[0051] The first convolutional layer CO1, whose convolution kernel K 1 The window size is 3×3 and the sliding step size is S1 1, used to output 1 feature map This layer serves as the input of the first batch of normalized layers B1; the first batch of normalized layers B1 is normalized as follows (where x is a batch of feature maps input to the batch normalization layer, μ is the mean of all elements at the corresponding positions of the batch of feature maps, and σ 2 is the variance of all elements at the corresponding positions of the batch of feature maps, ε is a very small number to prevent the denominator from being 0), used to output 32 feature maps This layer serves as the input of the first activation function layer R1; the first activation function layer R1, whose activation function is the ReLU function, is used to output 32 weight feature maps As the input of the second convolutional layer CO2; the second convolutional layer CO2, its convolution kernel K 2 The window size is 3×3 and the sliding step size is S 2 1, used to output 32 feature maps This layer serves as the input of the second batch normalization layer B2; the second batch normalization layer B2 is normalized as follows Used to output 32 feature maps This layer serves as the input of the second activation function layer R2; the second activation function layer R2, whose activation function is the ReLU function, is used to output 32 weight feature maps As the input of the first dense connection and transition module M1.

[0052] The first dense connection and transition module M1 includes a cascade of a dense connection unit and a transition unit, wherein the dense connection unit includes four subunits in series, each subunit includes a batch normalization layer, an activation function layer, a convolution layer with a convolution kernel size of 3×3×6 and a stride of 1, and a splicing layer, and the input and output of the previous subunit are spliced ​​in the splicing layer as the input of the next subunit, and the input and output of the last subunit are spliced ​​as the output of the dense connection unit. The transition unit includes a batch normalization layer, an activation function layer, a convolution layer with a convolution kernel size of 3×3×56 and a stride of 1, and a pooling window connected in sequence. The first dense connection and transition module M1 is used to output 56 feature maps This layer serves as the input to the second densely connected and transition module M2.

[0053] The second dense connection and transition module M2 includes a cascade of a dense connection unit and a transition unit, wherein the dense connection unit includes four subunits in series, each subunit includes a batch normalization layer, an activation function layer, a convolution layer with a convolution kernel size of 3×3×12 and a stride of 1, and a splicing layer in series, and the transition unit includes a batch normalization layer, an activation function layer, a convolution operation with a convolution kernel size of 3×3×104 and a stride of 1, and a pooling window in series. The second dense connection and transition module M2 is used to output 104 feature maps This layer serves as the input to the third densely connected and transition module M3.

[0054] Similarly, the third dense connection and transition module M3 includes a cascade of a dense connection unit and a transition unit, wherein the dense connection unit includes four subunits, each subunit includes a batch normalization layer, an activation function layer, a convolution layer with a convolution kernel size of 3×3×18 and a step size of 1, and a splicing layer in series, and the transition unit includes a batch normalization layer, an activation function layer, a convolution layer with a convolution kernel size of 3×3×176 and a step size of 1, and a pooling window in series. The third dense connection and transition module M3 is used to output 176 feature maps. This layer serves as the input to the fourth densely connected and transition module M4.

[0055] The fourth dense connection and transition module M4 includes a cascade of a dense connection unit and a transition unit, wherein the dense connection unit includes four subunits, each subunit includes a batch normalization layer, an activation function layer, a convolution layer with a convolution kernel size of 3×3×24 and a step size of 1, and a splicing layer, the transition unit includes a batch normalization layer, an activation function layer, a convolution layer with a convolution kernel size of 3×3×272 and a step size of 1, and a pooling window, and the fourth dense connection and transition module M4 is used to output 272 feature maps. This layer serves as the input of the first feature fusion module.

[0056] Continue to see Figure 2, the first feature fusion module of this embodiment includes four deconvolution layers (TransposeConvolution), four splicing layers and four convolution layers. Specifically, the first feature fusion module includes a first deconvolution layer T1, a first splicing layer CA1, a third convolution layer CO3, a second deconvolution layer T2, a second splicing layer CA2, a fourth convolution layer CO4, a third deconvolution layer T3, a third splicing layer CA3, a fifth convolution layer CO5, a fourth deconvolution layer T4, a fourth splicing layer CA4 and a sixth convolution layer CO6, wherein the input of the first deconvolution layer T1 is connected to the output of the fourth dense connection and transition module M4, the two inputs of the first splicing layer CA1 are respectively connected to the output of the third dense connection and transition module M3 and the output of the first deconvolution layer T1, the output of the first splicing layer CA1 is connected to the input of the third convolution layer CO3; the input of the second deconvolution layer T2 is connected to the output of the third convolution layer CO3, the second splicing layer The two inputs of the connection layer CA2 are respectively connected to the output of the second dense connection and transition module M2 and the output of the second deconvolution layer T2, and the output of the second splicing layer CA2 is connected to the input of the fourth convolution layer CO4; the input of the third deconvolution layer T3 is connected to the output of the fourth convolution layer CO4, the two inputs of the third splicing layer CA3 are respectively connected to the output of the first dense connection and transition module M1 and the output of the third deconvolution layer T3, and the output of the third splicing layer CA3 is connected to the input of the fifth convolution layer CO5; the input of the fourth deconvolution layer T4 is connected to the output of the fifth convolution layer CO5, the two inputs of the fourth splicing layer CA4 are respectively connected to the output of the second activation function layer R2 and the output of the fourth deconvolution layer T4, and the output of the fourth splicing layer CA4 is connected to the input of the sixth convolution layer CO6.

[0057] Specifically, the input of the first feature fusion module is That is, the output of the fourth dense connection and transition module M4, j = 1, 2...272, and the input, output and parameter settings and relationships of each layer are as follows:

[0058] The first deconvolution layer T1, whose convolution kernel TK 1 The window size is 4×4 and the sliding step size is TS 1 2, used to output 176 feature maps This layer serves as the input of the first concatenation layer CA1; the first concatenation layer CA1, whose concatenation dimension is the channel dimension, is used to output 352 feature maps This layer serves as the input of the third convolutional layer CO3; the third convolutional layer CO3, its convolution kernel K 3 The window size is 3×3 and the sliding step size is S 3 1, used to output 176 feature maps This layer serves as the input to the second deconvolution layer T2.

[0059] The second deconvolution layer T2, whose convolution kernel TK2 The window size is 4×4 and the sliding step size is TS 2 2, used to output 104 feature maps This layer serves as the input of the second concatenation layer CA2; the second concatenation layer CA2, whose concatenation dimension is the channel dimension, is used to output 208 feature maps This layer serves as the input of the fourth convolutional layer CO4; the fourth convolutional layer CO4, its convolution kernel K 4 The window size is 3×3 and the sliding step size is S 4 1, used to output 104 feature maps This layer serves as the input to the third deconvolution layer T3.

[0060] The third deconvolution layer T3, its convolution kernel TK 3 The window size is 4×4 and the sliding step size is TS 3 2, used to output 56 feature maps This layer serves as the input of the third concatenation layer CA3; the third concatenation layer CA3, whose concatenation dimension is the channel dimension, is used to output 112 feature maps This layer serves as the input of the fifth convolutional layer CO5; the fifth convolutional layer CO5, its convolution kernel K 5 The window size is 3×3 and the sliding step size is S 5 1, used to output 56 feature maps This layer serves as the input to the fourth deconvolution layer T4.

[0061] The fourth deconvolution layer T4, whose convolution kernel TK 4 The window size is 4×4 and the sliding step size is TS 4 2, used to output 32 feature maps This layer serves as the input of the fourth concatenation layer CA4; the fourth concatenation layer CA4 has a concatenation dimension of channel dimension and is used to output 64 feature maps. This layer serves as the input of the sixth convolutional layer CO6; the sixth convolutional layer CO6, its convolution kernel K 6 The window size is 3×3 and the sliding step size is S 6 1, used to output 32 feature maps This layer serves as the input of the second feature fusion module.

[0062] Furthermore, the second feature fusion module of this embodiment includes five convolutional layers, six batch normalization layers, six activation function layers, six pooling layers, four splicing layers, one flattening layer and two fully connected layers. Specifically, the second feature fusion module of this embodiment includes the seventh convolutional layer CO7, the third batch normalization layer B3, the third activation function layer R3, the first pooling layer P1, the fifth splicing layer CA5, the eighth convolutional layer CO8, the fourth batch normalization layer B4, the fourth activation function layer R4, the second pooling layer P2, the sixth splicing layer CA6, the ninth convolutional layer CO9, the fifth batch normalization layer B5, the fifth activation function layer R5, the third pooling layer P3, the seventh splicing layer CA7, the tenth convolutional layer CO 10 , the sixth batch normalization layer B6, the sixth activation function layer R6, the fourth pooling layer P4, the eighth splicing layer CA8, the eleventh convolution layer CO 11 , seventh batch normalization layer B7, seventh activation function layer R7, fifth pooling layer P5, eighth batch normalization layer B8, eighth activation function layer R8, sixth pooling layer P6, flattening layer FL, first fully connected layer FC1 and second fully connected layer FC2.

[0063] Specifically, the seventh convolutional layer CO7, the third batch normalization layer B3, the third activation function layer R3 and the first pooling layer P1 are connected in sequence, and the input of the seventh convolutional layer CO7 is connected to the output of the sixth convolutional layer CO6, and the output of the first pooling layer P1 and the output of the fifth convolutional layer CO5 are connected to the input of the fifth splicing layer CA5; the eighth convolutional layer CO8, the fourth batch normalization layer B4, the fourth activation function layer R4 and the second pooling layer P2 are connected in sequence, the input of the eighth convolutional layer CO8 is connected to the output of the fifth splicing layer CA5, the output of the second pooling layer P2 and the output of the fourth convolutional layer CO4 are connected to the input of the sixth splicing layer CA6; the ninth convolutional layer CO9, the fifth batch normalization layer B5, the fifth activation function layer R5 and the third pooling layer P3 are connected in sequence, the input of the ninth convolutional layer CO9 is connected to the output of the sixth splicing layer CA6, and the output of the third pooling layer P3 and the output of the third convolutional layer CO3 are connected to the input of the seventh splicing layer CA7.

[0064] The tenth convolutional layer CO 10 , the sixth batch normalization layer B6, the sixth activation function layer R6 and the fourth pooling layer P4 are connected in sequence, and the tenth convolutional layer CO 10 The input of the seventh convolution layer CA7 is the output of the fourth pooling layer P4, the output of the fourth dense connection and transition module M4 is connected to the input of the eighth convolution layer CA8; the eleventh convolution layer CO 11 , the seventh batch normalization layer B7, the seventh activation function layer R7 and the fifth pooling layer P5 are connected in sequence, and the eleventh convolution layer CO 11The input is connected to the output of the eighth concatenation layer CA8, and the output of the fifth pooling layer P5 is sequentially connected to the eighth batch normalization layer B8, the eighth activation function layer R8, the sixth pooling layer P6, the flattening layer FL, the first fully connected layer FC1 and the second fully connected layer FC2.

[0065] Specifically, the input of the second feature fusion module is That is, the output of the sixth convolutional layer CO6, j = 1, 2...32, and the input, output and parameter settings and relationships of each layer are as follows:

[0066] The seventh convolutional layer CO7: its convolution kernel K 7 The window size is 3×3 and the sliding step size is S 7 1, used to output 32 feature maps This layer serves as the input of the third batch normalization layer B3; the third batch normalization layer B3: its normalization method is Used to output 32 feature maps This layer serves as the input of the third activation function layer R3; the third activation function layer R3: its activation function is the ReLU function, which is used to output 32 weight feature maps As the input of the first pooling layer P1; the first pooling layer P1: its pooling dimension is the spatial dimension, the pooling window size is 2×2, and the sliding step size PS 1 2, used to output 32 weight feature maps This layer serves as the input of the fifth concatenation layer CA5; the fifth concatenation layer CA5: its concatenation dimension is the channel dimension, which is used to output 88 feature maps This layer serves as the input of the eighth convolutional layer CO8.

[0067] The eighth convolutional layer CO8: its convolution kernel K 8 The window size is 3×3 and the sliding step size is S 8 1, used to output 88 feature maps This layer serves as the input of the fourth batch normalization layer B4; the fourth batch normalization layer B4: its normalization method is Used to output 88 feature maps This layer serves as the input of the fourth activation function layer R4; the fourth activation function layer R4: its activation function is the ReLU function, which is used to output 88 weight feature maps This layer serves as the input of the second pooling layer P2; the second pooling layer P2: its pooling dimension is the spatial dimension, the pooling window size is 2×2, and the sliding step size PS 2 2, used to output 88 weight feature maps This layer serves as the input of the sixth splicing layer CA6; the sixth splicing layer CA6: its splicing dimension is the channel dimension, which is used to output 192 feature maps This layer serves as the input of the ninth convolutional layer CO9.

[0068] The ninth convolutional layer CO9: its convolution kernel K 9 The window size is 3×3 and the sliding step size is S 9 1, used to output 192 feature maps This layer serves as the input of the fifth batch normalization layer B5; the fifth batch normalization layer B5: its normalization method is Used to output 192 feature maps j=1,2…192, this layer serves as the input of the fifth activation function layer R5; the fifth activation function layer R5: its activation function is the ReLU function, which is used to output 192 weight feature maps This layer serves as the input of the third pooling layer P3; the third pooling layer P3: its pooling dimension is the spatial dimension, the pooling window size is 2×2, and the sliding step size PS 3 2, used to output 192 weight feature maps This layer serves as the input of the seventh splicing layer CA7; the seventh splicing layer CA7: its splicing dimension is the channel dimension, which is used to output 368 feature maps This layer serves as the tenth convolutional layer CO 10 input.

[0069] The tenth convolutional layer CO 10 : Its convolution kernel K 10 The window size is 3×3 and the sliding step size is S 10 1, used to output 368 feature maps This layer serves as the input of the sixth batch normalization layer B6; the sixth batch normalization layer B6: its normalization method is Used to output 368 feature maps j=1,2…368, this layer serves as the input of the sixth activation function layer R6; the sixth activation function layer R6: its activation function is the ReLU function, which is used to output 368 weight feature maps This layer serves as the input of the fourth pooling layer P4; the fourth pooling layer P4: its pooling dimension is the spatial dimension, the pooling window size is 2×2, the sliding step size PS4 is 2, and it is used to output 368 weight feature maps This layer serves as the input of the eighth concatenation layer CA8; the eighth concatenation layer CA8: its concatenation dimension is the channel dimension, which is used to output 640 feature maps This layer serves as the eleventh convolutional layer CO 11 input.

[0070] The eleventh convolutional layer CO 11 : Its convolution kernel K 11 The window size is 3×3 and the sliding step size is S 111, used to output 640 feature maps This layer serves as the input of the seventh batch normalization layer B7; the seventh batch normalization layer B7: its normalization method is Used to output 640 feature maps This layer serves as the input of the seventh activation function layer R7; the seventh activation function layer R7: its activation function is the ReLU function, which is used to output 640 weight feature maps This layer serves as the input of the fifth pooling layer P5; the fifth pooling layer P5: its pooling dimension is the spatial dimension, the pooling window size is 2×2, and the sliding step size PS 5 2, used to output 640 weight feature maps This layer serves as the input to the eighth batch normalization layer B8.

[0071] The eighth batch of normalized layers B8: its normalization method is Used to output 640 feature maps This layer serves as the input of the eighth activation function layer R8; the eighth activation function layer R8: its activation function is the ReLU function, which is used to output 640 weight feature maps This layer serves as the input of the sixth pooling layer P6; the sixth pooling layer P6: its pooling dimension is the spatial dimension, the pooling window size is 2×2, and the sliding step size PS 6 2, used to output 640 weight feature maps This layer serves as the input of the flattening layer FL; the flattening layer FL is used to output a 640-dimensional column vector X2 28 , which serves as the input of the first fully connected layer FC1; the first fully connected layer FC1: It has 128 neurons and is used to output a 128-dimensional column vector X2 29 , which serves as the input of the second fully connected layer FC2; the second fully connected layer FC2: It has 3 neurons and is used to output a 3D column vector X2 30 .

[0072] S2: Constructing a SimCLR contrastive learning network framework with the feature fusion network ψ as the feature extraction network, wherein the SimCLR contrastive learning network framework includes a cascaded feature extraction network and a feature mapping network.

[0073] Specifically, see Figure 5 , Figure 5 Schematic diagram of the structure of a SimCLR contrastive learning network framework provided by an embodiment of the present invention. The feature extraction network of this embodiment is the feature fusion network ψ, and the number of output nodes of the last fully connected layer of the feature fusion network ψ is modified to 128.

[0074] The feature mapping network is a multi-layer perceptron E, comprising two fully connected layers and one activation function layer, namely, a third fully connected layer FC3, a ninth activation function layer R9 and a fourth fully connected layer FC4 cascaded in sequence, and the input of the third fully connected layer FC3 is connected to the output of the second fully connected layer FC2.

[0075] Specifically, the input of the multilayer perceptron E is X2 30 , which is the output of the second fully connected layer FC2 of the second feature fusion module. The input, output and parameter settings and relationships of each layer are as follows:

[0076] The third fully connected layer FC3: It has 128 neurons and is used to output a 128-dimensional column vector X2 31 , this layer serves as the input of the ninth activation function layer R9; the ninth activation function layer R9: its activation function is the ReLU function, which is used to output a 128-dimensional column vector X2 32 , which serves as the input of the fourth fully connected layer FC4; the fourth fully connected layer FC4: It has 128 neurons and is used to output a 128-dimensional column vector X2 33 .

[0077] S3: Obtain multiple groups of mat format training data sets and multiple groups of JPG format image training data sets, and input the multiple groups of JPG image training data sets into the SimCLR contrastive learning network framework for training respectively to obtain multiple groups of pre-trained models.

[0078] In this embodiment, step S3 includes:

[0079] S3.1: Obtain training and testing datasets.

[0080] Specifically, the OpenSARShip dataset is divided into training and test sets in a ratio of 8:2 to obtain 5 training sets {φ1,φ2,φ3,φ4,φ5} and corresponding 5 test sets {t1,t2,t3,t4,t5}, ​​and the training set {φ1,φ2,φ3,φ4,φ5} is expanded and cropped to obtain the training set {Φ1,Φ2,Φ3,Φ4,Φ5}, and the test set {t1,t2,t3,t4,t5} is cropped to obtain the test set {T1,T2,T3,T4,T5}, and the mat format training set {Φ1,Φ2,Φ3,Φ4,Φ5} is converted into JPG format image training set {Φ1',Φ2',Φ3',Φ4',Φ5'}.

[0081] Specifically, the steps for dividing the data set and expanding the training set data in this step are as follows:

[0082] (a) For each of the three categories of data in the OpenSARShip dataset, the training data and test data are randomly divided into 8:2 ratios, and then the training data of the three categories of data are combined together as the training set φ for this division i , the test data sets of the three types of data are combined as the test set t for this division i Through 5 divisions, we get 5 training sets {φ1, φ2, φ3, φ4, φ5} and 5 test sets {t1, t2, t3, t4, t5}.

[0083] (b) Perform data expansion on the five training sets {φ1, φ2, φ3, φ4, φ5}. The specific expansion methods are: ① Flip the ship images in the training set, including horizontal and vertical flips; ② Rotate the ship slices 90°, 180°, and 270°; ③ Randomly translate the ship slices left and right by 0 to 5 pixels; ④ Add Gaussian noise to the ship images, where the mean of the Gaussian noise is 0 and the variance is 0.001. After data expansion, the number of training data in each category becomes 8 times the original, that is, the original samples and 7 times the expanded samples. Then the number of training data in the category with the least number is the number of training data in each category after expansion. The training data of the remaining categories are all the original training data plus some randomly selected expanded training data, so as to ensure that the number of training data in each category is the same. Then take the 64 pixels in the center of the ship image.

[0084] The ×64 area is the final mat format training set {Φ1,Φ2,Φ3,Φ4,Φ5}, and the test set {t1,t2,t3,t4,t5}

[0085] The 64×64 area in the center of the ship image is taken as the final test set {T1, T2, T3, T4, T5};

[0086] (2c) For the five training sets {Φ1, Φ2, Φ3, Φ4, Φ5}, each SAR image in each training set is converted into a JPG format image, and five training sets {Φ1', Φ2', Φ3', Φ4', Φ5'} are obtained.

[0087] S3.2: Input the JPG format image dataset {Φ1',Φ2',Φ3',Φ4',Φ5'} into the SimCLR contrastive learning framework Ω for pre-training to obtain the trained pre-trained model {ψ'1,ψ'2,ψ'3,ψ'4,ψ'5}.

[0088] That is to say, by inputting each of the five JPG image datasets into the SimCLR contrastive learning framework Ω, a corresponding pre-trained model can be obtained, and the loss function used is the contrastive loss:

[0089]

[0090]

[0091]

[0092] Among them, z i It represents the feature vector obtained by feature extraction and feature mapping of the i-th training data in the training data set, z j It represents the feature vector obtained by feature extraction and feature mapping of the j-th training data in the training data set, s i,j represents the similarity between the eigenvectors of the i-th training data item and the j-th training data item. l(i,j) represents the similarity of the eigenvectors after functional transformation. τ represents the temperature coefficient, a hyperparameter set to 0.07. L represents the average similarity of all pairs of samples obtained by data augmentation for each sample in a small batch of N samples. k is the variable in the summation formula, representing the k-th data item in the N data items. During the calculation process, for the k-th sample in the N data items, two data augmentation methods, random cropping and scaling and Gaussian blurring, are used to obtain the 2k-1 and 2k-th samples in the 2N data items after data augmentation.

[0093] S4: Loading the parameters of the multiple groups of pre-trained models into the feature fusion network, and further training the feature fusion network using the multiple groups of mat format training data sets to obtain a trained feature fusion network.

[0094] Specifically, the parameters of each pre-trained model {ψ'1, ψ'2, ψ'3, ψ'4, ψ'5} are loaded into the feature fusion network ψ, and the mat format data set {Φ1, Φ2, Φ3, Φ4, Φ5} corresponding to the JPG format image data set {Φ1', Φ2', Φ3', Φ4', Φ5'} is input into the feature fusion network ψ for fine-tuning training to obtain a trained feature fusion network. Exemplarily, the parameters of the pre-trained model ψ'1 are loaded into the feature fusion network ψ, and the mat format data set Φ1 corresponding to the JPG format image data set Φ1' is input into the feature fusion network ψ for fine-tuning training, and so on.

[0095] In this embodiment, 300 rounds of training are performed, and after each round of training, the test set is used for testing. The average accuracy of the last 10 rounds of testing is taken to obtain the final classification result. In this step, the loss function used in training is as follows:

[0096] Joint losses:

[0097] Among them, λ1 is the cross entropy loss function weight set to 0.6, J s (X, W, B) is the cross entropy loss function, X is the feature vector of the network output, W, B are the network parameter weights and bias, 1-λ1 is the triplet loss function weight set to 0.4, J t (X, W, B) is the triplet loss function, λ2 is the Fisher discriminant regularization term with a weight set to 0.005, and J f (X, W, B) is the Fisher discriminant regularization term, λ3 is the L2 regularization term with a weight set to 0.0005, and J w (W,B) is the weight decay regularization term, is the true probability that the i-th sample belongs to the c-th category, is the probability prediction value that the i-th sample belongs to the c-th class, and N is the data size of a batch of data; is the feature vector of the anchor sample in the triplet sample, is the feature vector of the positive sample in the triplet sample, is the feature vector of the negative sample in the triplet sample, Represents the Euclidean distance between the feature vector of the anchor sample and the feature vector of the positive sample, It represents the Euclidean distance between the feature vector of the anchor sample and the feature vector of the negative sample, and α is the Euclidean distance threshold set to 0.2; express When taking it itself, When , it is 0; m1 and m2 are the average values ​​of the Euclidean distances of the same type of sample data pairs and the different types of sample data pairs in the feature space, and are the corresponding variance values; Indicates the square of the F norm applied to the network parameter matrix, Indicates that the square of the 2-norm is applied to the network parameter vector.

[0098] The following comparative experiments illustrate the effect of the SAR ship classification method based on contrastive learning pre-training of the present invention. Figure 7 , Figure 7 This is a schematic diagram of the structure of a densely connected network based on a triplet network and the Fisher discriminant criterion used in comparative experiments in embodiments of the present invention. This densely connected network, referred to as TriDenseNet, is from the paper "Research on Ship Target Detection and Classification Methods in SAR Images," a doctoral dissertation by He Jinglu of Xidian University in 2019. The comparative experiments were trained directly on this densely connected network, TriDenseNet.

[0099] During direct training of TriDenseNet, experiments were conducted using the PyTorch framework, running on Ubuntu 16.04, Torch version 1.6.0, and CUDA version 10.0.130. The learning rate, momentum parameter, cross-entropy loss weight, and batch size were set to 0.1, 0.9, 0.6, and 100, respectively. The learning rate was decayed to 0.1 times the previous value at the 150th, 200th, and 250th epochs of training. The network was trained using only cross-entropy loss for the first 10 epochs. After 10 epochs, the cross-entropy loss function was combined with triplet loss and the Fisher discriminant criterion for 300 epochs.

[0100] In an experiment using the method of the present invention to fine-tune a pre-trained model loaded onto a feature fusion network, comparative learning pre-training was first performed using the PyTorch framework, using the Ubuntu 16.04 operating system, Torch version 1.6.0, and CUDA version 10.0.130. During comparative learning, the learning rate, temperature, and momentum parameters were set to 0.0003, 0.07, and 0.0001, respectively, and the pre-trained model was trained for 1000 epochs. During fine-tuning, the learning rate, momentum parameter, cross-entropy loss function weight, and batch size were set to 0.1, 0.9, 0.6, and 100, respectively, and the learning rate was decayed to 0.1 times the previous value at the 150th, 200th, and 250th training epochs. The network was trained using only cross-entropy loss for the first 10 epochs. After 10 epochs, the cross-entropy loss function was combined with triplet loss and the Fisher discriminant criterion for 300 epochs.

[0101] Table 1. Test accuracy of three types of SAR ship images using TriDenseNet and the method of the present invention

[0102]

[0103] from Figure 8 As shown in Table 1, both direct training and fine-tuning of the pre-trained model using the improved network achieved convergence on every dataset after 300 training rounds. Both training and test curves converged after 200 rounds for both methods. Table 1 shows that compared to TriDenseNet, the improved fine-tuning method using the pre-trained model achieved 0.64 percentage points higher accuracy and a 0.29 percentage point lower standard deviation, demonstrating higher classification accuracy and lower standard deviation. This demonstrates that incorporating a feature fusion module into a convolutional neural network and fine-tuning the pre-trained model using contrastive learning can improve classification accuracy for ship targets in SAR images.

[0104] The present invention's SAR ship classification method based on contrastive learning pre-training uses unsupervised contrastive learning to obtain a pre-trained model. Unsupervised contrastive learning is implemented using a simple SimCLR framework, enabling the network to achieve better feature extraction capabilities without the need for label information. Compared to direct training, the method of loading and fine-tuning the pre-trained model can obtain better network parameters, resulting in faster network convergence and higher classification accuracy. The present invention uses a convolutional neural network with a bidirectional feature fusion structure combined with a path aggregation network, combining shallow and deep features to improve the image feature extraction capability and classification accuracy of the convolutional neural network.

[0105] In the several embodiments provided herein, it should be understood that the apparatus and method disclosed herein can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the module division is merely a logical functional division. In actual implementation, other division methods may be used. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not implemented.

[0106] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing module, each module may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or hardware plus software functional modules.

[0107] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A SAR ship classification method based on contrastive learning pre-training, characterized by: include: S1: Constructing a feature fusion network, the feature fusion network comprising a feature extraction module, a first feature fusion module, and a second feature fusion module connected in sequence, wherein the feature extraction module is used to perform preliminary feature extraction on the input SAR image to obtain shallow features and deep features of the SAR image; the first feature fusion module is used to realize fusion from the deep features to the shallow features to obtain fused shallow features; the second feature fusion module is used to realize continued fusion from the fused shallow features to the deep features and output image category probability; S2: constructing a SimCLR contrastive learning network framework using the feature fusion network as a feature extraction network, wherein the SimCLR contrastive learning network framework includes a cascaded feature extraction network and a feature mapping network; S3: Obtain multiple sets of mat format training data sets and multiple sets of JPG format image training data sets, and input the multiple sets of JPG image training data sets into the SimCLR contrastive learning network framework for training, to obtain multiple sets of pre-trained models; S4: loading the parameters of the multiple sets of pre-trained models into the feature fusion network, and further training the feature fusion network using the multiple sets of mat format training data sets to obtain a trained feature fusion network; S5: Input the original SAR image to be classified into the trained feature fusion network to obtain the classification result.

2. The SAR ship classification method based on contrastive learning pre-training according to claim 1 is characterized in that: The feature extraction module includes a first convolution layer CO1, a first batch of normalization layers B1, a first activation function layer R1, a second convolution layer CO2, a second batch of normalization layers B2, a second activation function layer R2, a first dense connection and transition module M1, a second dense connection and transition module M2, a third dense connection and transition module M3, and a fourth dense connection and transition module M4, which are connected in sequence.

3. The SAR ship classification method based on contrastive learning pre-training according to claim 2 is characterized in that: The first dense connection and transition module M1, the second dense connection and transition module M2, the third dense connection and transition module M3 and the fourth dense connection and transition module M4 all have the same structure, and all include cascaded dense connection units and transition units, wherein: The densely connected unit includes four subunits connected in series, each subunit includes a batch normalization layer, an activation function layer, a convolution layer, and a splicing layer, and the input and output of the previous subunit are spliced ​​as the input of the next subunit, and the input and output of the last subunit are spliced ​​as the output of the densely connected unit; The transition unit includes a batch normalization layer, an activation function layer, a convolution layer and a pooling layer connected in sequence.

4. The SAR ship classification method based on contrastive learning pre-training according to claim 2 is characterized in that: The first feature fusion module includes a first deconvolution layer T1, a first splicing layer CA1, a third convolution layer CO3, a second deconvolution layer T2, a second splicing layer CA2, a fourth convolution layer CO4, a third deconvolution layer T3, a third splicing layer CA3, a fifth convolution layer CO5, a fourth deconvolution layer T4, a fourth splicing layer CA4 and a sixth convolution layer CO6, wherein: The input of the first deconvolution layer T1 is connected to the output of the fourth dense connection and transition module M4, the two inputs of the first splicing layer CA1 are respectively connected to the output of the third dense connection and transition module M3 and the output of the first deconvolution layer T1, and the output of the first splicing layer CA1 is connected to the input of the third convolution layer CO3; the input of the second deconvolution layer T2 is connected to the output of the third convolution layer CO3, the two inputs of the second splicing layer CA2 are respectively connected to the output of the second dense connection and transition module M2 and the output of the second deconvolution layer T2, and the output of the second splicing layer CA2 is connected to the fourth convolution layer CO 4; the input of the third deconvolution layer T3 is connected to the output of the fourth convolution layer CO4, the two inputs of the third splicing layer CA3 are respectively connected to the output of the first dense connection and transition module M1 and the output of the third deconvolution layer T3, and the output of the third splicing layer CA3 is connected to the input of the fifth convolution layer CO5; the input of the fourth deconvolution layer T4 is connected to the output of the fifth convolution layer CO5, the two inputs of the fourth splicing layer CA4 are respectively connected to the output of the second activation function layer R2 and the output of the fourth deconvolution layer T4, and the output of the fourth splicing layer CA4 is connected to the input of the sixth convolution layer CO6.

5. The SAR ship classification method based on contrastive learning pre-training according to claim 4 is characterized in that: The second feature fusion module includes the seventh convolution layer CO7, the third batch normalization layer B3, the third activation function layer R3, the first pooling layer P1, the fifth splicing layer CA5, the eighth convolution layer CO8, the fourth batch normalization layer B4, the fourth activation function layer R4, the second pooling layer P2, the sixth splicing layer CA6, the ninth convolution layer CO9, the fifth batch normalization layer B5, the fifth activation function layer R5, the third pooling layer P3, the seventh splicing layer CA7, the tenth convolution layer CO 10 , the sixth batch normalization layer B6, the sixth activation function layer R6, the fourth pooling layer P4, the eighth splicing layer CA8, the eleventh convolution layer CO 11 , seventh batch normalization layer B7, seventh activation function layer R7, fifth pooling layer P5, eighth batch normalization layer B8, eighth activation function layer R8, sixth pooling layer P6, flattening layer FL, first fully connected layer FC1 and second fully connected layer FC2, where, The seventh convolutional layer CO7, the third batch normalization layer B3, the third activation function layer R3 and the first pooling layer P1 are cascaded in sequence, and the input of the seventh convolutional layer CO7 is connected to the output of the sixth convolutional layer CO6, and the output of the first pooling layer P1 and the output of the fifth convolutional layer CO5 are connected to the input of the fifth splicing layer CA5; the eighth convolutional layer CO8, the fourth batch normalization layer B4, the fourth activation function layer R4 and the second pooling layer P2 are cascaded in sequence, and the input of the eighth convolutional layer CO8 is connected to the output of the fifth splicing layer CA5, and the output of the second pooling layer P2 and the output of the fourth convolutional layer CO4 are connected to the input of the sixth splicing layer CA6; The ninth convolutional layer CO9, the fifth batch normalization layer B5, the fifth activation function layer R5 and the third pooling layer P3 are cascaded in sequence, and the input of the ninth convolutional layer CO9 is connected to the output of the sixth splicing layer CA6, the output of the third pooling layer P3 and the output of the third convolutional layer CO3 are connected to the input of the seventh splicing layer CA7; the tenth convolutional layer CO 10 , the sixth batch normalization layer B6, the sixth activation function layer R6 and the fourth pooling layer P4 are cascaded in sequence, and the tenth convolutional layer CO 10 The input of the eleventh convolutional layer CO is connected to the output of the seventh concatenation layer CA7, the output of the fourth pooling layer P4 and the output of the fourth dense connection and transition module M4 are connected to the input of the eighth concatenation layer CA8; the eleventh convolutional layer CO 11 , the seventh batch normalization layer B7, the seventh activation function layer R7 and the fifth pooling layer P5 are connected in sequence, and the eleventh convolutional layer CO 11 The input is connected to the output of the eighth concatenation layer CA8, and the output of the fifth pooling layer P5 is sequentially connected to the eighth batch normalization layer B8, the eighth activation function layer R8, the sixth pooling layer P6, the flattening layer FL, the first fully connected layer FC1 and the second fully connected layer FC2.

6. The SAR ship classification method based on contrastive learning pre-training according to claim 5 is characterized in that: The feature mapping network is a multi-layer perceptron, including a third fully connected layer FC3, a ninth activation function layer R9 and a fourth fully connected layer FC4 cascaded in sequence, and the input of the third fully connected layer FC3 is connected to the output of the second fully connected layer FC2.

7. The SAR ship classification method based on contrastive learning pre-training according to claim 1 is characterized in that: The S3 includes: S3.1: The OpenSARShip dataset is divided into five training sets and test sets in a ratio of 8:2 to obtain five training sets {φ1, φ2, φ3, φ4, φ5} and five corresponding test sets {t1, t2, t3, t4, t5}. The training set {φ1, φ2, φ3, φ4, φ5} is expanded and cropped to obtain the training set {Φ1, Φ2, Φ3, Φ4, Φ5}. The test set {t1, t2, t3, t4, t5} is cropped to obtain the test set {T1, T2, T3, T4, T5}. The mat format training set {Φ1, Φ2, Φ3, Φ4, Φ5} is converted to the JPG format image training set {Φ1', Φ′2, Φ′3, Φ′4, Φ′5'}. S3.2: Input the JPG image dataset {Φ1', Φ'2, Φ'3, Φ'4, Φ'5} into the SimCLR contrastive learning framework for pre-training to obtain a pre-trained model {ψ'1, ψ'2, ψ'3, ψ'4, ψ'5}. The loss function used is contrastive loss: Among them, z i It represents the feature vector obtained by feature extraction and feature mapping of the i-th training data in the training data set, z j It represents the feature vector obtained by feature extraction and feature mapping of the j-th training data in the training data set, s i,j represents the similarity between the eigenvectors of the i-th training data and the j-th training data, l(i,j) represents the similarity of the eigenvectors after function transformation, τ represents the temperature coefficient, and L represents the average similarity of all sample pairs obtained by data enhancement in a data set containing N samples. In the calculation process, for the k-th sample in the data of N samples, the 2k-1 and 2k-th samples in the data of 2N samples after data enhancement are obtained by two data enhancement methods: random cropping and scaling and Gaussian blurring.

8. The SAR ship classification method based on contrastive learning pre-training according to claim 7 is characterized in that: The S4 includes: The parameters of each pre-trained model {ψ′1, ψ′2, ψ′3, ψ′4, ψ′5} are loaded into the feature fusion network ψ, and the mat format dataset {Φ1, Φ2, Φ3, Φ4, Φ5} corresponding to the JPG format image dataset {Φ1', Φ′2, Φ′3, Φ′4, Φ′5} is input into the feature fusion network ψ for fine-tuning training to obtain the trained feature fusion network.

Images