Ship image classification method based on local features and global features

Abstract

The present invention discloses a ship image classification method based on local features and global features. The method is based on a classification network model, the input of which is a ship image, and the model includes: an attention score module, a densely connected network, a local feature representation extraction branch, a convolution volume transformer, and a global feature representation extraction branch. The present invention first obtains the attention score, and then respectively uses a dense convolution network and a convolution visual transformer to obtain the local features and global features of the original image, and then uses the attention score for further feature enhancement, and respectively uses convolution kernels of different sizes and pooled attention to obtain the final representation of the ship image for the enhanced two features, and finally uses the cross entropy loss to perform back propagation optimization on the constructed network model, and finally effectively improves the accuracy of ship image classification.

Description

Technical Field

[0001] The invention belongs to the field of computer vision and relates to an image classification method, in particular to a ship image classification method. Background Art

[0002] Deep learning image classification technology has been widely used in various fields, and the field of ship image classification is no exception. Traditional methods based on manual features, such as SIFT and SILTP, use manually designed feature extractors for classification based on image texture, contour and other features. These methods require professional knowledge and complex parameter adjustments, and are highly targeted, with poor generalization and robustness.

[0003] In contrast, deep learning is a data-driven approach that can obtain deep, dataset-specific feature representations through learning from a large number of samples. These features have higher expressiveness and accuracy, stronger robustness and generalization capabilities, and can achieve end-to-end training and prediction.

[0004] In recent years, deep learning-based ship image classification methods and models, such as GoogleNet and ConvNeXt, have achieved remarkable results. However, these models only focus on the local features of the image and ignore the global features of the image, resulting in incomplete modeling of image representation and low classification accuracy. Summary of the invention

[0005] The present invention proposes a ship image classification method based on local features and global features, the purpose of which is to comprehensively consider the local features and global features of the image, improve the image representation, and improve the accuracy of classification.

[0006] The technical solution of the present invention is as follows:

[0007] A ship image classification method based on local features and global features comprises the following steps:

[0008] Step 1: construct a classification network model, wherein the input of the classification network model is a ship image; the classification network model includes:

[0009] The attention score module is used to obtain the attention score based on the input image of the classification network model;

[0010] Densely connected networks are used to obtain local features of the input image for the classification network model;

[0011] A local feature representation extraction branch, used for obtaining enhanced local features based on the local features and the attention score, then obtaining a final representation of the local features based on the enhanced local features, and obtaining a first classification result based on the final representation of the local features;

[0012] Convolutional volume transformer, used to obtain global features of the input image of the classification network model;

[0013] A global feature representation extraction branch is used to obtain enhanced global features based on the global features and the attention score, and then obtain a final representation of the global features based on the enhanced global features, and obtain a second classification result based on the final representation of the global features;

[0014] The output of the classification network model is a final classification result obtained based on the first classification result and the second classification result;

[0015] Step 2: training the classification network model;

[0016] Step 3: Input the ship image to be classified into the classification network model to obtain the image classification result.

[0017] As a further improvement of the ship image classification method based on local features and global features, in the attention score module, the attention score is obtained by:

[0018] Step A1: For the input image x∈R C×H×W Two independent convolution operations are performed, C is the number of image channels, H is the image height, and W is the image width, to obtain two features: F i1 and F i2 ;

[0019] Step A2: feature F i1 , F i2 Perform feature enhancement:

[0020] F i1′ =PConv(F i1 )+GConv(F i1 );

[0021] F i2′ =Concat(F i2 , PConv(F i2 ));

[0022] Among them, PConv(·) represents point convolution, GConv(·) represents group convolution; Concat(·) represents concatenation operation;

[0023] Step A3: For feature F i1′ , F i2′ Perform a pooling operation and then obtain the attention score S through Softmax activation:

[0024] S = Sovtmax(MaxPool(F i1′))+Softmax(MaxPoof(F i2′ ));

[0025] Among them, MaxPoof(·) represents the maximum pooling operation, and Softmax(·) represents the activation function that converts a set of arbitrary real numbers into real numbers representing probability distribution.

[0026] As a further improvement of the ship image classification method based on local features and global features, in the densely connected network, the local features of the input image are obtained by:

[0027] Step B1: Input the input image to three dense modules connected in sequence; each dense module contains four convolutional layers, and the input of each convolutional layer contains the output of all convolutional layers before the convolutional layer in the same dense module;

[0028] For any dense module, let its input be F D0 , then the calculation process of each convolutional layer in the dense module is as follows:

[0029] F D1 =ReLu(Conv(F D0 ));

[0030] F D2 =ReLu(Conv(Concat(F D0 , F D1 )));

[0031] F D3 =ReLu(Conv(Concat(F D0 , F D1 , F D2 )));

[0032] F D4 =ReLu(Conv(Concat(F D0 , F D1 , F D2 , F D3 )));

[0033] Among them, Conv(·) represents the convolution operation, ReLu(·) represents the ramp activation function, and Concat(·) represents the concatenation operation; F D1 , F D2 , F D3 , F D4 They represent the outputs of the 1st, 2nd, 3rd, and 4th convolutional layers in the dense module respectively;

[0034] Step B2: Output F of the third dense module D′The local feature Sigmoid (F D′ ), where Sigmoid(·) represents the S-type activation function.

[0035] As a further improvement of the ship image classification method based on local features and global features, the final representation of the local features in the local feature representation extraction branch is obtained as follows:

[0036] Step C1: transform the local feature Sigmoid (F D′ ) is matrix-multiplied with the obtained attention score S to obtain the enhanced local feature F D″ :

[0037] F D″ =S×Sigmoid(F D′ );

[0038] Step C2: Obtain the final representation F of the local features through convolution calculation and Sigmoid activation function D :

[0039] F D1″ =Conv 1×1 (F D″ );

[0040] F D2″ =Conv 3×1 (Conv 1×3 (Conv 3×3 (F D″ )));

[0041] F D3″ =Conv 5×1 (Conv 1×5 (Conv 5×3 (F D″ )));

[0042] F D4″ =Conv 7×1 (Conv 1×7 (Conv 7×7 (F D″ )));

[0043] F D =Sigmoid(F D1″ +F D2″ +F D3″ +F D4″ );

[0044] Among them, Conv i×j (·) represents a convolution operation with kernel i×j.

[0045] As a further improvement of the ship image classification method based on local features and global features, in the convolutional volume transformer, the global features of the input image are obtained by:

[0046] Step D1: For the input image x∈R C×H×W Perform convolution and flattening, C is the number of image channels, H is the image height, W is the image width, and get the matrices Q, K, and V for multi-head attention calculation:

[0047] Q = Flatten(Conv Q (x));

[0048] K = Flatten(Conv K (x));

[0049] V=Flatten(Conv V (x));

[0050] Among them, Conv Q (·), Conv K (·), Conv V (·) represents three convolution operations with different convolution kernel parameters, where the convolution kernel parameters are learnable parameters; Flatten(·) represents a flattening operation;

[0051] Step D2: Input the obtained Q, K, V into the multi-head attention module to obtain the global feature F T2 , the process is as follows:

[0052] First calculate the feature F T1 :

[0053] F T1 =MHSA(LN(Q,K,V))+Flatten(x);

[0054] Among them, LN(·) represents the normalization layer, MHSA(·) represents the multi-head attention mechanism;

[0055] The MHSA(·) process is as follows:

[0056] The normalized Q, K, V matrices are split into h parts along the channel dimension, where h is a hyperparameter representing the number of heads of multi-head attention, and the i-th segmentation feature is recorded as Q i , K i , V i ,in d k =C1 / h, in the multi-head attention calculation, the calculation result of each head is represented by Hi, i=1,…,h, then:

[0057]

[0058] Among them, Softmax(·) represents an activation function that converts a set of arbitrary real numbers into real numbers representing probability distribution;

[0059] MHSA(LN(Q,K,V))=Concat(H1,H2,…,H h );

[0060] Among them, Concat(·) represents the concatenation operation;

[0061] Then calculate the global feature F T2 :

[0062] F T2 =MLP(LN(F T1 ))+F T1 ;

[0063] Here, MLP(·) stands for Multi-Layer Perceptron.

[0064] As a further improvement of the ship image classification method based on local features and global features, the final representation of the global features is obtained in the global feature representation extraction branch as follows:

[0065] Step E1: Set feature F T2 Perform matrix multiplication with the attention score S to obtain the enhanced global features:

[0066] F T2′ =S×F T2 ;

[0067] Step E2: The enhanced global feature F T2′ Input into the pooling attention module to obtain feature F T2″ ; The pooling attention module includes a normalization layer, an average pooling layer, and a multi-head attention layer. The processing method of the pooling attention module is:

[0068] F T2″ =PMHSA(LN(F T2′ ));

[0069] Among them, LN(·) represents the normalization layer;

[0070] PMHSA(·) is a pooled multi-head attention, and its calculation process is:

[0071] Set three learnable parameter matrices W Q′ , W K′ , W V′ ∈R M×M , respectively with LN(FT2′ ) to obtain Three matrices;

[0072] Q′=W Q′ ·LN(F T2′ );

[0073] K′=W K′ ·LN(F T2′ );

[0074] V′=W V′ ·LN(F T2′ );

[0075] Perform average pooling on K′, V′ to reduce the computational complexity of multi-head attention:

[0076] K″=AvgPoof(K′);

[0077] V ″ =AvgPoof(V′);

[0078] in AvgPoof(·) is the average pooling layer;

[0079] Divide Q′, K″, V″ into h parts along the channel dimension, where h is a hyperparameter representing the number of heads of multi-head attention, and record the i-th segmentation feature as Q′ i , K i ″,V i ",in d k′ =C1 / h, in the multi-head attention calculation, the calculation result of each head is H′ i It means, i=1,…,h;

[0080]

[0081] Where Sovtmax(·) represents an activation function that converts a set of arbitrary real numbers into real numbers representing probability distributions;

[0082] PMHSA(LN(F T2′ ))=Concat(H′1,H′2,…,H′ h );

[0083] Among them, Concat(·) represents the concatenation operation;

[0084] Step E3: feature F T2″ After the 1×1 convolution operation, it is input into the depthwise separable convolution module and then passes through the Sigmoid activation function to obtain the final representation of the global feature F. T.

[0085] As a further improvement of the ship image classification method based on local features and global features, the depthwise separable convolution module includes DW convolution and two convolutions with 1×1 convolution kernels. The specific calculation process of step E3 is:

[0086] F T =Sigmoid(Conv(Conv(DWConv(Conv(F T2″ ))+Conv(F T2″ ))));

[0087] Among them, Sigmoid(·) represents the S-type activation function, Conv(·) represents the convolution operation, and DWConv(·) represents the DW convolution operation.

[0088] As a further improvement of the ship image classification method based on local features and global features, the first classification result is calculated as follows:

[0089]

[0090] in, is the first classification result, F D is the final representation of local features, Linear(·) represents the linear layer, and Softmax(·) represents the activation function that converts a set of arbitrary real numbers into real numbers representing probability distribution;

[0091] The calculation method of the second classification result is:

[0092]

[0093] in, is the second classification result, F T It is the final representation of the global features.

[0094] As a further improvement of the ship image classification method based on local features and global features, the final classification result is calculated as follows:

[0095]

[0096] Among them, Y is a one-dimensional vector, representing the final classification result predicted by the classification network model; Indicates that The value of the largest element in is set to 1, and the other elements are set to 0, that is The category corresponding to the largest element in is the final classification result.

[0097] As a further improvement of the ship image classification method based on local features and global features, the specific training method of step 2 is:

[0098] Step 2-1: Obtain ship images with image labels and construct a sample dataset; each ship image and its corresponding category are recorded as Among them, X i ∈R C×H×W represents the i-th ship image, C represents the number of image channels, H represents the image height, W represents the image width, and Y i Represents the ship image X i The corresponding real category; i = 1, 2, ..., N, N represents the number of ship images;

[0099] Step 2-2: Input the ship images in the sample data set into the classification network model to obtain the first classification result and the second classification result; let the first classification result corresponding to the i-th ship image be The second classification results are

[0100] Step 2-3, calculate the cross entropy loss:

[0101]

[0102] Step 2-4, calculate the total loss function:

[0103] L total =αL1+βL2;

[0104] Among them, α and β are preset hyperparameters;

[0105] Step 2-5: According to L total Update parameters in the classification network model;

[0106] Step 2-6: Determine whether the number of updates reaches the preset value. If not, return to step 2-2 to continue updating and optimizing. If it reaches the preset value, the training is completed.

[0107] Compared with the prior art, the present invention has the following beneficial effects:

[0108] The present invention first uses pointwise convolution and group convolution to obtain attention scores, and then uses dense convolution network (DenseNet) and convolutional visual transformer (CVT) to obtain local features and global features of the original image respectively. The attention scores are then used for further feature enhancement. The final representation of the ship image is obtained by using convolution kernels of different sizes and pooling attention for the two enhanced features. Finally, the cross entropy loss is used to perform back propagation optimization on the constructed network model, which effectively improves the accuracy of ship image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0109] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0110] The technical solution of the present invention is described in detail below with reference to the accompanying drawings:

[0111] like Figure 1 , a ship image classification method based on local features and global features, comprising the following steps:

[0112] Step 1: construct a classification network model, the input of which is a ship image. The classification network model includes:

[0113] 1. Attention score module, which is used to obtain attention scores based on the input image of the classification network model.

[0114] Specifically, in the attention score module, the attention score is obtained in the following way:

[0115] Step A1: For the input image x∈R C×H×W Two independent convolution operations are performed, C is the number of image channels, H is the image height, and W is the image width, to obtain two different features: F i1 and F i2 .

[0116] Step A2: feature F i1 , F i2 Perform feature enhancement:

[0117] F i1′ =PConv(F i1 )+GConv(F i1 );

[0118] F i2′ =Concat(F i2 , PConv(F i2 ));

[0119] Among them, PConv(·) represents point convolution, GConv(·) represents group convolution, and Concat(·) represents concatenation operation.

[0120] Step A3: For feature F i1′ , F i2′ Perform a pooling operation and then obtain the attention score S through Softmax activation:

[0121] S = Softmax(MaxPool(F i1″ ))+Softmax(MaxPool(F i2′ ));

[0122] Among them, MaxPoof(·) represents the maximum pooling operation, and Softmax(·) represents the activation function that converts a set of arbitrary real numbers into real numbers representing probability distribution.

[0123] 2. Densely connected network (DenseNet), used to obtain local features of the input image of the classification network model.

[0124] Specifically, in the densely connected network, the local features of the input image are obtained in the following manner:

[0125] Step B1: Input the input image to three dense modules connected in sequence; each dense module contains four convolutional layers, and the input of each convolutional layer contains the output of all convolutional layers before the convolutional layer in the same dense module.

[0126] For any dense module, let its input be F D0 , then the calculation process of each convolutional layer in the dense module is as follows:

[0127] F D1 =ReLu(Conv(F D0 ));

[0128] F D2 =ReLu(Conv(Concat(F D0 , F D1 )));

[0129] F D3 =ReLu(Conv(Concat(F D0 , F D1 , F D2 )));

[0130] F D4 =ReLu(Conv(Concat(F D0 , F D1 , F D2 , F D3)));

[0131] Among them, Conv(·) represents the convolution operation, ReLu(·) represents the ramp activation function, and Concat(·) represents the concatenation operation; F D1 , F D2 , F D3 , F D4 They represent the outputs of the 1st, 2nd, 3rd, and 4th convolutional layers in the dense module respectively.

[0132] Step B2: Output F of the third dense module D′ The local feature Sigmoid (F D′ ), where Sigmoid(·) represents the S-type activation function.

[0133] 3. A local feature representation extraction branch is used to obtain enhanced local features based on local features and attention scores, and then obtain a final representation of the local features based on the enhanced local features, and obtain a first classification result based on the final representation of the local features.

[0134] Specifically, the final representation of the local features in the local feature representation extraction branch is obtained as follows:

[0135] Step C1: transform the local feature Sigmoid (F D′ ) is matrix-multiplied with the obtained attention score S to obtain the enhanced local feature F D″ :

[0136] F D″ =S×Sigmoid(F D′ ).

[0137] Step C2: Obtain the final representation F of the local features through convolution calculation and Sigmoid activation function D :

[0138] F D1″ =Conv 1×1 (F D″ );

[0139] F D2″ =Conv 3×1 (Conv 1×3 (Conv 3×3 (F D″ )));

[0140] F D3″ =Conv 5×1 (Conv 1×5 (Conv 5×3 (FD″ )));

[0141] F D4″ =Conv 7×1 (Conv 1×7 (Conv 7×7 (F D ″)));

[0142] FD=Sigmoid(F D1″ +F D2″ +F D3″ +F D4″ );

[0143] Among them, Conv i×j (·) represents a convolution operation with kernel i×j.

[0144] 4. Convolutional Volume Transformer (CVT), which is used to obtain the global features of the input image of the classification network model.

[0145] Specifically, in the convolutional volume transformer, the global features of the input image are obtained in the following manner:

[0146] Step D1: For the input image x∈R C×H×W Perform convolution and flattening, C is the number of image channels, H is the image height, W is the image width, and get the matrices Q, K, and V for multi-head attention calculation:

[0147] Q = Flatten(Conv Q (x));

[0148] K = Flatten(Conv K (x));

[0149] V=Flatten(Conv V (x));

[0150] Among them, Conv Q (·), Conv K (·), Conv V (·) represents three convolution operations with different convolution kernel parameters, where the convolution kernel parameters are learnable parameters; Flatten(·) represents the flattening operation; Q, K,

[0151] Step D2: Input the obtained Q, K, V into the multi-head attention module to obtain the global feature F T2 , the process is as follows:

[0152] First calculate the feature F T1 :

[0153] FT1 =MHSA(LN(Q,K,V))+Flatten(x);

[0154] Among them, LN(·) represents the normalization layer and MHSA(·) represents the multi-head attention mechanism.

[0155] The MHSA(·) process is as follows:

[0156] The normalized Q, K, V matrices are split into h parts along the channel dimension, where h is a hyperparameter representing the number of heads of multi-head attention, and the i-th segmentation feature is recorded as Q i , K i , V i ,in d k =C1 / h, in the multi-head attention calculation, the calculation result of each head is represented by Hi, i=1,…,h, then:

[0157]

[0158] Among them, Softmax(·) represents an activation function that converts a set of arbitrary real numbers into real numbers representing probability distribution;

[0159] MHSA(LN(Q,K,V))=Concat(H1,H2,…,H h );

[0160] Among them, Concat(·) represents the concatenation operation;

[0161] Then calculate the global feature F T2 :

[0162] F T2 =MLP(LN(F T1 ))+F T1 ;

[0163] Here, MLP(·) stands for Multi-Layer Perceptron.

[0164] 5. A global feature representation extraction branch is used to obtain enhanced global features based on global features and attention scores, and then obtain the final representation of the global features based on the enhanced global features, and obtain the second classification result based on the final representation of the global features.

[0165] Specifically, the final representation of the global feature is obtained in the global feature representation extraction branch as follows:

[0166] Step E1: Set feature F T2 Perform matrix multiplication with the attention score S to obtain the enhanced global features:

[0167] FT2′ =S×F T2 .

[0168] Step E2: The enhanced global feature F T2′ Input into the pooling attention module to obtain feature F T2″ ; The pooling attention module includes a normalization layer, an average pooling layer, and a multi-head attention layer. The processing method of the pooling attention module is:

[0169] F T2″ =PMHSA(LN(F T2′ ));

[0170] Among them, LN(·) represents the normalization layer;

[0171] PMHSA(·) is a pooled multi-head attention, and its calculation process is:

[0172] Set three learnable parameter matrices W Q′ , W K′ , W V′ ∈R M×M , respectively with LN(F T2′ ) to obtain Three matrices;

[0173] Q′=W Q′ .LN(F T2′ );

[0174] K′=W K′ ·LN(F T2′ );

[0175] V′=W V′ ·LN(F T2′ );

[0176] Perform average pooling on K′, V′ to reduce the computational complexity of multi-head attention:

[0177] K″=AvgPool(K′);

[0178] V″=AvgPool(V′);

[0179] in AvgPool(·) is the average pooling layer;

[0180] Divide Q′, K″, V″ into h parts along the channel dimension, where h is a hyperparameter representing the number of heads of multi-head attention, and record the i-th segmentation feature as Q′ i , K″ i , V″ i ,in dk′ =C1 / h, in the multi-head attention calculation, the calculation result of each head is H′ i It means, i=1,…,h;

[0181]

[0182] Among them, Softmax(·) represents an activation function that converts a set of arbitrary real numbers into real numbers representing probability distribution;

[0183] PMHSA(LN(F T2′ ))=Concat(H′1,H′2,…,H′ h );

[0184] Among them, Concat(·) represents the concatenation operation.

[0185] Step E3: feature F T2″ After the 1×1 convolution operation, it is input into the depthwise separable convolution module and then passes through the Sigmoid activation function to obtain the final representation of the global feature F. T .

[0186] The depthwise separable convolution module includes DW convolution and two convolutions with 1×1 convolution kernels. The specific calculation process of step E3 is as follows:

[0187] F T =Sigmoid(Conv(Conv(DWConv(Conv(F T2″ ))+Conv(F T2″ ))));

[0188] Among them, Sigmoid(·) represents the S-type activation function, Conv(·) represents the convolution operation, and DWConv(·) represents the DW convolution operation.

[0189] The output of the classification network model is a final classification result obtained based on the first classification result and the second classification result.

[0190] The calculation method of the first classification result is:

[0191]

[0192] in, is the first classification result, F D is the final representation of local features, Linear(·) represents the linear layer, and Softmax(·) represents the activation function that converts a set of arbitrary real numbers into real numbers representing probability distribution.

[0193] The calculation method of the second classification result is:

[0194]

[0195] in, is the second classification result, F T It is the final representation of the global features.

[0196] The final classification result is calculated as:

[0197]

[0198] Among them, Y is a one-dimensional vector, representing the final classification result predicted by the classification network model; Indicates that The value of the largest element in is set to 1, and the other elements are set to 0, that is The category corresponding to the largest element in is the final classification result.

[0199] Step 2: training the classification network model.

[0200] The specific training methods are:

[0201] Step 2-1: Obtain ship images with image labels and construct a sample dataset; each ship image and its corresponding category are recorded as Among them, X i ∈R C×H×W represents the i-th ship image, C represents the number of image channels, H represents the image height, M represents the image width, and Y i Represents the ship image X i The corresponding true category; i = 1, 2, ..., N, where N represents the number of ship images.

[0202] Step 2-2: Input the ship images in the sample data set into the classification network model to obtain the first classification result and the second classification result; let the first classification result corresponding to the i-th ship image be The second classification results are

[0203] Step 2-3, calculate the cross entropy loss:

[0204]

[0205] Step 2-4, calculate the total loss function:

[0206] L total =αL1+βL2;

[0207] Wherein α and β are both preset hyperparameters; in this embodiment, α is set to 0.5 and β is set to 0.5.

[0208] Step 2-5: According to L totalUpdate the learnable parameters in the classification network model.

[0209] Step 2-6: Determine whether the number of updates reaches the preset value. If not, return to step 2-2 to continue updating and optimizing. If it reaches the preset value, the training is completed.

[0210] In this embodiment, the Adam optimizer is used to optimize and solve the loss function.

[0211] Step 3: Input the ship image to be classified into the classification network model to obtain the image classification result.

Claims

1. A ship image classification method based on local features and global features, characterized by: The following steps are involved: Step 1: construct a classification network model, wherein the input of the classification network model is a ship image; Classification network models include: An attention score module is used to obtain an attention score based on the input image of the classification network model; a densely connected network is used to obtain local features of the input image of the classification network model; A local feature representation extraction branch, used for obtaining enhanced local features based on the local features and the attention score, then obtaining a final representation of the local features based on the enhanced local features, and obtaining a first classification result based on the final representation of the local features; Convolutional volume transformer, used to obtain global features of the input image of the classification network model; A global feature representation extraction branch is used to obtain enhanced global features based on the global features and the attention score, and then obtain a final representation of the global features based on the enhanced global features, and obtain a second classification result based on the final representation of the global features; The output of the classification network model is a final classification result obtained based on the first classification result and the second classification result; In the densely connected network, the local features of the input image are obtained in the following way: Step B1: Input the input image to three dense modules connected in sequence; each dense module contains four convolutional layers, and the input of each convolutional layer contains the output of all convolutional layers before the convolutional layer in the same dense module; For any dense module, let its input be F D0 , then the calculation process of each convolutional layer in the dense module is as follows: F D1 =ReLu(Conv(F D0 )); F D2 =ReLu(Conv(Concat(F D0 ,F D1 ))); F D3 =ReLu(Conv(Concat(F D0 ,F D1 ,F D2 ))); F D4 =ReLU(Conv(Concat(F D0 ,F D1 ,F D2 ,F D3 ))); Among them, Conv(·) represents the convolution operation, ReLu(·) represents the ramp activation function, and Concat(·) represents the concatenation operation; F D1 ,F D2 ,F D3 ,F D4 They represent the outputs of the 1st, 2nd, 3rd, and 4th convolutional layers in the dense module respectively; Step B2: Output F of the third dense module D′ The local feature Sigmoid (F D′ ), where Sigmoid(·) represents the S-type activation function; The final representation of local features in the local feature representation extraction branch is obtained as follows: Step C1: transform the local feature Sigmoid (F Dρ ) is matrix-multiplied with the obtained attention score S to obtain the enhanced local feature F D″ : F D″ =S×Sigmoid(F D′ ); Step C2: Obtain the final representation F of the local features through convolution calculation and Sigmoid activation function D : F D1″ =Conv 1×1 (F D″ ); F D2″ =Conv 3×1 (Conv 1×3 (Conv 3×3 (F D″ ))); F D3″ =Conv 5×1 (Conv 1×5 (Conv 5×3 (F d″ ))); F D4″ =Conv 7×1 (Conv 1×7 (Conv 7×7 (F D″ ))); F D =Sigmoid(F D1″ +F D2″ +F D3″ +F D4″ ); Among them, Conv i×j (·) represents the convolution operation with kernel i×j; Step 2: training the classification network model; Step 3: Input the ship image to be classified into the classification network model to obtain the image classification result.

2. The ship image classification method based on local features and global features as claimed in claim 1, characterized in that: In the attention score module, the attention score is obtained in the following way: Step A1: For the input image x∈R C×H×W Two independent convolution operations are performed, with C being the number of image channels, H being the image height, and W being the image width, to obtain two features: F i1 and F i2 ; Step A2: feature F i1 ,F i2 Perform feature enhancement: F i1′ =PConv(F i1 )+GConv(F i1 ); F i2′ =Concat(F i2 ,PConv(F i2 )); Among them, PConv(·) represents point convolution, GConv(·) represents group convolution; Concat(·) represents concatenation operation; Step A3: For feature F i1′ ,F i2′ Perform a pooling operation and then obtain the attention score S through Softmax activation: S=Softmax(MaxPool(F i1′ ))+Softmax(MaxPool(F i2′ )); Among them, MaxPool(·) represents the maximum pooling operation, and Softmax(·) represents the activation function that converts a set of arbitrary real numbers into real numbers representing probability distribution.

3. The ship image classification method based on local features and global features as claimed in claim 1, characterized in that: In the convolutional volume transformer, the global features of the input image are obtained in the following way: Step D1: For the input image x∈R C×H×W Perform convolution and flattening, C is the number of image channels, H is the image height, W is the image width, and get the matrices Q, K, and V for multi-head attention calculation: Q=Flatten(Conv Q (x)); K=Flatten(Conv K (x)); V=Flatten(Conv V (x)); Among them, Conv Q (·),Conv K (·),Conv V (·) represents three convolution operations with different convolution kernel parameters, where the convolution kernel parameters are learnable parameters; Flatten(·) represents a flattening operation; Step D2: Input the obtained Q, K, V into the multi-head attention module to obtain the global feature F T2 , the process is as follows: First calculate the feature F T1 : F T1 =MHSA(LN(Q,K,V))+Flatten(x); Among them, LN(·) represents the normalization layer, MHSA(·) represents the multi-head attention mechanism; The MHSA(·) process is as follows: The normalized Q, K, V matrices are divided into h parts along the channel dimension, where h is a hyperparameter representing the number of heads of multi-head attention, and the i-th segmentation feature is recorded as Q i ,K i ,V i ,in d k =C1 / h, in the multi-head attention calculation, the calculation result of each head is represented by H i Indicates that i=1,···,h, then: Among them, Softmax(·) represents an activation function that converts a set of arbitrary real numbers into real numbers representing probability distribution; MHSA(LN(Q,K,V))=Concat(H1,H2,···,H h ); Among them, Concat(·) represents the concatenation operation; Then calculate the global feature F T2 : F T2 =MLP(LN(F T1 ))+F T1 ; Here, MLP(·) stands for Multi-Layer Perceptron.

4. The ship image classification method based on local features and global features as claimed in claim 3, characterized in that: The final representation of the global feature in the global feature representation extraction branch is obtained as follows: Step E1: Set feature F T2 Perform matrix multiplication with the attention score S to obtain the enhanced global features: F T2′ =S×F T2 ; Step E2: The enhanced global feature F T2′ Input into the pooling attention module to obtain feature F T2″ ; The pooling attention module includes a normalization layer, an average pooling layer, and a multi-head attention layer. The processing method of the pooling attention module is: F T2″ =PMHSA(LN(F T2′ )); Among them, LN(·) represents the normalization layer; PMHSA(·) is a pooled multi-head attention, and its calculation process is: Set three learnable parameter matrices W Q′ ,W K′ ,W V ′∈R M×M , respectively with LN(F T2′ ) to obtain Three matrices; Q′=W Q′ ·LN(F T2′ ); K′=W K′ ·LN(F T2′ ); V′=W V′ ·LN(F T2′ ); Perform average pooling on K′, V′ to reduce the computational complexity of multi-head attention: K″=AvgPool(K′); V″=AvgPool(V′); in AvgPool(·) is the average pooling layer; Divide Q′, K″, V″ into h parts along the channel dimension, where h is a hyperparameter representing the number of heads of multi-head attention, and record the i-th segmentation feature as Q′ i ,K″ i ,V″ i ,in d k′ =C1 / h, in the multi-head attention calculation, the calculation result of each head is H′ i It means, i=1,···,h; Among them, Softmax(·) represents an activation function that converts a set of arbitrary real numbers into real numbers representing probability distribution; PMHSA(LN(F T2′ ))=Concat(H′1,H′2,···,H′ h ); Among them, Concat(·) represents the concatenation operation; Step E3: feature F T2″ After the 1×1 convolution operation, it is input into the depthwise separable convolution module and then passes through the Sigmoid activation function to obtain the final representation of the global feature F. T .

5. The ship image classification method based on local features and global features as claimed in claim 4, characterized in that: The depthwise separable convolution module includes DW convolution and two convolutions with 1×1 convolution kernels. The specific calculation process of step E3 is as follows: F T =Sigmoid(Conv(Conv(DWConv(Conv(F T2″ ))+Conv(F T2″ )))); Among them, Sigmoid(·) represents the S-type activation function, Conv(·) represents the convolution operation, and DWConv(·) represents the DW convolution operation.

6. The ship image classification method based on local features and global features as claimed in claim 1, characterized in that: The calculation method of the first classification result is: in, is the first classification result, F D is the final representation of local features, Linear(·) represents the linear layer, and Softmax(·) represents the activation function that converts a set of arbitrary real numbers into real numbers representing probability distribution; The calculation method of the second classification result is: in, is the second classification result, F T It is the final representation of the global features.

7. The ship image classification method based on local features and global features as claimed in claim 6, characterized in that: The final classification result is calculated as: Among them, Y is a one-dimensional vector, representing the final classification result predicted by the classification network model; Indicates that The value of the largest element in is set to 1, and the other elements are set to 0, that is The category corresponding to the largest element in is the final classification result.

8. The ship image classification method based on local features and global features as claimed in claim 6, characterized in that: The specific training method for step 2 is: Step 2-1: Obtain ship images with image labels and construct a sample dataset; each ship image and its corresponding category are recorded as Among them, X i ∈R C×H×W represents the i-th ship image, C represents the number of image channels, H represents the image height, W represents the image width, and Y i Represents the ship image X i The corresponding real category; i = 1, 2, ..., N, N represents the number of ship images; Step 2-2: Input the ship images in the sample data set into the classification network model to obtain the first classification result and the second classification result; let the first classification result corresponding to the i-th ship image be The second classification results are Step 2-3, calculate the cross entropy loss: Step 2-4, calculate the total loss function: L total =αL1+βL2; Among them, α and β are preset hyperparameters; Step 2-5: According to L total Update parameters in the classification network model; Step 2-6: Determine whether the number of updates reaches the preset value. If not, return to step 2-2 to continue updating and optimizing. If it reaches the preset value, the training is completed.

Images