A few-shot ship re-identification method and device based on global-to-local progressive learning, storage medium and computing device

Abstract

The application discloses a few-shot ship re-identification method and device based on global-to-local progressive learning, a storage medium and computing equipment, and the method comprises three stages in the whole. In the first stage, a feature pyramid attention module and a global-to-local progressive learning module are added to a backbone network ResNet50 to perform feature extraction on few-shot ship images. In the second stage, a spatial improvement attention mechanism is added to perform spatial feature extraction on the feature map extracted in the first stage, to obtain an enhanced feature map, and to fuse the enhanced feature map with the output of the original backbone network to obtain a fused feature. In the third stage, a relationship score of a query set image and a support set image is calculated based on the fused feature, and then a label score is output, and after sorting, the highest score is selected as the output result of ship target identification. The application effectively improves the accuracy of ship re-identification and enhances the generalization ability of the network.

Description

Technical Field

[0001] This invention belongs to the field of ship re-identification technology in intelligent transportation, and in particular to a few-sample ship re-identification method, device, storage medium and computing equipment based on global to local progressive learning. Background Technology

[0002] As maritime transport vehicles, ships undertake the majority of international trade and cargo transportation tasks, providing people with a rich variety of goods and resources to meet their daily material needs. Secondly, in the context of globalization, ships connect countries and regions, promoting cultural, economic, and technological exchanges and cooperation, which is of great significance to social production and economic development. Furthermore, ships play a crucial role in video surveillance. Through ship monitoring systems, the position, status, and safety of ships can be monitored in real time, ensuring navigational safety and preventing accidents, which is essential for maritime traffic management and safety assurance. Vessel re-identification aims to retrieve all images of a given query ship from image data collected by surveillance cameras, and can be treated as a sub-problem of image retrieval. Although some good methods exist for vessel re-identification, it still faces some challenges. Specifically, the challenges can be summarized into three major categories: 1) Ships are affected by environmental conditions such as sea conditions and weather when operating at sea, resulting in high noise in the collected images and low recognition accuracy; 2) Ship re-identification involves the collection, storage, and analysis of a large amount of ship feature information, requiring efficient information processing and recognition algorithms, which in turn requires a large number of samples; 3) Unlike pedestrians and vehicles, ships have differences in shape, size, and appearance, requiring the exploration of solutions to address these issues.

[0003] The ability of ship re-identification networks to learn rapidly from a small number of samples is crucial for further advancing the development of intelligent transportation. However, obtaining high-quality ship images is often difficult due to the following reasons: 1) Environmental limitations: The maritime environment is highly variable, affected by factors such as weather, lighting, and waves, making it difficult to acquire clear and stable ship images; 2) Long distance and mobility: Ships typically move in distant sea areas, making it difficult to approach them, and their own mobility makes maintaining a stable shooting distance and angle challenging; 3) Nighttime and inclement weather: Image clarity and quality deteriorate at night and in inclement weather conditions. Therefore, traditional methods struggle to effectively re-identify ships in these scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide a method, apparatus, storage medium, and computing device for re-identifying ships with fewer samples based on global-to-local progressive learning. Using ResNet50 as the backbone and combining a feature pyramid attention module and a global-to-local progressive learning module, more detailed ship features are extracted, effectively reducing the network's dependence on samples.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a few-sample ship re-identification method based on global-to-local progressive learning, comprising:

[0007] A small number of ship images are acquired and divided into a query set and a support set. The ship images in the query set and the support set are then input into the improved backbone network ResNet50 for feature extraction.

[0008] The feature map extracted by the improved backbone network ResNet50 is passed through the spatial improvement attention module to obtain an enhanced feature map. The enhanced feature map is then fused with the output features of the unimproved backbone network ResNet50 to obtain a fused feature.

[0009] Similarity is calculated based on the fusion features of ship images in the query set and the fusion features of ship images in the support set, and the ship label with the highest similarity is selected as the ship re-identification result.

[0010] Furthermore, the improved backbone network ResNet50 is as follows:

[0011] Construct a feature pyramid attention module and place it after Layer 4 of the deep convolutional layer in the ResNet50 backbone network;

[0012] A global-to-local progressive learning module is constructed and placed after the deep convolutional layers Layer1, Layer2, and Layer3 of the backbone network ResNet50;

[0013] The outputs of the feature pyramid attention module and the three global-to-local progressive learning modules are summed to form the output of the improved backbone network ResNet50.

[0014] The feature pyramid attention module includes four convolutional layer branches of different scales, a global average pooling layer that integrates max pooling, a 1*1 convolutional layer, and an upsampling layer; the four convolutional layer branches of different scales are a 1*1 convolutional layer branch, a 3*3 convolutional layer branch after a 1*1 convolution, a 5*5 convolutional layer branch after a 1*1 convolution, and a 7*7 convolutional layer branch after a 1*1 convolution;

[0015] The global-to-local progressive learning module includes a 1*1 convolution, a sigmoid activation function, a 5*5 convolution, two batch normalization (BN) operations, and average pooling and max pooling operations.

[0016] Furthermore, the step of inputting the query set and support set ship images into the improved backbone network ResNet50 for feature extraction includes:

[0017] Ship images from the support set and query set are input into four convolutional layers (Layer 1, Layer 2, Layer 3, and Layer 4) of the ResNet50 backbone network to generate feature maps at different levels.

[0018] The output of Layer 4 is input into the feature pyramid attention module. After passing through four convolutional layer branches of different scales, feature maps F1, F2, F3, and F4 are obtained. Feature maps F1 and F2 are added together to obtain feature map F. q Adding feature maps F3 and F4 together yields feature map F. v , feature map F q With F v The summation generates a multi-scale local feature map; the output of Layer4 is sequentially subjected to global average pooling with fused max pooling, 1*1 convolution, and upsampling to obtain a feature map, which is then added to the multi-scale local feature map to obtain the global feature map F. a ;

[0019] The output of Layer 3 and the global feature map F output by the feature pyramid attention module are combined. a The input is fed into the global-to-local progressive learning module after Layer 3. After the output of Layer 3 undergoes 5×5 convolution, batch normalization (BN), and average pooling, the output feature map F is produced. w3 Global feature map F a After 1×1 convolution, batch normalization (BN), and max pooling, the sigmoid activation function is used to select key features for output; the selected key features and feature maps F are then compared. w3 Multiply to generate a key information feature map F e3 ;

[0020] The output of Layer2 and the key information feature map F e3 The input is fed into the global-to-local progressive learning module after Layer 2. After the output of Layer 2 is subjected to 5×5 convolution, batch normalization (BN), and average pooling, the output feature map F is obtained. w2 Key information feature map F e3 After 1×1 convolution, batch normalization (BN), and max pooling, the sigmoid activation function is used to select key features for output; the selected key features and feature map F are then analyzed. w2Multiply to generate a key information feature map F e2 ;

[0021] The output of Layer 1 and the key information feature map F e2 The input is fed into the global-to-local progressive learning module after Layer 1. After the output of Layer 1 undergoes 5×5 convolution, batch normalization (BN), and average pooling, the output feature map F is produced. w1 Key information feature map F e2 After 1×1 convolution, batch normalization (BN), and max pooling, the sigmoid activation function is used to select key features for output; the selected key features and feature map F are then analyzed. w1 Multiply to generate a key information feature map F e1 ;

[0022] The global feature map F a Key Information Feature Map F e3 Key Information Feature Map F e2 and key information feature map F e1 Adding them together yields the feature map F. b This is the output of the improved backbone network ResNet50.

[0023] Furthermore, the feature maps extracted from the improved backbone network ResNet50 are processed by a spatially improved attention module to obtain enhanced feature maps, including:

[0024] A spatially improved attention module is constructed, including average pooling layer, max pooling layer, 1*1 convolutional layer, batch normalization layer, ReLU non-linear activation and sigmoid activation function;

[0025] The feature map F extracted from the improved backbone network ResNet50 b After passing through the max pooling layer and the average pooling layer respectively, the data is concatenated according to the channel dimension.

[0026] The concatenated feature map is then dimensionality-reduced using a 1*1 convolutional layer, followed by batch normalization (BN) and ReLU non-linear activation.

[0027] Spatial attention map M is obtained by 1×1 convolution and sigmoid activation function. s ;

[0028] Feature map F b After performing 1×1 convolution and spatial attention map M s The addition yields the enhanced feature map F. c .

[0029] Furthermore, the enhanced feature map is fused with the output features of the unmodified backbone network ResNet50 to obtain fused features, including:

[0030] Enhance feature map F c The fused feature F is obtained by fusing the output of Layer 4, a deep convolutional layer of the ResNet50 backbone network.

[0031] Furthermore, the similarity calculation based on the fusion features of the query set ship images and the fusion features of the support set ship images includes:

[0032] The fused feature F is reduced in size by Layer5 convolution and BN batch normalization. Then, a fully connected layer is used to align and classify the number of ships. The similarity score between the ship images in the query set and the ship images in the support set is calculated based on the feature cosine distance.

[0033] After sorting the similarity scores from high to low, the output with the highest similarity score is taken as the ship re-identification result.

[0034] Furthermore, the method also includes:

[0035] We select the weighted sum of mean squared error loss and triplet loss as the loss function to optimize the parameters of each module.

[0036] Secondly, the present invention provides a few-shot ship re-identification device based on global-to-local progressive learning, used to implement the aforementioned few-shot ship re-identification method based on global-to-local progressive learning, the device comprising:

[0037] The data acquisition module is used to acquire a small number of ship images and divide them into query sets and support sets;

[0038] The feature extraction module is used to input the query set and support set ship images into the improved backbone network ResNet50 for feature extraction;

[0039] The feature fusion module is used to obtain an enhanced feature map by passing the feature map extracted by the improved backbone network ResNet50 through the spatial improvement attention module, and to fuse the enhanced feature map with the output features of the unimproved backbone network ResNet50 to obtain fused features;

[0040] The recognition output module is used to calculate the similarity based on the fusion features of the ship images in the query set and the fusion features of the ship images in the support set, and select the ship label with the highest similarity as the ship re-identification result.

[0041] Thirdly, the present invention provides a computer-readable storage medium for storing one or more programs, said one or more programs including instructions that, when executed by a computing device, cause the computing device to perform any of the methods described above.

[0042] Fourthly, the present invention provides a computing device comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing any of the methods described above.

[0043] The beneficial effects achieved by this invention are as follows:

[0044] This invention proposes a novel few-shot ship re-identification method based on global-to-local progressive learning. By employing a meta-learning strategy, the input image is divided into a support set and a query set. Using ResNet50 as the backbone and combining a feature pyramid attention module and a global-to-local progressive learning module, more detailed ship features are extracted, effectively reducing the network's dependence on samples. Furthermore, by introducing a spatially improved attention mechanism and calculating similarity relationships, the accuracy of ship re-identification is effectively improved, and the network's generalization ability is enhanced. Attached Figure Description

[0045] Figure 1 An architecture for a few-shot ship re-identification method based on global-to-local progressive learning is provided as an embodiment of the present invention;

[0046] Figure 2 This is a Feature Pyramid Attention Module (FPAM) architecture in one embodiment of the present invention;

[0047] Figure 3 This is a global-to-local progressive learning module (GLLM) architecture in one embodiment of the present invention;

[0048] Figure 4 This is a spatially improved attention module (SIAM) architecture in one embodiment of the present invention. Detailed Implementation

[0049] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The embodiments described below with reference to the accompanying drawings are illustrative and intended to explain the present invention, and should not be construed as limiting the present invention.

[0050] It should be understood that the term "an embodiment" or "embodiment" as used in this invention refers to a specific feature, structure, or characteristic that may be included in at least one implementation of this invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that excludes other embodiments.

[0051] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0052] This invention provides a few-shot ship re-identification method based on global-to-local progressive learning, which consists of three stages. The first stage involves adding the Feature Pyramid Attention (FPAM) module and the Global-to-Local Progressive Learning (GLLM) module to the ResNet50 backbone network to extract features from few-shot ship images. The second stage adds the Spatial Improvement Attention (SIAM) mechanism after the GLLM module, fuses its output with the output of the backbone network, and then introduces a metric method to calculate similarity. The third stage involves determining the relationship scores between the images in the query set and the images in the support set, outputting label scores, and then selecting the highest score as the output result for ship target identification.

[0053] Example 1, see Figure 1 This is the first embodiment of the present invention. Based on the above-described inventive concept, this embodiment provides a few-sample ship re-identification method based on global-to-local progressive learning, comprising:

[0054] Acquire a small number of ship images, divide them into query set and support set, and input the ship images of query set and support set into the improved backbone network ResNet50 for feature extraction;

[0055] The feature maps extracted by the improved backbone network ResNet50 are processed through a spatial improvement attention mechanism to obtain enhanced feature maps, and the enhanced feature maps are fused with the output features of the unimproved backbone network ResNet50 to obtain fused features.

[0056] Similarity is calculated based on the fusion features of ship images in the query set and the fusion features of ship images in the support set, and the ship label with the highest similarity is selected as the ship re-identification result.

[0057] See Figure 1 In this embodiment, a small number of ship images are acquired, and a query set and a support set are divided. The ship images of the query set and the support set are then input into the improved backbone network ResNet50 for feature extraction. This process is divided into four steps:

[0058] Step 1: Employing a meta-learning training strategy, the network parameters are updated by updating the synthetic gradients obtained from various tasks in parallel. This approach requires a large number of training tasks, but in the testing phase, only a small number of labels are needed to verify the meta-learner's adaptability to new tasks. Few-shot learning is described as a C-way K-shot problem, where each meta-task contains C classes. K samples are randomly selected from each class, resulting in a support set of C×K samples for the meta-task. Then, a number of samples are randomly selected from the remaining data of these C classes as the prediction targets, i.e., the query set, for this meta-task. This embodiment uses this meta-learning strategy to divide the acquired few-shot ship images into a support set and a query set.

[0059] Step 2: The ship images from the support set and query set are fed into four convolutional layers (Layer 1, Layer 2, Layer 3, and Layer 4) of the ResNet50 backbone network with different numbers of channels to generate feature maps at different levels. The deep features contain high-level semantic information about the ships, while the low-level features contain local details of the images, such as texture and component positions.

[0060] Step 3: Construct the Feature Pyramid Attention Module (FPAM) and place it after the Layer 4 deep convolutional layer of the backbone network. This FPAM consists of four convolutional layer branches of different scales, global average pooling with max pooling, 1*1 convolution, and upsampling operations. The four convolutional layer branches of different scales are a 1*1 convolutional layer branch, a 3*3 convolutional layer branch after a 1*1 convolution, a 5*5 convolutional layer branch after a 1*1 convolution, and a 7*7 convolutional layer branch after a 1*1 convolution.

[0061] Specifically, the output of Layer 4 is processed through four convolutional branches to obtain outputs F1, F2, F3, and F4. The output of Layer 4 is then processed sequentially through global average pooling with max pooling, 1x1 convolution, and upsampling to obtain feature maps. The feature map F is obtained by adding F1 and F2. q The feature map F is obtained by adding F3 and F4. v The summation generates a multi-scale local feature map; then, this multi-scale local feature map is added to the upsampled feature map to obtain the output of the Feature Pyramid Attention Module (FPAM). FPAM can guide the global features to continuously decode, reconstruct, and weight local information.

[0062] Step 4: Construct the Global-to-Local Progressive Learning (GLLM) module, which consists of a 1x1 convolution, a sigmoid activation function, a 5x5 convolution, two batch normalization (BN) operations, and average and max pooling operations.

[0063] Specifically, after placing the global-to-local progressive learning module GLLM into Layer 3, Layer 2, and Layer 1,

[0064] The global-to-local progressive learning module GLLM after Layer 3 takes the output of the feature pyramid attention module FPAM and the output of Layer 3 as input;

[0065] The Global-to-Local Progressive Learning (GLLM) module after Layer 2 takes the output of the previous Global-to-Local Progressive Learning (GLLM) module and the output of Layer 2 as inputs;

[0066] The Global-to-Local Progressive Learning (GLLM) module after Layer 1 takes the output of the previous Global-to-Local Progressive Learning (GLLM) module and the output of Layer 1 as inputs;

[0067] The Global-to-Local Progressive Learning (GLLM) module allows the model to focus on both global and local features simultaneously.

[0068] The feature extraction process of the Global-to-Local Progressive Learning (GLLM) module is as follows:

[0069] The system is divided into two channels. The bottom channel performs 5×5 convolutions on the output features of Layer1 / Layer2 / Layer3, followed by batch normalization (BN) and average pooling before outputting the feature map F. w ;

[0070] The advanced channel passes the input features through a 1×1 convolution with BN and ReLU activations, and then uses a sigmoid activation function to select key features for output after max pooling.

[0071] Finally, the outputs of the high-level channels and the low-level channels are multiplied together to obtain the output of the global-to-local progressive learning module GLLM.

[0072] It should be noted that the input features of the advanced channels are: for Layer 3, the output features of the Feature Pyramid Attention Module (FPAM); for Layer 2 and Layer 1, the output features of the upper-level Global-to-Local Progressive Learning Module (GLLM).

[0073] Step 5: Add the output of the Feature Pyramid Attention Module (FPAM) and the outputs of the three Global-to-Local Progressive Learning Modules (GLLM) to obtain the output of the improved backbone network ResNet50.

[0074] See Figure 1 In this embodiment, the feature map extracted by the improved backbone network ResNet50 is processed by a spatial improvement attention module to obtain an enhanced feature map. The enhanced feature map is then fused with the output features of the unimproved backbone network ResNet50 to obtain a fused feature map. This process involves two steps:

[0075] Step 1: Construct the Spatial Improvement Attention Module (SIAM), which consists of average pooling, max pooling, 1*1 convolutions, batch normalization (BN) layers, ReLU non-linear activation, and a sigmoid activation function.

[0076] The feature extraction process is as follows: the feature maps extracted by the improved backbone network ResNet50 are passed through a max pooling layer f. mp and average pooling layer f apNext, the feature map dataset is concatenated according to the channel dimension using ||c. Then, after dimensionality reduction through a Conv1×1 convolutional layer, batch normalization (BN) and ReLU non-linear activation are used to accelerate the network training process and enhance the network's expressive power to generate feature maps. Finally, 1×1 convolutions are used to increase the dimensionality, and then combined with the sigmoid activation function to obtain the final attention vector, thereby generating a spatial attention map.

[0077] SIAM can generate a spatial attention map that highlights important areas, guiding the network to focus on important ship features.

[0078] Step 2: Add the generated spatial attention map to the output of the GLLM after 1×1 convolution to obtain a spatial attention weighted feature map;

[0079] The spatial attention-weighted feature map is then fused with the original feature map output by Layer4 to obtain the fused features.

[0080] See Figure 1 In this embodiment, the similarity between the features fused from the query set and the features fused from the support set is calculated, and the highest similarity is taken as the ship re-identification result. This is specifically divided into the following three steps:

[0081] Step 1: The fused output features are reduced in size by passing them through a Layer 5 convolutional layer and a batch normalization (BN) layer. Then, a fully connected (FC) layer is used to align and classify the number of ships. It should be noted that Layer 5 has a structure similar to Layer 1 / 2 / 3 / 4 of a backbone network.

[0082] Step Two: Regarding the loss function in re-identification, this embodiment chooses a weighted combination of mean squared error (MSE) loss and triplet loss. For support samples and predicted samples of the same class, the relationship between them is assumed to be 1; for support samples and predicted samples of different classes, the relationship is assumed to be 0. In this case, the relationship between support samples and predicted samples can be considered a regression problem, which allows the use of mean squared error (MSE). MSE, as a loss function, can be used to evaluate the model's predictive ability by calculating the average of the squared errors between the predicted and actual values. The smaller the prediction error, the smaller the MSE value, indicating a better fit of the model to the data. Gradient descent and other optimization algorithms can also be used to update the model parameters. Triplet loss ensures that ship images with the same identity are as close as possible in the feature space, while ship images with different identities are as far apart as possible. This improves the accuracy of ship identification and enhances the model's generalization ability and robustness in real-world scenarios. Therefore, this embodiment uses a combination of mean squared error and triplet loss as the loss function.

[0083] Step 3: Calculate the similarity relationship score between the ship image features in the query set and the ship image features in the support set, output the label score, and sort them from high to low to obtain the output result with the highest similarity.

[0084] Based on this embodiment, using ResNet50 as the backbone and combining a feature pyramid attention module and a global-to-local progressive learning module, more detailed ship features are extracted, effectively reducing the network's dependence on samples. Furthermore, by introducing a spatially improved attention mechanism and calculating similarity relationships, the accuracy of ship re-identification is effectively improved, enhancing the network's generalization ability.

[0085] Example 2: The second embodiment of the present invention uses the method of Example 1 for few-sample ship re-identification. The specific implementation process is as follows:

[0086] Phase 1: Feature extraction for a small sample of ships, as follows:

[0087] Step 1: Using a meta-learning strategy, the input few-sample ship images are divided into a support set and a query set, where elements in the support set are denoted by x. i,k This indicates that the number of samples in each category of the support set is K, x i,k This represents the k-th sample of the i-th category; elements in the query set are represented by x. j express.

[0088] Step 2: Input the ship images from the support set and query set into four convolutional layers (Layer 1, Layer 2, Layer 3, and Layer 4) of the ResNet50 backbone network at different depths to generate feature maps at different levels to complement each other. Lower-level location features have smaller receptive fields and weaker expressive power, while higher-level location features have larger receptive fields, stronger expressive power, and can extract more global contextual information.

[0089] Step 3: Compared to traditional feature extraction networks, the Feature Pyramid Attention (FPA) module can effectively increase the global feature perception field and extract key global feature information at different scales. It is more suitable for natural and complex background environments without introducing too much computation.

[0090] Construct a Feature Pyramid Attention Module (FPAM).

[0091] It should be noted that the existing Feature Pyramid Attention (FPA) module can fuse pyramid features of three different scales extracted from a U-shaped network. It uses 3×3, 5×5, and 7×7 convolutional kernels to better extract contextual information from pyramid features at different scales. Then, after 1×1 convolution processing, the original features extracted by the CNN are multiplied pixel-by-pixel through the pyramid attention features. Furthermore, a global pooling branch is introduced to connect the output features, which further improves the performance of the FPA module. Unlike FPA, the FPAM constructed in this embodiment only utilizes the superposition of 1*1, 3*3, 5*5, and 7*7 convolutional kernel branches of different scales to extract features, which greatly reduces the computational burden. Here, the feature map F is first obtained by adding the 1*1 convolutional branch and the 3*3 convolutional branch. q The feature map F is obtained by adding the 5x5 convolutional branch and the 7x7 convolutional branch. v Finally, F q With F v The final multi-scale local feature maps are obtained by superimposing the data. This choice is made because the 1x1 convolutional branch can be used for feature fusion and dimensionality reduction between channels, helping to preserve and enhance some small-scale feature representations, thus reducing the number of model parameters, computational complexity, and memory consumption. The 3x3 convolutional branch, on the other hand, has greater spatial feature extraction capabilities and is more suitable for capturing local details. By adding them together, features from different scales and channels can be fully fused while maintaining a relatively small number of parameters. The 5x5 and 7x7 convolutional branches can capture a wider range of local features, which is also crucial for ship re-identification tasks. Since the shape and structure of a ship may span a large spatial area, larger convolutional kernels are better able to capture detailed features in the image, such as the ship's texture and structure. Adding the 5x5 and 7x7 convolutional branches makes the model more sensitive to these details, helping to improve recognition accuracy. Meanwhile, in the subsequent branches at different scales, 1*1 convolution was first used for dimensionality reduction. Batch normalization (BN) and ReLU activation function were also used to improve training stability and non-linear expression ability. By superimposing the features extracted from the above four branches, local features can be aggregated at different scales, retaining more detailed information.

[0092] In addition, its global pooling branch is retained. After global average pooling, 1*1 convolution, and upsampling, the features are added again with the superimposed features of convolution kernels of different scales, thus outputting the final global feature map F. aHere, because average pooling treats each pixel of the feature map equally, the summation method is used to integrate max pooling into global pooling. This combines the advantages of both methods, not only extracting the overall features of the entire feature map but also capturing important regions, thus focusing more on the important features of the ship.

[0093] like Figure 2 As shown, the Feature Pyramid Attention (FPAM) module constructed in this embodiment is added to Layer 4 of the backbone network. FPAM fuses features at four pyramid scales with different contexts. The four pyramid kernel convolutional branches are a 1*1 convolutional layer branch, a 3*3 convolutional layer branch after a 1*1 convolution, a 5*5 convolutional layer branch after a 1*1 convolution, and a 7*7 convolutional layer branch after a 1*1 convolution. Let the outputs after these four different scale convolutional branches be F1, F2, F3, and F4, respectively, and the calculation formulas are as follows:

[0094] F1=RBN(Conv1*1(Layer4)) (1)

[0095] F2=RBN(Conv3*3(Conv1*1(Layer4))) (2)

[0096] F3=RBN(Conv5*5(Conv1*1(Layer4))) (3)

[0097] F4=RBN(Conv7*7(Conv1*1(Layer4))) (4)

[0098] Here, RBN represents the Rectified Linear Unit (ReLU) activation function and the Batch Normalization (BN) operation. Through all these operations, the output local features have a pyramid structure that progressively integrates contextual information and smooths features at different scales.

[0099] Let the feature map obtained by adding F1 and F2 be F. q The feature map resulting from the sum of F3 and F4 is set as F. v F q and F v The summation generates a multi-scale local feature map.

[0100] The lower half of the FPAM architecture (global pooling, max pooling, 1x1 convolution, upsampling) is used to sum the multi-scale feature maps to obtain the global feature map F. a The calculation formula is as follows:

[0101] F a = UP(Conv1*1(GAP(Layer4)+MaxP(Layer4))) + RBN(F q+F v (5)

[0102] UP is an upsampling operation using nearest neighbor interpolation, GAP is global average pooling, and MaxP is max pooling.

[0103] Step 4: Construct the Global-to-Local Progressive Learning Module (GLLM) Figure 3 As shown, after placing it in Layer3, Layer2, and Layer1,

[0104] In each GLLM,

[0105] The low-level channel features (Layer 1 / Layer 2 / Layer 3 output features) are convolved with 5×5, and then subjected to batch normalization and average pooling to obtain the feature map F. w As shown in equation (6);

[0106] Advanced channel features (FPAM output features / upper-layer GLLN output features) are processed by 1×1 convolution under BN (batch normalization) and ReLU activation, max pooling, and then the sigmoid activation function is used to select key features for output.

[0107] Connect key feature values ​​with the underlying feature map F w Multiply to generate a key information feature map F e As shown in equation (7),

[0108] F w =AvgP(RBN(Conv5*5(Layer))) (6)

[0109] F e =σ(MaxP(RBN(Conv1*1(F)) r )))) ◎ F w (7)

[0110] Where AvgP() represents average pooling, MaxP() represents max pooling, σ represents the Sigmoid activation function, and ◎ represents multiplication.

[0111] This allows high-level global feature information to be used to gradually guide low-level features in learning discriminative local detail features of the target region, thus increasing the ship's detailed feature information. The final output F... b The specific calculation formula is as follows:

[0112] F b = F e + RBN(UP((F r ))) (8)

[0113] F r This indicates advanced channel characteristics.

[0114] The specific operations of each global-to-local progressive learning module GLLM are as follows:

[0115] The output of Layer 3 and the global feature map F output by the feature pyramid attention module are combined. a The input is fed into the global-to-local progressive learning module after Layer 3. The output of Layer 3 undergoes 5×5 convolution, batch normalization (BN), and average pooling operations to output the feature map F. w3 Global feature map F a After 1×1 convolution, batch normalization (BN), and max pooling, the sigmoid activation function is used to select key features for output; the selected key features and feature maps F are then compared. w3 Multiply to generate a key information feature map F e3 ;

[0116] The output of Layer2 and the key information feature map F e3 The input is fed into the global-to-local progressive learning module after Layer 2. The output of Layer 2 undergoes 5×5 convolution, batch normalization (BN), and average pooling operations to output the feature map F. w2 Key information feature map F e3 After 1×1 convolution, batch normalization, and max pooling, the sigmoid activation function is used to select key features for output; the selected key features and feature map F are then analyzed. w2 Multiply to generate a key information feature map F e2 ;

[0117] The output of Layer 1 and the key information feature map F e2 The input is fed into the global-to-local progressive learning module after Layer 1. The output of Layer 1 undergoes 5×5 convolution, batch normalization (BN), and average pooling operations, resulting in the output feature map F. w1 Key information feature map F e2 After 1×1 convolution, batch normalization, and max pooling, the sigmoid activation function is used to select key features for output; the selected key features and feature map F are then analyzed. w1 Multiply to generate a key information feature map F e1 ;

[0118] The global feature map F a Key Information Feature Map F e3 Key Information Feature Map F e2 and key information feature map F e1 Adding them together yields the feature map F. b This is the output of the improved backbone network ResNet50.

[0119] Phase Two: Constructing Spatial Improvement Attention Mechanisms (SIAM) Figure 4 The output feature map from the first stage is then processed by SIAM for feature extraction and fused with the output of the backbone network to obtain the fused features, as detailed below:

[0120] Step 1: Convert the output F of the first stage b Each through the max pooling layer f mp and average pooling layer f ap Next, feature maps are concatenated using ||c according to the channel dimension; then, after dimensionality reduction using a 1×1 convolutional layer, batch normalization (BN) and ReLU non-linear activation are used to accelerate the network training process and enhance the network's expressive power. Finally, a 1×1 convolutional dimension increase operation is performed, followed by a sigmoid activation function to ensure that the channel attention weights are between 0 and 1, thereby generating a spatial attention map M. s Finally, spatial attention diagram M s The feature map F after 1×1 convolution b The addition yields the enhanced feature map F. c M s The specific calculations are as follows:

[0121] M s = σ (Conv1*1(RBN(Conv1*1(MaxP(F b ) ||c AvgP(F b ))))) (9)

[0122] In this step, by combining max pooling and average pooling feature maps, the network achieves a more comprehensive representation, thus better distinguishing ships. Secondly, SIAM utilizes convolutional layers and activation functions to identify regions crucial for ship re-identification. By adding these key regions to the original feature maps, the network can focus on essential features in a targeted manner. Therefore, SIAM achieves a balanced integration of global and local information by optimizing pooling fusion and utilizing generated spatial attention, emphasizing key regions and enhancing the network's ability to distinguish ships.

[0123] Step 2: Convert the output F of GLLM c The feature map is fused with the feature map output from Layer 4 for subsequent metric calculation, as shown below:

[0124] F = Cat(F4, F c (10)

[0125] Where F represents the features used for measurement after fusion, F4 represents the features extracted from Layer4, and Cat(:,:) is the fusion operator.

[0126] The third stage: Based on the fused features, the similarity scores between the query set image and each image in the support set are obtained, sorted from high to low, and then the target ship to be queried is identified, as follows:

[0127] Step 1: The fused output features are reduced in size by passing them through Layer 5 and Batch Normalization (BN) layers, and then the number of ships is aligned and classified using a fully connected (FC) layer. The feature cosine distance metric between the query sample and each class prototype of the given support set samples is output as shown in Equation (11):

[0128]

[0129] Among them, S i,j x represents i,k and x j The characteristic cosine distance; Used for relation calculations; K represents the number of samples contained in each category of the support set; Responsible for extracting features from sample images and generating feature representations for the support set and query set samples; x i,k x represents the k-th sample of the i-th category; j The query indicates an element in the image set; || indicates feature concatenation.

[0130] Step 2: After measuring the feature cosine distance between the support set and the query set, calculate the error between the model output and the real label based on the loss function, and adjust the model parameters through the backpropagation algorithm so that the model can better distinguish different ships and improve the accuracy of re-identification.

[0131] Because the output similarity score is not a fixed 0 or 1 value, but rather a value between 0 and 1, it can be viewed as a regression problem. In this embodiment, the mean squared error (MSE) is chosen as the loss function. It evaluates the model's predictive ability by calculating the average of the squared errors between the predicted and actual values. The smaller the prediction error, the smaller the MSE value, indicating a better fit of the model to the data. The specific formula for calculating the mean squared error is shown below:

[0132]

[0133]

[0134] Loss MSE1 and Loss MSE2 This represents the mean squared error under two different scenarios. Where C is the number of ship categories in the support set, and Q is the number of query samples. i and y j These are the label values ​​for the supporting sample and the query sample, respectively. i =yj It is 1 if it is true, otherwise it is 0.

[0135] Regarding triplet loss, by introducing class center loss and adaptive sample mining strategy, the convergence speed and performance of the model during training can be improved, thereby improving the accuracy of the ship identification model.

[0136] The specific calculation formula is as follows:

[0137] Loss Triplet =∑max(0,||f(a)–f(p)||^2-||f(a)-f(n)||^2+margin)+β*D(c(a),c(n)) (14)

[0138] Where max(0, ||f(a)–f(p)||^2-||f(a)-f(n)||^2+margin)) is the loss term for each triple. It consists of three parts: ||f(a)–f(p)||^2 represents the squared feature representation distance between the anchor sample a and the positive sample p; ||f(a)-f(n)||^2 represents the squared feature representation distance between the anchor sample a and the negative sample n; and margin is a boundary value that represents the minimum distance difference between positive and negative samples, used to ensure effective separation between samples.

[0139] β*D(c(a),c(n)) is the class center loss term. It consists of two parts: D(c(a),c(n)): representing the distance between the class center c(a) of the anchor sample a and the class center c(n) of the negative sample n, used to measure the distribution of samples in the class space. β is the weight used to balance the triplet loss and the class center loss, which can be used to specify the degree of contribution of the two losses to the total loss.

[0140] The goal of this loss function is to effectively drive the model to learn better feature representations by minimizing the distance between the anchor point and the positive example, while maximizing the distance between the anchor point and the negative example. Simultaneously, the class center loss term enables effective clustering and distribution learning of samples in the class space, thereby improving the model's ability to distinguish between samples of different classes.

[0141] Based on the above overview of mean squared error loss and triplet loss, combining these two loss functions allows the model to consider both the relative relationships between samples and the class information of the samples when learning feature representations. The specific formula for combining them is as follows:

[0142] Loss Total =λ1Loss MSE +λ2Loss Triplet (15)

[0143] Here, λ1 and λ2 represent the weights of two different losses.

[0144] This combined approach helps improve the model's ability to distinguish between different ship categories, while also increasing the similarity between samples of the same type and the difference between samples of different types.

[0145] Step 3: Calculate the similarity relationship score between the ship image features in the query set and the ship image features in the support set, output the label score, and sort them from high to low to obtain the output result with the highest similarity.

[0146] Based on the above inventive concept, the present invention also provides a few-shot ship re-identification device based on global-to-local progressive learning, used to implement the few-shot ship re-identification method based on global-to-local progressive learning in Embodiment 1 or Embodiment 2 above. The device includes:

[0147] The data acquisition module is used to acquire a small number of ship images and divide them into query sets and support sets;

[0148] The feature extraction module is used to input the query set and support set ship images into the improved backbone network ResNet50 for feature extraction;

[0149] The feature fusion module is used to obtain an enhanced feature map by passing the feature map extracted by the improved backbone network ResNet50 through the spatial improvement attention module, and to fuse the enhanced feature map with the output features of the unimproved backbone network ResNet50 to obtain fused features;

[0150] The recognition output module is used to calculate the similarity based on the fusion features of the ship images in the query set and the fusion features of the ship images in the support set, and select the ship label with the highest similarity as the ship re-identification result.

[0151] It is worth noting that this device embodiment corresponds to the above method embodiment. The implementation methods of the above method embodiments are all applicable to this device embodiment and can achieve the same or similar technical effects, so they will not be described in detail here.

[0152] On the other hand, the present invention also provides a computer-readable storage medium for storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform the few-sample ship re-identification method based on global-to-local progressive learning described in Embodiment 1 or Embodiment 2 above.

[0153] On the other hand, the present invention also provides a computing device, including one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing the few-sample ship re-identification method based on global-to-local progressive learning according to Embodiment 1 or Embodiment 2 described above.

[0154] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0155] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0156] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0157] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A few-shot ship re-identification method based on global-to-local progressive learning, characterized in that, include: A small number of ship images are acquired and divided into a query set and a support set. The ship images in the query set and support set are then input into an improved backbone network, ResNet50, for feature extraction. The improved backbone network, ResNet50, is as follows: Construct a feature pyramid attention module and place it after Layer 4 of the deep convolutional layer in the ResNet50 backbone network; A global-to-local progressive learning module is constructed and placed after the deep convolutional layers Layer1, Layer2, and Layer3 of the backbone network ResNet50; The outputs of the feature pyramid attention module and the three global-to-local progressive learning modules are summed to form the output of the improved backbone network ResNet50. The feature pyramid attention module includes four convolutional layer branches of different scales, a global average pooling layer that integrates max pooling, a 1×1 convolutional layer, and an upsampling layer; the four convolutional layer branches of different scales are a 1×1 convolutional layer branch, a 3×3 convolutional layer branch after a 1×1 convolution, a 5×5 convolutional layer branch after a 1×1 convolution, and a 7×7 convolutional layer branch after a 1×1 convolution; The global-to-local progressive learning module includes a 1×1 convolution, a sigmoid activation function, a 5×5 convolution, two batch normalization (BN) operations, and average pooling and max pooling operations. The feature maps extracted by the improved ResNet50 backbone are processed by a spatially improved attention module to obtain enhanced feature maps, including: A spatially improved attention module is constructed, including average pooling layer, max pooling layer, 1×1 convolutional layer, batch normalization layer, ReLU non-linear activation and sigmoid activation function; Feature maps extracted from the improved backbone network ResNet50 After passing through the max pooling layer and the average pooling layer respectively, the data is concatenated according to the channel dimension. The concatenated feature map is reduced in dimensionality by a 1×1 convolutional layer, and then subjected to batch normalization (BN) and ReLU non-linear activation. Spatial attention map obtained by 1×1 convolution and sigmoid activation function ; feature map After performing 1×1 convolution and spatial attention graph Adding them together yields the enhanced feature map. ; The enhanced feature map is fused with the output features of the unmodified backbone network ResNet50 to obtain fused features, including: Enhance feature maps The fused feature F is obtained by fusing the output of Layer 4, a deep convolutional layer of the ResNet50 backbone network. Similarity is calculated based on the fusion features of ship images in the query set and the fusion features of ship images in the support set, and the ship label with the highest similarity is selected as the ship re-identification result.

2. The few-shot ship re-identification method based on global-to-local progressive learning according to claim 1, characterized in that, The step of inputting the query set and support set ship images into the improved backbone network ResNet50 for feature extraction includes: Ship images from the support set and query set are input into four convolutional layers (Layer 1, Layer 2, Layer 3, and Layer 4) of the ResNet50 backbone network to generate feature maps at different levels. The output of Layer 4 is input into the feature pyramid attention module, and the feature map is obtained after passing through four convolutional layer branches of different scales. , , and , feature map and Adding them together yields the feature map. , feature map and Adding them together yields the feature map. , feature map and The summation generates a multi-scale local feature map; the output of Layer4 is then subjected to global average pooling with fused max pooling, 1*1 convolution, and upsampling to obtain a feature map, which is then added to the multi-scale local feature map to obtain the global feature map. ; The output of Layer 3 and the global feature map output by the feature pyramid attention module are combined. The input is fed into the global-to-local progressive learning module after Layer 3. The output of Layer 3 undergoes 5 × 5 convolution, batch normalization (BN), and average pooling operations to output the feature map. Global feature map After 1 × 1 convolution, batch normalization (BN), and max pooling, the sigmoid activation function is used to select key features for output; the selected key features and feature maps are then compared. Multiply to generate a key information feature map. ; The output of Layer2 and key information feature maps The input is fed into the global-to-local progressive learning module after Layer 2. The output of Layer 2 is then subjected to 5 × 5 convolution, batch normalization (BN), and average pooling to output the feature map. Key information feature map After 1 × 1 convolution, batch normalization (BN), and max pooling, the sigmoid activation function is used to select key features for output; the selected key features and feature maps are then analyzed. Multiply to generate a key information feature map. ; The output of Layer 1 and key information feature maps The input is fed into the global-to-local progressive learning module after Layer 1. The output of Layer 1 is then subjected to 5 × 5 convolution, batch normalization (BN), and average pooling to output the feature map. Key information feature map After 1 × 1 convolution, batch normalization (BN), and max pooling, the sigmoid activation function is used to select key features for output; the selected key features and feature maps are then analyzed. Multiply to generate a key information feature map. ; global feature map Key information feature map Key information feature map and key information feature map Add them together to obtain the feature map. This is the output of the improved backbone network ResNet50.

3. The few-sample ship re-identification method based on global-to-local progressive learning according to claim 2, characterized in that, The similarity calculation based on the fusion features of ship images from the query set and the fusion features of ship images from the support set includes: The fused feature F is reduced in size by Layer5 convolution and BN batch normalization. Then, a fully connected layer is used to align and classify the number of ships. The similarity score between the ship images in the query set and the ship images in the support set is calculated based on the feature cosine distance. After sorting the similarity scores from high to low, the output with the highest similarity score is taken as the ship re-identification result.

4. The few-shot ship re-identification method based on global-to-local progressive learning according to claim 3, characterized in that, The method further includes: We select the weighted sum of mean squared error loss and triplet loss as the loss function to optimize the parameters of each module.

5. A few-sample ship re-identification device based on global-to-local progressive learning, characterized in that, The apparatus for implementing the few-sample ship re-identification method based on global-to-local progressive learning as described in any one of claims 1 to 4 includes: The data acquisition module is used to acquire a small number of ship images and divide them into query sets and support sets; The feature extraction module is used to input the query set and support set ship images into the improved backbone network ResNet50 for feature extraction; The feature fusion module is used to obtain an enhanced feature map by passing the feature map extracted by the improved backbone network ResNet50 through the spatial improvement attention module, and to fuse the enhanced feature map with the output features of the unimproved backbone network ResNet50 to obtain fused features; The recognition output module is used to calculate the similarity based on the fusion features of the ship images in the query set and the fusion features of the ship images in the support set, and select the ship label with the highest similarity as the ship re-identification result.

6. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods according to claims 1 to 4.

7. A computing device, characterized in that, include, One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods according to claims 1 to 4.

Images