Ship detection method based on remote sensing image

Abstract

The invention belongs to the field of remote sensing image processing, and particularly relates to a ship detection method based on a remote sensing image. The invention provides a ship detection method based on a remote sensing image, and aims to solve the problem of low recognition accuracy in a ship model detection task through the existing remote sensing image. The method comprises the following steps: constructing a ship detection model based on a remote sensing image for a fine-grained target identification task of a marine ship; inputting the ship remote sensing image of the to-be-identified area into the trained ship detection model based on the remote sensing image to obtain the ship model of the ship of the to-be-identified area; according to the method, a fine-grained target detection task is decoupled into a positioning sub-task and a classification sub-task, a CycleGAN-based data expansion strategy and a contrast learning module are designed for the problem of shortage of marine ship training samples, and the recognition precision is improved. The problem of poor recognition precision caused by high sight distance, small target, less training data and the like in the existing fine-grained target recognition technology of the remote sensing image is solved.

Description

Technical Field

[0001] The invention belongs to the field of remote sensing image processing, and in particular relates to a ship detection method based on remote sensing images. Background Art

[0002] Remote sensing image processing for ships at sea can identify and monitor ships on the sea, and analyze the type, size, and course of the ships, which is very important for maritime traffic management, fishery management, and rational use of marine resources.

[0003] Current object detectors for marine vessels are not tuned for fine-grained objects. In remote sensing images, objects can only be classified from a bird’s-eye view, and are more difficult to identify at high viewing distances because of the lack of texture features. For example, DOTA, the largest OBB-annotated remote sensing dataset, contains 15 categories. The appearance of these categories varies significantly. Even in remote sensing images, it is not difficult to distinguish between different categories (such as bridges and airplanes).

[0004] However, in the HRSC2016 dataset, specific ship types need to be identified and distinguished (such as cargo ships and medical ships), and there are few effective features that can be used to distinguish between the top views of different types of ships. In this case, the current general object detection methods perform much better in identifying categories with large differences such as ships and cars than in identifying specific ship types.

[0005] At present, remote sensing fine-grained target detection for marine ships faces many challenges. First of all, there are very few open-source fine-grained ship remote sensing datasets. Most remote sensing datasets only contain coarse-grained identification objects such as ships, but do not contain fine-grained labels for specific ship models. For example, the geographic remote sensing dataset NWPUVHR-10 contains only the coarse-grained label of ships among the 10 categories. The HRSC2016 dataset also adds fine-grained labels for specific ship models in addition to the coarse-grained label of ships. However, among all the pictures, there are only 617 pictures with fine-grained labels. Compared with other fine-grained remote sensing datasets, there are fewer training samples, and using ordinary target recognition models will result in serious overfitting.

[0006] In the field of remote sensing, object detection can be mainly divided into two categories: two-stage and single-stage object detection methods. High-performance two-stage object detection methods usually follow the RCNN architecture, which consists of a region proposal network and a region CNN detection head. In recent years, many variants of the RCNN framework have emerged, such as Oriented RCNN, which introduces a box encoding system to solve the problem of unstable training loss. A representative single-stage object detection method is YOLOv5, and the YOLO architecture has also been widely used in the field of remote sensing. For example, BiFA-YOLO uses bidirectional feature fusion and angle classification to detect objects in different directions.

[0007] Image augmentation has been shown to improve the accuracy of fine-grained image recognition and object detection tasks. Image augmentation includes traditional operations such as rotation, cropping, and exposure adjustment, as well as more advanced strategies such as image blending and deletion, which provide greater diversity compared to traditional methods. Representative blending strategies include AlignMixUp, where two images are aligned and interpolated in feature space, and TokenMixup, which best matches images in the same mini-batch to increase important information in the output contrast.

[0008] Currently, commonly used target recognition usually recognizes the entire object, while fine-grained target recognition focuses more on the detailed features inside the object to distinguish different subcategories in the same category. Fine-grained target recognition for remote sensing images faces huge challenges in accurately identifying the category of the target while ensuring a high recall rate due to reasons such as high viewing distance, small targets, and small training data. At the same time, image enhancement for fine-grained target recognition is basically an operation on the image itself. Even if a hybrid strategy is added, no new ship features are introduced, which may not be enough to create representative diversity and make it difficult to capture complex transformations in actual scenes, further causing the model to overfit on certain specific changes and not adapt well to actual data.

[0009] In summary, the existing fine-grained target recognition technology for remote sensing images has the problem of poor recognition accuracy due to high viewing distance, small targets, and small training data. Summary of the invention

[0010] Aiming at the problem of low recognition accuracy in the existing remote sensing image ship model detection task, the present invention proposes a ship detection method based on remote sensing images. The specific process includes:

[0011] S1: Construct a ship detection model based on remote sensing images; obtain a trained ship detection model based on remote sensing images;

[0012] S2: Collect remote sensing images of ships in the area to be identified, input the remote sensing images of ships in the area to be identified into a trained ship detection model based on remote sensing images, and obtain the ship models of ships in the area to be identified;

[0013] In S1, a ship detection model based on remote sensing images is constructed; a trained ship detection model based on remote sensing images is obtained; the specific process is:

[0014] S1.1: Acquire ship remote sensing image sets;

[0015] S1.2: Construct a ship detection model based on remote sensing images; train the ship detection model based on remote sensing images according to the ship remote sensing image set to obtain a trained ship detection model based on remote sensing images.

[0016] The ship remote sensing image set in S1.1 is the HRSC2016 remote sensing image ship dataset;

[0017] The remote sensing image-based ship detection model in S1.2 includes: a data enhancement module, a target detection module, and a fine-grained image classification module;

[0018] In S1.2, the ship detection model based on remote sensing images is trained according to the ship remote sensing image set to obtain a trained ship detection model based on remote sensing images. The specific process is as follows:

[0019] S1.2.1: Train the data enhancement module according to the ship remote sensing image set to obtain a trained data enhancement module;

[0020] Then, the images of the ship remote sensing image set are input into the trained data enhancement module to obtain an enhanced ship remote sensing image set;

[0021] S1.2.2: training the target detection module according to the enhanced ship remote sensing image set to obtain a trained target detection module;

[0022] Then the enhanced ship remote sensing image set is input into the trained target detection module; the ship object and the position of the ship object in the enhanced ship remote sensing image are obtained;

[0023] The position of the ship object is the coordinates of the square range where the ship object is located in the picture;

[0024] S1.2.3: cutting all the ship objects in the enhanced ship remote sensing image according to the positions of the ship objects to obtain a cut ship remote sensing image set;

[0025] Each cut picture in the cut ship remote sensing image set contains only one ship;

[0026] The fine-grained image classification module is trained according to the cut ship remote sensing image set to obtain a trained fine-grained image classification module;

[0027] The cut ship remote sensing image set is input into the trained fine-grained image classification module to obtain the specific model of each ship;

[0028] S1.2.4: Combine the trained data enhancement module, the trained target detection module and the trained fine-grained image classification module to obtain a trained remote sensing image-based ship detection model.

[0029] The inventive concept of the present invention is briefly introduced in combination with the above steps, that is, firstly, the target detection model based on remote sensing images is used to detect the target of ships in the input picture to obtain the position coordinates of the ships, and the original picture is cut using the position coordinates so that each of the cut pictures contains only one ship, and the cut pictures are sent to the ship classification model based on remote sensing images to classify the ships in the pictures;

[0030] The beneficial effects of the present invention are:

[0031] The present invention designs a set of architectures for the fine-grained target recognition task of marine ships, decouples the fine-grained target detection task into positioning subtasks and classification subtasks, and designs an effective data expansion strategy and contrastive learning module based on CycleGAN to address the problem of shortage of marine ship training samples, which greatly improves the recognition accuracy. It achieves good results on the data set and solves the problem of poor recognition accuracy in the existing fine-grained target recognition technology of remote sensing images due to high viewing distance, small targets, and small training data. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a schematic diagram of a fine-grained remote sensing ship detection example;

[0033] Figure 2 This is a schematic diagram of the framework of a small sample fine-grained remote sensing ship detection system;

[0034] Figure 3 Schematic diagram of image processing results during data enhancement. DETAILED DESCRIPTION

[0035] Specific implementation method 1: Combination Figure 1 The invention is described, comprising:

[0036] S1: Construct a ship detection model based on remote sensing images; obtain a trained ship detection model based on remote sensing images;

[0037] S2: Collect remote sensing images of ships in the area to be identified, input the remote sensing images of ships in the area to be identified into a trained ship detection model based on remote sensing images, and obtain the ship models of ships in the area to be identified;

[0038] In S1, a ship detection model based on remote sensing images is constructed; a trained ship detection model based on remote sensing images is obtained; the specific process is:

[0039] S1.1: Acquire ship remote sensing image sets;

[0040] S1.2: Construct a ship detection model based on remote sensing images; train the ship detection model based on remote sensing images according to the ship remote sensing image set to obtain a trained ship detection model based on remote sensing images.

[0041] Specific implementation method 2: The difference between this implementation method and specific implementation method 1 is that:

[0042] The ship remote sensing image set in S1.1 is the HRSC2016 remote sensing image ship dataset;

[0043] The ship detection model of the remote sensing image in S1.2 includes: a data enhancement module, a target detection module, and a fine-grained image classification module.

[0044] The ship detection model based on remote sensing images is trained according to the ship remote sensing image set to obtain a trained ship detection model based on remote sensing images. The specific process is as follows:

[0045] S1.2.1: Train the data enhancement module according to the ship remote sensing image set to obtain a trained data enhancement module;

[0046] Then, the images of the ship remote sensing image set are input into the trained data enhancement module to obtain an enhanced ship remote sensing image set;

[0047] S1.2.2: training the target detection module according to the enhanced ship remote sensing image set to obtain a trained target detection module;

[0048] Then the enhanced ship remote sensing image set is input into the trained target detection module; the ship object and the position of the ship object in the enhanced ship remote sensing image are obtained;

[0049] The position of the ship object is the coordinates of the square range where the ship object is located in the picture;

[0050] S1.2.3: cutting all the ship objects in the enhanced ship remote sensing image according to the positions of the ship objects to obtain a cut ship remote sensing image set;

[0051] Each cut picture in the cut ship remote sensing image set contains only one ship;

[0052] The fine-grained image classification module is trained according to the cut ship remote sensing image set to obtain a trained fine-grained image classification module;

[0053] The cut ship remote sensing image set is input into the trained fine-grained image classification module to obtain the specific model of each ship;

[0054] S1.2.4: Combine the trained data enhancement module, the trained target detection module and the trained fine-grained image classification module to obtain a trained remote sensing image-based ship detection model.

[0055] The inventive concept of the present invention is briefly introduced in combination with the above steps, that is, firstly, the target detection model based on remote sensing images is used to detect the target of ships in the input picture to obtain the position coordinates of the ships, and the original picture is cut using the position coordinates so that each of the cut pictures contains only one ship, and the cut pictures are sent to the ship classification model based on remote sensing images to classify the ships in the pictures;

[0056] The other steps and parameters are the same as those in the first specific implementation.

[0057] Specific implementation method three: This implementation method is different from the specific implementation method one in that:

[0058] The data enhancement module in S1.2.1 includes an image segmentation network and an image style conversion network;

[0059] In S1.2.1, the data enhancement module is trained according to the ship remote sensing image set to obtain a trained data enhancement module; the specific process is:

[0060] S1.2.1.1: Input each ship remote sensing image in the ship remote sensing image set into the image segmentation network in the data enhancement module for extraction processing to obtain a mask of each ship remote sensing image in the ship remote sensing image set;

[0061] S1.2.1.2: Based on the mask of each ship remote sensing image extracted in S1.2.1.1, ship entity extraction processing is performed on each ship image in the ship remote sensing image set to obtain a ship entity image of each ship remote sensing image and a background image of each ship remote sensing image;

[0062] The ship entity images of all ship remote sensing images are formed into an entity image set; the background images of all ship remote sensing images are formed into a background image set;

[0063] S1.2.1.3: Construct a reconstructed ship image set based on the entity image set and background image set obtained in S1.2.1.2;

[0064] S1.2.1.4: Screen the reorganized ship image set obtained in S1.2.1.3 to obtain a screened ship image set; the specific process is as follows:

[0065] Manually filter out images that do not meet the requirements,

[0066] Such as the overlap of ships and ports.

[0067] The present invention obtains 200 new images with ship entities and background reconstruction through this method, thereby increasing the number of training samples

[0068] S1.2.1.5: Train the image style transfer network based on the selected ship image set to obtain a trained data enhancement module

[0069] The other steps and parameters are the same as those in the first to second embodiments.

[0070] Specific implementation method 4: This implementation method is different from specific implementation methods 1 to 4 in that:

[0071] The image segmentation network in S1.2.1.1 is a mask2former image segmentation model;

[0072] The mask2former image segmentation model is a general image segmentation model

[0073] In S1.2.1.3, a reorganized ship image set is constructed based on the entity image set and the background image set obtained in S1.2.1.2. The specific process is as follows:

[0074] S1.2.1.3.1: Randomly select an entity image and a background image from the entity image set and background image set obtained in S1.2.1.2, and combine them to obtain a reconstructed ship image;

[0075] S1.2.1.3.2: Repeat S1.2.1.3.1 N times to obtain N reconstructed ship images; N is a positive integer; the N reconstructed ship images are used to construct a reconstructed ship image set;

[0076] The image style transfer network in S1.2.1.5 includes: a feature extraction network and a CycleGAN network;

[0077] In S1.2.1.5, the image style conversion network is trained based on the selected ship image set to obtain a trained data enhancement module; the specific process is:

[0078] S1.2.1.5.1: Use image A in the screened ship image set and background image A' in the background image set corresponding to image A as input to the data enhancement module;

[0079] The input image A is processed by the generator G in the CycleGAN model to obtain the style-transferred image B;

[0080] The style-converted image B is used as the output of the data augmentation module;

[0081] S1.2.1.5.2: Calculate the feature mask M(A) of image A based on image A and the background image A' in the background image set corresponding to image A.

[0082] Among them, the feature mask of the i-th row and j-th column of image A is represented by M ij , expressed as:

[0083]

[0084] In the formula, A ij Indicates the pixel point in the i-th row and j-th column of the ship image set A, A′ ij Represents the pixel in the i-th row and j-th column of the original background image A' of image A; M of all pixels in image A ij The feature masks are merged into M(A);

[0085] Calculate the feature transformation loss L in the style transfer process based on the input image A, image B and the feature mask M(A) of image A F ;

[0086] S1.2.1.5.3: Using the discriminator D in the CycleGAN model Y Calculate the stylization loss L for image B GAN-G ;

[0087] L GAN-G It is the well-known standard adversarial loss in the CycleGAN model, which is called stylization loss here according to its function, and is well known to people in this field

[0088] S1.2.1.5.4: Input image B into generator F in the CycleGAN model to obtain the style-transferred image C;

[0089] S1.2.1.5.5: Using the discriminator D in the CycleGAN model X Calculate the stylization loss L for image C GAN-F ;

[0090] L GAN-FIt is the well-known standard adversarial loss in the CycleGAN model, which is called the stylized loss here according to its function and is well known to people in this field.

[0091] S1.2.1.5.6: Circulation loss L calculated from images A and C cyc ;

[0092] L cyc It is the well-known cycle-cycle consistency loss in the CycleGAN model, referred to here as cycle loss, which is well known to people in this field

[0093] S1.2.1.5.7: According to the feature transformation loss L F , stylization loss L GAN-F , stylization loss L GAN-F and circulation loss L cyc Calculate the total loss L style , repeat iterations S1.2.1.5.1 to S1.2.1.5.6, stop the iteration when the total loss function is minimized, and obtain the trained data enhancement module.

[0094] The other steps and parameters are the same as those in the first to third embodiments.

[0095] Specific implementation method 5: This implementation method is different from specific implementation methods 1 to 4 in that:

[0096] The feature transformation loss L is calculated in S1.2.1.5.2 F The formula is:

[0097]

[0098] in, represents expectation, A~Pdata(A) represents the data distribution of image A, || ||2 represents the binary norm, ⊙ represents the XOR operation, and M(A) represents the feature mask obtained by S1.2.1.5.2.

[0099] The feature extraction module selects the changed part between the image A of the ship image set and the original background image A' of the image A as the selected feature, that is, the part where the newly added hull is located.

[0100] In S1.2.1.5.7, according to the feature transformation loss L F , stylization loss L GAN-F , stylization loss L GAN-F and circulation loss L cyc Calculate the total loss L style The formula is:

[0101] L style =L GAN-G +LGAN-F +λ1L cyc +λ2L F

[0102] Where λ1 and λ2 represent weight parameters

[0103] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0104] Specific implementation method 6: This implementation method is different from specific implementation methods 1 to 5 in that:

[0105] The target detection module in S1.2.2 is an LSKNet network;

[0106] In S1.2.2, the target detection module is trained according to the enhanced ship remote sensing image set to obtain a trained target detection module; the specific process is:

[0107] S1.2.2.1: Randomly determine a filling position for each image in the enhanced ship remote sensing image set; use a fixed-size rectangular area to fill Gaussian noise in all filling positions to obtain the filled ship remote sensing image set;

[0108] S1.2.2.2: Input the filled ship remote sensing image set into the LSKNet network to obtain the coordinates of the ship objects and the square range in the image in the ship remote sensing image.

[0109] S1.2.2.3: Calculate the training loss based on the coordinates of the ship object and the square range in the ship remote sensing image, and repeat S1.2.2.1 to S1.2.2.2. When the training loss is the smallest, the trained target detection module is obtained.

[0110] ; The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0111] Specific implementation method 7: This implementation method is different from specific implementation methods 1 to 6 in that:

[0112] The fine-grained image classification module includes: a vit visual conversion network and a HERBS network in sequence;

[0113] In S1.2.3, the fine-grained image classification module is trained according to the cut ship remote sensing image set to obtain a trained fine-grained image classification module; the specific process is:

[0114] S1.2.3.1: Construct a negative sample image set and a positive sample image set for each image in the cut ship remote sensing image set;

[0115] S1.2.3.2: Input the cut image A in the ship remote sensing image set, the negative sample image set of image A, and the positive sample image of image A into the vit visual conversion network to obtain the feature map A* and the feature map set A* of the negative sample. — And the feature map A of the positive sample + ;

[0116] S1.2.3.3: The feature map A* and the feature map set A of the negative sample — And the feature map A of the positive sample + Input the HERBS network respectively to obtain the feature map B* and its negative sample feature map set B — And the feature map B of the positive sample + ;

[0117] The feature map B* is used as the output of the fine-grained image classification module;

[0118] The trained fine-grained image classification module outputs this feature map B*, and the specific model of the ship in the input image D is obtained based on the feature map B*;

[0119] S1.2.3.4: Based on the feature map B* and its negative sample feature map set B — And the feature map B of the positive sample + Calculate the contrast loss L con ;

[0120] S1.2.3.5: According to the contrast loss L con Calculate the total loss L, repeat S1.2.3.1 to S1.2.3.4, and when the total loss is the smallest, stop the iteration to obtain the trained fine-grained image classification module;

[0121] The HERBS network model is a network model well known to those skilled in the art. The structure of the HERBS network model is briefly introduced below, including a background suppression module and a high-temperature refinement module.

[0122] The first step of the background suppression (BS) module processing process is to generate a classification map from the feature map. The feature map output by the HERBS network model of the present invention is the classification map; then, according to the classification map Y i Calculate the maximum score graph P max,i , this process can be expressed as:

[0123] Y i =W i hs i +b i

[0124] P max,i =max((Softmax(Y i ))

[0125] Next, top-K feature selection is performed based on the scores in all predictions.

[0126] Then the graph convolutional network of the background suppression module is used to fuse the selected features and make predictions based on the fused features. The combined classification prediction is recorded as Y m .

[0127] Use cross entropy to calculate the prediction distribution P m The similarity between the true label y and the true label y. The loss after merging is calculated as follows:

[0128] P m =Softmax(Y m )

[0129]

[0130] Another purpose of the BS module is to suppress features in the lost classification map and increase the gap between foreground and background.

[0131] Apply the hyperbolic tangent function tanh to the deleted categorical graph Y d , the loss loss is calculated as the mean square error between the prediction and the pseudo target -1:

[0132] P d =tanh(Y d )

[0133]

[0134] The high temperature refinement module includes classifiers k1 and k2. The goal is to make classifier k1 learn the output distribution of classifier k2. Define the output of classifier k1 as Y i1 , the output of classifier k1 is Y i2 The refinement objective function helps the model learn more diverse and stronger representations in early layers while allowing later layers to focus on finer details. The high-temperature refinement module enables classifier k1 to discover wider areas, and classifier k2 to focus on learning fine-grained and discriminative features. The refinement loss is calculated as follows:

[0135] P i1 =LogSoftmax(Y i1 / T e )

[0136] P i2 =Softmax(Y i2 / T e )

[0137]

[0138] Compared to the knowledge distillation method, the initial temperature T is set to a higher value. The purpose of this is to encourage the model to explore various features, even if the initial predictions are inaccurate. Then, as the training progresses, the temperature is gradually reduced, allowing the model to focus more on the target class and learn more discriminative features. By using this decay strategy, the model can obtain diverse and refined representations and make accurate predictions.

[0139] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0140] Specific implementation eight: This implementation differs from specific implementations one to seven in that:

[0141] In S1.2.3.1, a negative sample image set and a positive sample image are constructed for each cut image in the cut ship remote sensing image set; the specific process is:

[0142] S1.2.3.1.1: Determine the ship models of all cut images; the confused ship model of each ship model; the specific process is: determine the ship models of all cut images according to the labels of the data set by the HERBS network;

[0143] According to the confusion matrix of the initial classification results of the HERBS network in the data set, find 1 to 3 confusing ship models that are misclassified as this model for each ship model;

[0144] The confused ship models are the top three in terms of error rate among the misidentified ship models;

[0145] S1.2.3.1.2: Construct negative sample image sets and positive sample images for all cut images based on the obfuscated ship models;

[0146] The specific process of constructing a negative sample image set and a positive sample image for a certain cut image is as follows:

[0147] Randomly select a ship remote sensing image with a confused ship model corresponding to the cut image from the enhanced ship remote sensing image set,

[0148] For each confused ship model, one image is randomly selected. If there are less than three models, the selection will be repeated until three remote sensing images are obtained as negative sample images of the cut image.

[0149] Randomly select a ship remote sensing image of the ship model corresponding to the cut image from the enhanced ship remote sensing image set as the positive sample image of the cut image

[0150] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0151] Specific implementation method 9: This implementation method is different from specific implementation methods 1 to 8 in that:

[0152] In S1.2.3.4, according to the feature map B* and its negative sample feature map set B — And the feature map B of the positive sample + Calculate the contrast loss L con The formula is:

[0153]

[0154] Among them, B * represents the feature map, B + is the positive sample feature map, B - is a negative sample feature map set, Sim(.) represents the dot product of two feature map vectors; max(.) represents the maximum value; α is a constant margin, so max(Sim(z i ,z n )-α,0) makes it so that only negative samples with similarity greater than α will cause loss, thus improving the accuracy of network prediction.

[0155] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0156] Specific implementation method 10: This implementation method is different from specific implementation methods 1 to 9 in that:

[0157] According to the contrast loss L con The formula for calculating the total loss L is:

[0158] L=λ m L m +λ d L d +λ l L l +λ con L con

[0159] Among them, L m is the classification graph prediction loss in the HERBS network, L d is the classification graph discarding loss in the HERBS network, L l is the distillation refinement loss in the HERBS network, L m , L d and L l is the well-known loss function in the HERBS network, λ m ,λ d ,λ l ,λ con is the weight coefficient.

[0160] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0161] Combining the simulation analysis of specific implementation methods one to ten

[0162] Since the present invention is aimed at fine-grained target recognition of sea surface remote sensing images, and is divided into two subtasks: coarse-grained target recognition for ships and fine-grained image classification for ship models, for the target detection subtask, the first-level labels (i.e., ship labels) in HRSC2016 are used for training, and for the fine-grained image classification subtask, 436 images with third-level labels in the training set (i.e., 23 labels of specific types) are used for training. Finally, the overall performance of the overall architecture was tested on 181 three-level labeled images in the HRSC2016 test set. The overall performance of the architecture was evaluated separately. For the evaluation of the target detection subtask and the final model, mAP is used as the evaluation indicator. For the fine-grained image classification subtask, accuracy is used as the evaluation indicator.

[0163] As a comparison, the present invention tests the performance of existing target detection methods on HRSC2016, and tests the fine-grained target detection performance of faster-RCNN, yolov5, LSKNet, DETReg and STD on sea surface remote sensing images. The performance of the model on the fine-grained object detection task is reported in Table 1.

[0164] Table 1 Performance comparison of different target detection methods on HRSC2016

[0165]

[0166] It can be seen that the proposed system improves mAP by 1.26% over the strongest baseline. This proves that the adopted architecture is superior to the previous general target detection model in capturing small target details in complex backgrounds. At the same time, the added data enhancement and contrastive learning modules also effectively solve the problems of scarce training samples and unbalanced number of categories in the current fine-grained target detection in the ship field.

[0167] Replacement tests on some key steps of the present invention

[0168] Here, the present invention replaces the implementation methods of some key steps to observe the performance changes, and the optimization contribution results of each key step are shown in Table 2.

[0169] The impact of removing augmented data. This section studies the importance of data augmentation. The present invention generates new images in a stylized manner through entity background separation and random splicing. The subtask of image classification is trained using the pre-augmented dataset and the augmented dataset, and then tested on a unified test set. Taking TransFG, which currently performs well in fine-grained image classification, as a control, the experimental results are shown in Table 2. It can be seen from the results that after removing the augmented data, the classification accuracy of the two models has decreased, with TransFG decreasing by 1.9% and the model proposed in the present invention decreasing by 1.8%. This is because the addition of new images effectively alleviates the overfitting problem and introduces different background features.

[0170] Table 2 Ablation experiment results of data augmentation on HRSC2016 dataset

[0171]

[0172] Removing the impact of contrastive learning. This section studies the importance of contrastive learning, and the present invention removes the contrastive learning module in the fine-grained image classification module. Table 3 shows the performance comparison of image classification models with and without contrastive loss to verify its effectiveness. From the results, it can be seen that when the contrastive loss is removed, the accuracy is reduced by 1.0% for transFG and 0.7% for our model. This is because the contrastive loss can effectively expand the representation distance between similar subcategories and reduce the representation distance between the same subcategories.

[0173] Table 3 Ablation experiment results of comparative learning on HRSC2016 dataset

[0174]

[0175] The above only describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the above specific implementation methods. Although the present invention has been disclosed as above in the preferred embodiments, it is not used to limit the present invention. Any technician familiar with the profession can make some changes or modify the technical contents disclosed above into equivalent embodiments with equivalent changes without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A ship detection method based on remote sensing images, characterized in that: The following steps are involved: S1: Construct a ship detection model based on remote sensing images; obtain a trained ship detection model based on remote sensing images; S2: collecting remote sensing images of ships to be identified, inputting the remote sensing images of ships to be identified into a trained ship detection model based on remote sensing images, and obtaining the ship model of the remote sensing images of ships to be identified; In S1, a ship detection model based on remote sensing images is constructed; a trained ship detection model based on remote sensing images is obtained; the specific process is: S1.1: Acquire ship remote sensing image sets; S1.2: Construct a ship detection model based on remote sensing images; train the ship detection model based on remote sensing images according to the ship remote sensing image set to obtain a trained ship detection model based on remote sensing images.

2. A ship detection method based on remote sensing images according to claim 1, characterized in that: The ship remote sensing image set in S1.1 is the HRSC2016 remote sensing image ship dataset; The remote sensing image-based ship detection model in S1.2 includes: a data enhancement module, a target detection module and a fine-grained image classification module; In S1.2, the ship detection model based on remote sensing images is trained according to the ship remote sensing image set to obtain a trained ship detection model based on remote sensing images. The specific process is as follows: S1.2.1: Train the data enhancement module according to the ship remote sensing image set to obtain a trained data enhancement module; Then, the images of the ship remote sensing image set are input into the trained data enhancement module to obtain an enhanced ship remote sensing image set; S1.2.2: training the target detection module according to the enhanced ship remote sensing image set to obtain a trained target detection module; Then the enhanced ship remote sensing image set is input into the trained target detection module; the ship object and the position of the ship object in the enhanced ship remote sensing image are obtained; The position of the ship object is the coordinates of the square range where the ship object is located in the picture; S1.2.3: cutting all the ship objects in the enhanced ship remote sensing image according to the positions of the ship objects to obtain a cut ship remote sensing image set; Each cut picture in the cut ship remote sensing image set contains only one ship; The fine-grained image classification module is trained according to the cut ship remote sensing image set to obtain a trained fine-grained image classification module; S1.2.4: Combine the trained data enhancement module, the trained target detection module and the trained fine-grained image classification module to obtain a trained remote sensing image-based ship detection model.

3. A ship detection method based on remote sensing images according to claim 2, characterized in that: The data enhancement module in S1.2.1 includes an image segmentation network and an image style conversion network; In S1.2.1, the data enhancement module is trained according to the ship remote sensing image set to obtain a trained data enhancement module; the specific process is: S1.2.1.1: Input each ship remote sensing image in the ship remote sensing image set into the image segmentation network in the data enhancement module for extraction processing to obtain a mask of each ship remote sensing image in the ship remote sensing image set; S1.2.1.2: Based on the mask of each ship remote sensing image extracted in S1.2.1.1, ship entity extraction processing is performed on each ship image in the ship remote sensing image set to obtain a ship entity image of each ship remote sensing image and a background image of each ship remote sensing image; The ship entity images of all ship remote sensing images form an entity image set; the background images of all ship remote sensing images form a background image set; S1.2.1.3: Construct a reconstructed ship image set based on the entity image set and background image set obtained in S1.2.1.2; S1.2.1.4: Screening the reorganized ship image set obtained in S1.2.1.3 to obtain a screened ship image set; S1.2.1.5: Train the image style transfer network based on the screened ship image set to obtain a trained data enhancement module.

4. A ship detection method based on remote sensing images according to claim 3, characterized in that: The image segmentation network in S1.2.1.1 is a mask2former image segmentation model; In said S1.2.1.3, a reorganized ship image set is constructed based on the entity image set and the background image set obtained in S1.2.1.2; the specific process is as follows: S1.2.1.3.1: Randomly select an entity image and a background image from the entity image set and background image set obtained in S1.2.1.2, and combine them to obtain a reconstructed ship image; S1.2.1.3.2: Repeat S1.2.1.3.1 N times to obtain N reconstructed ship images; N is a positive integer; Construct a reconstructed ship image set from N reconstructed ship images; The image style transfer network in S1.2.1.5 includes: a feature extraction network and a CycleGAN network; The CycleGAN network includes: a generator G, a discriminator D Y , generator F and discriminator D X ; In S1.2.1.5, the image style conversion network is trained based on the selected ship image set to obtain a trained data enhancement module; the specific process is: S1.2.1.5.1: Use image A in the screened ship image set and background image A' in the background image set corresponding to image A as input to the data enhancement module; The input image A is processed by the generator G in the CycleGAN network to obtain image B; The style-converted image B is used as the output of the data augmentation module; S1.2.1.5.2: Calculate the feature mask M(A) of image A based on image A and the background image A' in the background image set corresponding to image A. Among them, the feature mask of the i-th row and j-th column of image A is represented by M ij , expressed as: In the formula, A ij Indicates the pixel point in the i-th row and j-th column of the ship image set A, A′ ij Represents the pixel at the i-th row and j-th column of the background image A'; Calculate the feature transformation loss L in the style transfer process based on the input image A, image B and the feature mask M(A) of image A F ; S1.2.1.5.3: Using the discriminator D in the CycleGAN network Y Calculate the stylization loss L for image B GAN-G ; S1.2.1.5.4: Input image B into generator F in the CycleGAN network to obtain image C; S1.2.1.5.5: Using the discriminator D in the CycleGAN model X Calculate the stylization loss L for image C GAN-F ; S1.2.1.5.6: Circulation loss L calculated from images A and C cyc ; S1.2.1.5.7: According to the feature transformation loss L F , stylization loss L GAN-F , stylization loss L GAN-F and circulation loss L cyc Calculate the total loss L style , repeat iterations S1.2.1.5.1 to S1.2.1.5.6, stop the iteration when the total loss function is minimized, and obtain the trained data enhancement module.

5. A ship detection method based on remote sensing images according to claim 4, characterized in that: The feature transformation loss L is calculated in S1.2.1.5.2 F The formula is: in, represents expectation, A~Pdata(A) represents the data distribution that image A obeys, || ||2 represents the binary norm, and ⊙ represents the XOR operation; In S1.2.1.5.7, according to the feature transformation loss L F , stylization loss L GAN-F , stylization loss L GAN-F and circulation loss L cyc Calculate the total loss L sytle The formula is expressed as: L style =L GAN-G +L GAN-F +λ1L cyc +λ2L F Where λ1 and λ2 represent weight parameters.

6. A ship detection method based on remote sensing images according to claim 5, characterized in that: The target detection module in S1.2.2 is an LSKNet network; In S1.2.2, the target detection module is trained according to the enhanced ship remote sensing image set to obtain a trained target detection module; the specific process is: S1.2.2.1: Randomly determine a filling position for each image in the enhanced ship remote sensing image set; use a fixed-size rectangular area to fill Gaussian noise in all filling positions to obtain the filled ship remote sensing image set; S1.2.2.2: Input the filled ship remote sensing image set into the LSKNet network to obtain the coordinates of the ship objects and the square range in the image in the ship remote sensing image. S1.2.2.3: Calculate the training loss based on the coordinates of the ship object and the square range in the ship remote sensing image, repeat S1.2.2.1 to S1.2.2.2, and when the training loss is the smallest, obtain the trained target detection module.

7. A ship detection method based on remote sensing images according to claim 6, characterized in that: The fine-grained image classification module includes: a vit visual conversion network and a HERBS network in sequence; In S1.2.3, the fine-grained image classification module is trained according to the cut ship remote sensing image set to obtain a trained fine-grained image classification module; the specific process is: S1.2.3.1: Construct a negative sample image set and a positive sample image set for each image in the cut ship remote sensing image set; S1.2.3.2: Input the cut image D in the ship remote sensing image set, the negative sample image set of image D, and the positive sample image of image D into the vit visual conversion network to obtain the feature map D*, the feature map set D of the negative sample, and the positive sample image D — And the feature map D of the positive sample + ; S1.2.3.3: The feature map D* and the feature map set D of the negative sample are — And the feature map D of the positive sample + Input the HERBS network respectively to obtain the feature map B* and its negative sample feature map set B — And the feature map B of the positive sample + ; The feature map B* is used as the output of the fine-grained image classification module; the specific model of the ship in the input image D is obtained according to the feature map B*; S1.2.3.4: Based on the feature map B* and its negative sample feature map set B — And the feature map B of the positive sample + Calculate the contrast loss L con ; S1.2.3.5: According to the contrast loss L con Calculate the total loss L, repeat S1.2.3.1 to S1.2.3.4, and when the total loss is the smallest, stop the iteration to obtain the trained fine-grained image classification module.

8. A ship detection method based on remote sensing images according to claim 7, characterized in that: In S1.2.3.1, a negative sample image set and a positive sample image are constructed for each cut image in the cut ship remote sensing image set; the specific process is: S1.2.3.1.1: Determine the ship models of all cut images; the confused ship model of each ship model; the specific process is: determine the ship models of all cut images according to the labels of the data set by the HERBS network; According to the confusion matrix of the initial classification results of the HERBS network in the data set, find 1 to 3 confusing ship models that are misclassified as this model for each ship model; S1.2.3.1.2: Construct negative sample image sets and positive sample images for all cut images based on the obfuscated ship models; The specific process of constructing a negative sample image set and a positive sample image for a certain cut image is as follows: Randomly select a ship remote sensing image with a confused ship model corresponding to the cut image from the enhanced ship remote sensing image set, For each confused ship model, one image is randomly selected as the negative sample image of the cut image. A ship remote sensing image of a ship model corresponding to the cut image is randomly selected from the enhanced ship remote sensing image set as a positive sample image of the cut image.

9. A ship detection method based on remote sensing images according to claim 8, characterized in that: In S1.2.3.4, according to the feature map B* and its negative sample feature map set B — And the feature map B of the positive sample + Calculate the contrast loss L con The formula is: Among them, B * represents the feature map, B + is the positive sample feature map, B - is the negative sample feature atlas, Sim(.) represents the dot product processing; max(.) represents the maximum value; α is the constant margin.

10. A ship detection method based on remote sensing images according to claim 9, characterized in that: According to the contrast loss L con The formula for calculating the total loss L is: L=λ m L m +λ d L d +λ l L l +λ con L con Among them, L m is the classification graph prediction loss in the HERBS network, L d is the classification graph discarding loss in the HERBS network, L l is the distillation refinement loss in the HERBS network, λ m ,λ d ,λ l ,λ con is the weight coefficient.

Images