Small sample SAR image target detection method based on optical domain knowledge migration
By constructing multi-datasets and transfer learning methods, pseudo-optical remote sensing images are generated for model training, which solves the problem of small samples and domain differences in SAR image object detection, and improves detection accuracy and generalization capabilities.
Patent Information
- Application Number
- CN202410957571.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-11-12
AI Technical Summary
SAR image object detection faces the problem of small samples. The lack of labeled data leads to poor generalization capabilities of the model, and the domain differences between optical remote sensing images and SAR images are large, making it difficult to effectively detect the object.
By constructing domain adaptive training data sets, pre-trained data sets and domain transfer learning data sets, the pre-trained object detection model is used to learn optical domain feature, generate pseudo-optical remote sensing images, and further train the model to realize the use of SAR images to optical Domain transfer learning of domains improves detection accuracy.
It improves the accuracy of SAR image object detection, reduces the requirements of domain adaptation performance, can effectively detect in limited labels or no labels, and solves the problem of small sample and domain differences.
Smart Images

Figure CN118941978A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of radar target detection, and in particular relates to a small sample SAR image target detection method based on optical domain knowledge migration. Background Art
[0002] Synthetic Aperture Radar (SAR) is an active microwave imaging radar that is not affected by light factors and has a certain surface penetration ability. It can observe the earth all day and all weather. It is widely used in tasks such as land change detection, land scene segmentation, and weather forecasting, and has promoted the development of SAR image target detection and recognition technology. The target detection task of SAR images requires the ability to detect aircraft, ships, vehicles, etc. in the image, while the target recognition task requires the ability to identify the type or model of the object.
[0003] With the rapid development of machine learning technology, SAR image target detection and recognition based on deep learning has also developed rapidly. The widely used fully supervised training often requires a large amount of labeled data as model input. When the supervision information is not rich enough, the model often overfits, resulting in poor generalization ability of the model, thereby reducing the accuracy of target detection and recognition. Due to the imaging characteristics of SAR images themselves, there are few labeled SAR images that can be obtained. The lack of a large amount of labeled data for full supervision training of the model is an important problem facing current SAR image target detection.
[0004] In remote sensing technology, optical remote sensing images have a large amount of labeled data, and usually have rich color information and high resolution, which can be easily recognized by the human eye. SAR images not only have less labeled data, but also usually have no color information. In addition, the presence of some noise makes the target edge blurred and difficult to identify. Since optical remote sensing images and SAR images have similar scenes and target types, domain transfer learning between the two can solve the small sample problem in SAR image target detection tasks. Summary of the invention
[0005] In view of the deficiencies in the prior art, the technical problem that the present invention intends to solve is to propose a small sample SAR image target detection method based on optical domain knowledge transfer.
[0006] The present invention solves the technical problem by adopting the following technical solution:
[0007] A small sample SAR image target detection method based on optical domain knowledge transfer, characterized in that the method comprises the following steps:
[0008] Step 1: Construct domain adaptation training dataset, pre-training dataset and domain transfer learning dataset;
[0009] Acquire multiple labeled and unlabeled original optical remote sensing images, and use a sliding window to crop the original optical remote sensing images to obtain several labeled and unlabeled optical remote sensing images;
[0010] Obtain multiple labeled and unlabeled original SAR images, and use a sliding window to crop the original SAR images to obtain several labeled and unlabeled SAR images;
[0011] All unlabeled optical remote sensing images and SAR images constitute the domain adaptation training dataset, all labeled optical remote sensing images constitute the pre-training dataset, and all labeled SAR images constitute the domain transfer learning dataset;
[0012] Step 2: Use the pre-training data set to pre-train the target detection model to obtain a pre-trained target detection model;
[0013] Step 3: Use the domain adaptation training dataset to train the domain adaptation model to obtain the trained domain adaptation model;
[0014] Step 4: Generate pseudo optical remote sensing images using the trained domain adaptation model based on the domain transfer learning dataset; further train the pre-trained target detection model using the pseudo optical remote sensing images to achieve domain transfer learning of the target detection model and obtain the trained target detection model;
[0015] Step 5: Use a sliding window to crop the original SAR image to be tested to obtain the sub-SAR image to be tested; input the sub-SAR image to be tested into the trained domain adaptation model to generate the pseudo-optical remote sensing image to be tested; input the pseudo-optical remote sensing image to be tested into the trained target detection model to obtain the target frame coordinates; then, according to the position of the sub-SAR image to be tested in the original SAR image to be tested, map the target frame coordinates to the original SAR image to be tested to complete the target detection.
[0016] Compared with the prior art, the present invention has the following advantages:
[0017] 1. Aiming at the problem of small amount of labeled data of SAR images, the present invention combines domain adaptation method, uses optical remote sensing images with a large amount of labeled data to assist SAR image detection training, and proposes a new idea of domain adaptation assisted target detection. First, a large number of labeled optical remote sensing images are used to pre-train the target detection model, so that the model can learn the optical domain features more accurately during the pre-training stage; then, pseudo optical remote sensing images are generated from SAR images through the domain adaptation model, and the pre-trained target detection model is further trained using the pseudo optical remote sensing images, and the parameters of the target detection model are fine-tuned so that the target detection model can learn the features of the pseudo optical remote sensing images; finally, the SAR image to be tested is converted into a pseudo optical remote sensing image during the test stage, and then the trained target detection model is used for detection. Converting the input image from the SAR domain to the optical domain not only realizes single-stage pixel-level domain adaptation and solves the small sample problem in SAR image target detection, but also improves the detection accuracy and reduces the requirements for domain adaptation performance.
[0018] 2. The current SAR image target detection and recognition based on domain adaptation mainly converts optical remote sensing images into SAR images, and directly uses the converted SAR images to train the model, which may lead to deviations in the results due to the inability to fully align the features. In order to solve this problem, the present invention takes a different approach to convert SAR images into optical remote sensing images for training and testing the target detection model. On the basis of the pre-trained target detection model, the SAR image is converted into the optical domain, and the pseudo-optical remote sensing image is used to perform domain transfer learning on the pre-trained target detection model. When the pseudo-optical remote sensing image is not exactly the same as the original optical remote sensing image, the detection performance of the model is still improved, and the target detection task can be performed even when no labeled image is used at all, which solves the problem of large domain differences between optical remote sensing images and SAR images. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 is a flow chart of the target detection model training phase of the present invention;
[0020] Figure 2 is a flow chart of the testing phase of the present invention;
[0021] Figure 3 The pseudo optical remote sensing image of the present invention is compared with the original SAR image;
[0022] Figure 4 It is a comparison of the detection results of different methods. DETAILED DESCRIPTION
[0023] Specific embodiments are given below in conjunction with the accompanying drawings. The specific embodiments are only used to introduce the technical solutions of the present invention in detail and are not intended to limit the protection scope of the present application.
[0024] The present invention provides a small sample SAR image target detection method based on optical domain knowledge transfer (hereinafter referred to as method, see Figures 1 to 4 ), including the following steps:
[0025] Step 1: Construct domain adaptation training dataset, pre-training dataset and domain transfer learning dataset;
[0026] Acquire multiple labeled and unlabeled original optical remote sensing images with an image size of 1024×1024 pixels; crop the original optical remote sensing images in the form of a sliding window to obtain several labeled and unlabeled optical remote sensing images with an image size of 256×256 pixels; acquire multiple labeled and unlabeled original SAR images with an image size of 3000×3000 pixels; similarly, crop the original SAR images in the form of a sliding window to obtain several labeled and unlabeled SAR images with an image size of 300×300 pixels;
[0027] All unlabeled optical remote sensing images and SAR images constitute the domain adaptation training dataset, which is used for unsupervised training of the domain adaptation model; all labeled optical remote sensing images constitute the pre-training dataset, which is used for pre-training of the target detection model; all labeled SAR images constitute the domain transfer learning dataset, which is used for domain transfer learning of the pre-trained target detection model;
[0028] Step 2: Use the pre-training data set to perform supervised pre-training on the target detection model to obtain a pre-trained target detection model; the target detection model of this embodiment adopts the classic single-stage target detection network SSD.
[0029] Step 3: Use the domain adaptation training dataset to perform unsupervised training on the domain adaptation model to obtain the trained domain adaptation model;
[0030] The domain adaptation model adopts a cyclic generative network CycleGAN, which includes a first generator and a second generator with the same structure, as well as a first discriminator and a second discriminator with the same structure; the first generator generates a pseudo SAR image based on the optical remote sensing image, and the first discriminator judges the performance of the first generator based on the real SAR image and the pseudo SAR image to achieve image conversion from optical domain to SAR domain; the second generator generates a pseudo optical remote sensing image based on the SAR image, and the second discriminator judges the performance of the second generator based on the real optical remote sensing image and the pseudo optical remote sensing image.
[0031] The domain adaptation model is trained unsupervisedly using the domain adaptation training dataset. The optical remote sensing images of the domain adaptation training dataset are input into the first generator to generate pseudo SAR images. The SAR images of the domain adaptation training dataset are input into the second generator to generate pseudo optical remote sensing images. The first discriminator and the second discriminator respectively discriminate the performance of the first generator and the second generator to obtain the trained domain adaptation model.
[0032] Step 4: Generate pseudo optical remote sensing images, perform domain transfer learning on the target detection model, and obtain the trained target detection model;
[0033] According to the domain transfer learning dataset, the labeled SAR images are input into the second generator of the trained domain adaptation model to generate pseudo optical remote sensing images, which are labeled.
[0034] Based on the pre-training of the target detection model, the pre-trained target detection model is further trained using pseudo optical remote sensing images, and the parameters of the pre-trained target detection model are fine-tuned to enable the model to learn the knowledge of pseudo optical remote sensing images, realize domain transfer learning of the target detection model, and obtain the trained target detection model.
[0035] Step 5: Perform target detection on the SAR image based on the trained domain adaptation model and target detection model;
[0036] The original SAR image to be tested is cropped in the form of a sliding window to obtain a sub-SAR image to be tested; the sub-SAR image to be tested is input into the second generator of the trained domain adaptation model to generate a pseudo-optical remote sensing image to be tested; the pseudo-optical remote sensing image to be tested is input into the trained target detection model to obtain the target detection result, i.e., the target frame coordinates; according to the position of the sub-SAR image to be tested in the original SAR image to be tested, the target frame coordinates are mapped to the original SAR image to be tested to complete the target detection.
[0037] Example
[0038] This embodiment uses 720 labeled and unlabeled original optical remote sensing images, all of which are 1024×1024 pixels in size. The original optical remote sensing images include a large number of ships, not only ships on the sea, but also abundant ships near the coast. The original optical remote sensing images are cropped by sliding window cropping to obtain optical remote sensing images with a size of 256×256 pixels. Among them, there are 19,838 labeled optical remote sensing images and 300 unlabeled optical remote sensing images.
[0039] The raw SAR images with and without labels are from the AIR-SARShip-1.0 dataset, totaling 31 images, with an image size of 3000×3000 pixels, image resolutions of 1m and 3m, scene types including ports, islands and reefs, and sea surfaces of different levels of sea conditions, and targets covering nearly a thousand ships of more than ten categories, including transport ships, oil tankers, and fishing boats. The raw SAR images are cropped by sliding window cropping to obtain SAR images of 300×300 pixels; among them, there are 300 labeled SAR images and 541 unlabeled SAR images.
[0040] Table 1. Number of samples at different training stages
[0041]
[0042] Experiment 1: SAR image target detection is performed according to the method of the present invention, that is, first, a large number of labeled optical remote sensing images are used to pre-train the target detection model to obtain a pre-trained target detection model; then, the domain adaptation model is unsupervisedly trained using unlabeled SAR images, and the trained domain adaptation model generates pseudo optical remote sensing images based on the labeled SAR images, and the pre-trained target detection model performs transfer learning based on the pseudo optical remote sensing images to obtain a trained target detection model.
[0043] Comparison between original SAR images and pseudo-optical remote sensing images Figure 3 , (a) is the original SAR image, (b) is the pseudo optical remote sensing image. Figure 3 It can be seen that the pseudo-optical remote sensing image clearly retains the boundary information of the target, and produces detailed information inside the ship target that is very similar to the ship target in the real optical remote sensing image, and the colors of the sea background and the land background are well restored. It can be intuitively seen that the pseudo-optical remote sensing image has good image quality.
[0044] In order to verify the effectiveness of the method in the case of small samples, different proportions of labeled pseudo optical remote sensing images are used to perform domain transfer learning on the target detection model, and the trained target detection model is obtained. The trained domain adaptation model generates pseudo optical remote sensing images for the SAR image to be tested, and the trained target detection model performs target detection based on the pseudo optical remote sensing images. The experimental results are shown in Table 2, and the detection results are shown in Table 2. Figure 4 (a).
[0045] Experiment 2: To verify the effectiveness of the proposed method for SAR target detection, the same number of labeled SAR images as in Experiment 1 are used to directly supervise the target detection model (denoted as SSD). The parameters of the target detection model are randomly initialized and no pre-training is required. The experimental results are shown in Table 2. The detection results are shown in Table 2. Figure 4 (b).
[0046] Table 2 Comparison of experimental results of different methods
[0047]
[0048] Experiment 3: In order to further verify the effectiveness of the method of the present invention, about 10,000 optical remote sensing images are input into the trained domain adaptation model to generate a large number of pseudo SAR images. These SAR images are used together with the original labeled SAR images to supervise the target detection model for training. The SAR to be tested is directly input into the trained target detection model for detection. The specific experimental results are shown in Table 3.
[0049] Table 3 Comparison of experimental results under different domain adaptation ideas
[0050]
[0051] From the results of experiments 1, 2, and 3 above, it can be seen that the method of the present invention has achieved a significant performance improvement when using labeled pseudo-optical remote sensing images for domain transfer learning. It is worth noting that without any SAR images for training, the method of the present invention still obtains an F1 score of 36.89%, which shows that the method of the present invention can still be used with limited labeled SAR images or even without labeled images.
[0052] Figure 4 The green box represents a correct detection, the blue box represents a missed detection, and the red box represents a false alarm. From the comparison in the figure, it can be seen that there are many missed detections and false alarms in SSD detection. The method of the present invention reduces missed detections and keeps the false alarm rate at a low level, and achieves good results in various scenes and images of different resolutions.
[0053] Compared with general deep learning methods, the method of the present invention includes two stages: pre-training and domain transfer learning. In the pre-training stage, the target detection model learns more optical domain knowledge based on rich labeled optical remote sensing images. At the same time, compared with the method of converting labeled optical remote sensing images into pseudo-SAR images and directly applying them to training, due to the huge domain differences between the optical domain and the SAR domain, it is not only difficult to better learn features by directly using the pseudo-SAR images generated by CycleGAN for training, but it even causes confusion in the target detection model during learning, which leads to the performance of the target detection model being deteriorated. In addition, directly using the generated pseudo-SAR images for training puts higher requirements on the performance of the domain adaptation model. In contrast, even when the deep domain adaptation effect is not ideal, the performance of the target detection model of the method of the present invention can still be effectively improved.
[0054] Any matters not described in the present invention are applicable to the prior art.
Claims
1. A small sample SAR image target detection method based on optical domain knowledge transfer, characterized in that: The method comprises the following steps: Step 1: Construct domain adaptation training dataset, pre-training dataset and domain transfer learning dataset; Acquire multiple labeled and unlabeled original optical remote sensing images, and use a sliding window to crop the original optical remote sensing images to obtain several labeled and unlabeled optical remote sensing images; Obtain multiple labeled and unlabeled original SAR images, and use a sliding window to crop the original SAR images to obtain several labeled and unlabeled SAR images; All unlabeled optical remote sensing images and SAR images constitute the domain adaptation training dataset, all labeled optical remote sensing images constitute the pre-training dataset, and all labeled SAR images constitute the domain transfer learning dataset; Step 2: Use the pre-training data set to pre-train the target detection model to obtain a pre-trained target detection model; Step 3: Use the domain adaptation training dataset to train the domain adaptation model to obtain the trained domain adaptation model; Step 4: Generate pseudo optical remote sensing images using the trained domain adaptation model based on the domain transfer learning dataset; further train the pre-trained target detection model using the pseudo optical remote sensing images to achieve domain transfer learning of the target detection model and obtain the trained target detection model; Step 5: Use a sliding window to crop the original SAR image to be tested to obtain the sub-SAR image to be tested; input the sub-SAR image to be tested into the trained domain adaptation model to generate the pseudo-optical remote sensing image to be tested; input the pseudo-optical remote sensing image to be tested into the trained target detection model to obtain the target frame coordinates; then, according to the position of the sub-SAR image to be tested in the original SAR image to be tested, map the target frame coordinates to the original SAR image to be tested to complete the target detection.
2. The small sample SAR image target detection method based on optical domain knowledge transfer according to claim 1 is characterized in that: The domain adaptation model includes a first generator and a second generator having the same structure, and a first discriminator and a second discriminator having the same structure; The first generator generates a pseudo SAR image according to the optical remote sensing image, and the first discriminator discriminates the performance of the first generator according to the real SAR image and the pseudo SAR image; The second generator generates a pseudo optical remote sensing image according to the SAR image, and the second discriminator discriminates the performance of the second generator according to the real optical remote sensing image and the pseudo optical remote sensing image.
3. The small sample SAR image target detection method based on optical domain knowledge transfer according to claim 1 or 2 is characterized in that: The target detection model adopts the SSD network.
