A method for target detection in remote sensing images with a small number of labels

Abstract

A remote sensing image target detection method under a small amount of labeling conditions belongs to the technical field of remote sensing image intelligent processing. The method is based on a one-stage detection framework, uses a convolutional neural network with a residual structure as a feature extraction network, and obtains multi-scale feature maps through a feature pyramid network to fully utilize feature information; using a rotated bounding box as the detector output, the foreground and background can be accurately divided on a bird's-eye view image; in addition, a model-independent meta-learning method MAML is used to update the meta-knowledge of a region recommendation network, and an anchor box that may contain a target at each feature point position is recommended; the detection network calculates the detection result based on the recommended anchor box; after the meta-knowledge update step, the region recommendation network learns the prior knowledge of the same distribution detection task, so the network can obtain accurate detection results of unseen class targets after fine-tuning with a small sample of unseen class.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing image intelligent processing, and in particular relates to a remote sensing image target detection method under the condition of a small amount of markings. Background Art

[0002] With the development of remote sensing technology, its role in civil and military fields is becoming more and more extensive, and the intelligent processing technology of remote sensing images has gradually become an important research direction in the field of remote sensing. Applying the target detection task to remote sensing scenes is an important step in the intelligent processing of remote sensing images. The target detection task is a basic task in machine vision, which can detect targets of a given category and the target position in a given image. Traditional target detection methods use sliding windows to detect based on manually defined features; deep learning detectors use neural networks to extract features with higher abstraction, and first select areas where targets may exist through a region recommendation network, and then use the detection network to obtain the detection results. Neural networks can extract feature information with higher generalization, and at the same time, the region recommendation network can more accurately and efficiently mine the potential location of the target, so deep learning detectors have better performance.

[0003] Due to the above advantages, detectors based on deep learning have a wide range of applications. In terms of detector training, most deep detectors use an end-to-end training method. As the network depth increases, a large amount of training data is usually required to ensure sufficient gradient information during training. In remote sensing images, the long-tail distribution of targets is obvious, that is, common targets are easy to obtain and the sample size is large; uncommon targets are difficult to obtain and the sample size is small. When the detector is trained with a small sample, it will not be able to reach the best optimization state, or even diverge. Summary of the invention

[0004] To solve the above problems, the present invention provides a remote sensing image target detection method under a small number of label conditions, the method comprising the steps of:

[0005] Divide the dataset into pre-training set, meta-training set and test set, and divide the meta-training set and test set into their respective support set and query set;

[0006] Meta-support training of the region proposal network using the support set in the meta-training set;

[0007] Perform meta-queries using the query set in the meta-training set;

[0008] Based on the error in the meta-query, the back-propagation algorithm is used to update the parameters of the region recommendation network.

[0009] Fine-tune the entire network using the support set in the test set;

[0010] Compute the detection results of the query set in the test set.

[0011] Preferably, the data set is divided into a pre-training set, a meta-training set and a test set, wherein the pre-training set and the meta-training set respectively contain different image samples of the same category, and the test set contains image samples of different categories from the pre-training set and the meta-training set; further, the pre-training set, the meta-training set and the test set are composed of their own support set and query set, wherein the support set and the query set contain different image samples of the same category.

[0012] Preferably, the network parameters are initialized by using samples in a pre-training set to pre-train the network through a gradient descent algorithm. Furthermore, the network structure includes a feature extraction backbone network composed of a convolutional neural network, a feature pyramid network for obtaining multi-size feature maps, a fully convolutional network responsible for region recommendation, and a fully convolutional network for refining the recommended region.

[0013] Preferably, the meta-support training uses samples in the meta-training set as several detection tasks, uses samples in the support set, and calculates temporary parameters of each detection task through a gradient descent algorithm. The calculation formula is:

[0014]

[0015] Among them, θ is the pre-trained parameter of the region recommendation network, Γ i is the i-th detection task, θ i ′ is the temporary parameter of the i-th detection task, is the loss of the i-th detection task under parameter θ, α represents the learning rate, Represents the calculation of gradient.

[0016] Preferably, the meta-query uses samples of the query set in the meta-training set to calculate the temporary parameter θ for each detection task through a gradient descent algorithm. i Loss under '

[0017] Preferably, the calculation formula for updating the regional recommendation network parameters according to the error in the meta-query is:

[0018]

[0019] Among them, θ is the pre-trained parameter of the region recommendation network, p(Γ) is a number of identically distributed detection tasks, Γ i is the i-th detection task, θ i ′ is the temporary parameter of the i-th detection task, Re-parameter θ for the i-th detection task i ′, β represents the learning rate, Represents the calculation of gradient.

[0020] Preferably, the overall network is tuned by using a small sample image of the support set in the test set to train all parameters of the network, and the sample image of the query set in the test set is used to evaluate the training effect.

[0021] The invention provides a remote sensing image target detection method under a small number of labeling conditions, which has the following advantages:

[0022] (1) The meta-learning training method can learn the common characteristics of tasks with the same distribution through existing data. After meta-learning, the detector can adapt to new tasks in small sample detection tasks by using only a small amount of labeled data for gradient descent updates.

[0023] (2) The region recommendation network can mine the region that may contain the target based on the image feature information. Therefore, it is the most important part of the detection framework and the part that is most sensitive to the target features. Meta-learning only the region recommendation network can simplify the training complexity and obtain better training results. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0025] Figure 1 It is a flow chart of a remote sensing image target detection method under a small number of marking conditions provided by the present invention;

[0026] Figure 2 It is a model structure diagram of a remote sensing image target detection method under a small number of marking conditions provided by the present invention;

[0027] Figure 3 It is a detection result diagram of a remote sensing image target detection method under a small number of marking conditions provided by the present invention; DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0029] like Figure 1The present invention provides a remote sensing image target detection method under a small number of labeling conditions, and the method steps are as follows:

[0030] Step S1: Divide the data set into a pre-training set, a meta-training set and a test set, and divide the meta-training set and the test set into their respective support sets and query sets;

[0031] Step S2: Initialize network parameters using the pre-training set;

[0032] Step S3: Use the support set in the meta-training set to perform meta-support training on the region recommendation network;

[0033] Step S4: Perform meta-query using the query set in the meta-training set;

[0034] Step S5: according to the error in the meta-query, the back propagation algorithm is used to update the parameters of the region recommendation network;

[0035] Step S6: Use the support set in the test set to fine-tune the entire network;

[0036] Step S7: Calculate the detection results of the query set in the test set.

[0037] By pre-training the detection framework and performing MAML meta-learning on the region recommendation network, the network reserves meta-knowledge of tasks with the same distribution. The samples in the support set of the test set are used as the training set for the small sample task to fine-tune the overall network, so that it can quickly master the current detection task and complete the small sample training.

[0038] As a remote sensing image target detection method under a small number of labeling conditions in an embodiment of the present invention, further, its model structure diagram is as follows: Figure 2 As shown in the figure, the feature extraction backbone network uses ResNet with a residual structure, and then enters the feature pyramid network, which can semantically expand the feature maps of different sizes generated by ResNet during the feature extraction process. The feature pyramid network upsamples the high-level feature map and adds it to the low-level feature map channel by channel, so that the low-level feature map not only retains the detailed features, but also has high-level abstract semantics. During detection, large-size feature maps are used to detect small targets, and small-size feature maps are used to detect large targets.

[0039] As a remote sensing image target detection method under a small amount of labeling conditions in an embodiment of the present invention, further, anchor frames of various size ratios of 1:1, 1:2, 2:1, 1:3, 3:1, 1:5 and 5:1 are set for each point on the feature map, and a region recommendation network composed of a full convolutional network is used for region prediction, and the anchor frame with the highest confidence in handling each feature point is regressed as a recommended region as an anchor frame in the detection stage. The detection network adjusts these anchor frames again to make them closer to the real boundary to obtain a detection result with better accuracy.

[0040] As a remote sensing image target detection method under a small amount of labeling conditions in an embodiment of the present invention, further, in the training stage, all data are divided into a pre-training set, a meta-training set and a test set, wherein the pre-training set and the meta-training set respectively contain different image samples of the same category, and the test set contains image samples of different categories from the pre-training set and the meta-training set; further, the pre-training set, meta-training set and test set are composed of their own support set and query set, wherein the support set and the query set contain different image samples of the same category.

[0041] Furthermore, the network is pre-trained using samples in the pre-training set through a gradient descent algorithm. Furthermore, the network structure includes a feature extraction backbone network composed of a convolutional neural network, a feature pyramid network for obtaining multi-size feature maps, a fully convolutional network responsible for region recommendation, and a fully convolutional network for refining the recommended region.

[0042] As a remote sensing image target detection method under a small number of labeling conditions in an embodiment of the present invention, further, in the meta-support training stage, the samples in the meta-training set are used as several detection tasks. In this embodiment, each detection task contains 3 categories, 1 sample per category in the support set, and 5 samples per category in the query set. Using the samples in the support set, the temporary parameters of each detection task are calculated by the gradient descent algorithm. Formula (1) is the calculation formula:

[0043]

[0044] Among them, θ is the pre-trained parameter of the region recommendation network, Γ i is the i-th detection task, θ i ′ is the temporary parameter of the i-th detection task, is the loss of the i-th detection task under parameter θ, α represents the learning rate, Represents the calculation of gradient.

[0045] Furthermore, in the meta-query stage, the samples of the query set in the meta-training set are used to calculate the temporary parameters θ of each detection task through the gradient descent algorithm. i Loss under ' And update the regional recommendation network parameters according to the error in the meta-query. The calculation formula is shown in formula (2):

[0046]

[0047] Among them, θ is the pre-trained parameter of the region recommendation network, p(Γ) is a number of identically distributed detection tasks, Γ i is the i-th detection task, θ i ′ is the temporary parameter of the i-th detection task, Re-parameter θ for the i-th detection task i ′, β represents the learning rate, Represents the calculation of gradient.

[0048] As a remote sensing image target detection method under a small number of labeling conditions in an embodiment of the present invention, further, after the above steps, the meta-knowledge of the same distribution detection task is saved in the regional recommendation network, and small sample learning is performed on this basis using the samples in the test set, which can adapt to the detection task in the test set. Specifically, the test set of this embodiment contains samples of 3 categories, wherein the support set contains 1 sample of each category, and the query set contains 5 samples of each category. The small sample sample images of the support set in the test set are used to tune and train all the parameters of the network, and the sample images of the query set in the test set are used to evaluate the training effect. The detection results are as follows: Figure 3 shown.

[0049] This paper proposes a remote sensing image target detection method under the condition of a small number of labels. The model-independent meta-learning method MAML is used to meta-learn the region recommendation network, so that it has the meta-knowledge of region recommendation for tasks with the same distribution, so that the entire detection network can quickly adapt to new detection tasks after fine-tuning with a small number of training samples.

[0050] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, and do not constitute a limitation of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or the equivalent forms of such scope and boundaries.

Claims

1. A method for remote sensing image target detection under a small number of label conditions, characterized in that: The method comprises the steps of: Divide the dataset into pre-training set, meta-training set and test set, and divide the meta-training set and test set into their respective support set and query set; Initializing network parameters using a pre-training set; the network structure includes a feature extraction backbone network composed of a convolutional neural network, a feature pyramid network for obtaining multi-size feature maps, a full convolutional network for region recommendation, and a full convolutional network for refining the recommended regions; Meta-support training of the region proposal network using the support set in the meta-training set; Perform meta-queries using the query set in the meta-training set; Based on the error in the meta-query, the back-propagation algorithm is used to update the parameters of the region recommendation network. Fine-tune the entire network using the support set in the test set; Compute the detection results of the query set in the test set.

2. The method for remote sensing image target detection under a small number of labeling conditions according to claim 1, characterized in that: The pre-training set and meta-training set respectively contain different image samples of the same category, and the test set contains image samples of different categories from the pre-training set and the meta-training set.

3. The remote sensing image target detection method under a small amount of labeling conditions according to claim 2 is characterized in that: The pre-training set, meta-training set and test set are composed of their respective support sets and query sets, wherein the support set and query set contain different image samples of the same category.

4. The method for remote sensing image target detection under a small number of labeling conditions according to claim 1, characterized in that: The network parameters are initialized and the network is pre-trained using the samples in the pre-training set through a gradient descent algorithm.

5. The remote sensing image target detection method under a small amount of labeling conditions according to claim 1 is characterized in that: The meta-support training uses samples in the meta-training set as several detection tasks, and uses samples in the support set to calculate the temporary parameters of each detection task through a gradient descent algorithm. The calculation formula is: Among them, θ is the pre-trained parameter of the region recommendation network, Γ i is the i-th detection task, θ i ' is the temporary parameter of the i-th detection task, is the loss of the i-th detection task under parameter θ, α represents the learning rate, ▽ . Represents the calculation of gradient.

6. The method for remote sensing image target detection under a small amount of labeling according to claim 5, characterized in that: The meta-query uses the samples of the query set in the meta-training set and calculates the temporary parameter θ for each detection task through the gradient descent algorithm. i 'Loss under 7. The method for remote sensing image target detection under a small number of labeling conditions according to claim 1, characterized in that: The calculation formula for updating the regional recommendation network parameters according to the error in the meta-query is as follows: Among them, θ is the pre-trained parameter of the region recommendation network, p(Γ) is a number of identically distributed detection tasks, Γ i is the i-th detection task, θ i ' is the temporary parameter of the i-th detection task, Re-parameter θ for the i-th detection task i ', β represents the learning rate, ▽ . Represents the calculation of gradient.

8. The method for remote sensing image target detection under a small amount of labeling according to claim 1, characterized in that: The fine-tuning of the overall network uses a small sample image of the support set in the test set to tune all parameters of the network, and uses sample images of the query set in the test set to evaluate the training effect.

Images