A scene classification method for small sample and high resolution remote sensing images

Abstract

The present invention discloses a small sample high-resolution remote sensing image scene classification method. The proposed method mainly includes three parts: first, the source domain and target domain training sample sets are prepared, and appropriate images are selected from different remote sensing scene data sets to construct a source training sample set with a large number of samples and a target domain training sample set with a small number of samples, respectively, to simulate the situation of small sample classification; secondly, AlexNet is selected as the DCNN main network, and the deep convolution feature extraction of the sample image is performed in combination with the FPN network layer fusion strategy; finally, the improved single sample weighted distribution adaptation algorithm WS-BDA is used to realize the feature distribution migration from the source domain to the target domain, and the target classifier is trained using the migrated feature data to complete the classification of remote sensing scene images.

Description

Technical Field

[0001] The invention relates to a small sample high spatial resolution remote sensing image scene classification method, belonging to the technical field of image processing and pattern recognition. Background Art

[0002] Scene classification is an important research topic in the field of remote sensing image processing. It has been widely used in spatial modeling, land planning, agricultural surveys, disaster monitoring and other fields. Compared with ordinary images, high spatial resolution remote sensing images contain richer spatial, shape, texture and spectral information, and the imaging results are easily affected by factors such as shooting angle, shooting time and weather interference. It is very difficult to extract discriminative features. In addition, when completing the actual remote sensing image classification task, it can be found that there are very few labeled samples available for training for certain land object categories, which leads to insufficient classification accuracy. At this time, resampling and labeling for specific scenes is time-consuming and costly, so the research on remote sensing scene classification under small sample conditions becomes very meaningful.

[0003] First, for remote sensing scene classification under small sample conditions, when extracting features, the spatial information, shape information, spectral information, and texture information of the image are all important references for feature representation. These information extracted based on pixel level belong to low-level features and cannot meet higher requirements such as semantic analysis. In order to bridge the "semantic gap" between image features at different levels, high-order feature encoding methods such as Bag of Words (BoWs), SPM (Spatial Pyramid Matching) model, LLC (Locality-constrained Linear Coding) model, etc., have become a hot topic of research to obtain descriptions with semantic information. Since then, with the development of deep learning, the use of deep convolutional neural networks (DCNN) containing rich hierarchical information to extract differentiated image features has occupied the field of vision of researchers. DCNN imitates the biological mechanism of human recognition of images. By designing the convolutional layer as the basic structure, the image is scanned from top to bottom and from left to right to obtain the corresponding feature map. Similar to the structure of the human visual cortex, the shallower convolutional layers in the convolutional neural network obtain local detail features through a smaller receptive field, and the deeper convolutional layers can capture abstract and complex information of the target by increasing the scale of the receptive field, thereby obtaining feature representations of different scales of the image. However, it should be noted that the image can obtain multi-level features after passing through DCNN, in which the feature details of the lower network layers are clear but less semantic, while the features of the higher network layers are semantically rich but the detail information is coarse. When extracting image features through DCNN, the output of the highest layer network is usually selected as the feature representation, thus ignoring the detailed information of the bottom layer of the image.

[0004] Secondly, for the design of remote sensing scene classifiers under small sample conditions, there are currently many algorithms with excellent performance, such as SVM, K-Nearest Neighbors (KNN), Extreme Learning Machine (ELM), etc. Although these traditional machine learning classifiers can achieve good performance, they also have obvious disadvantages. They need to strictly meet two prerequisites: (1) the training samples and test samples come from the same feature space and meet the same data distribution; (2) a sufficient number of calibration samples are required to participate in the training, and finally a model with excellent performance can be obtained. Even if these conditions can be met, when the sample data distribution changes or the output scene of the model changes, the original model cannot be applied to the new scene and needs to be retrained and learned, which has poor flexibility. In recent years, the emerging transfer learning theory has just relaxed the above strict restrictions. First, in the absence of labeled data, transfer learning can use labeled data in related fields to help build the target model. In addition, when the data distribution is not completely consistent or the application scenario of the model has changed to a certain extent, transfer learning can select similar fields for adaptive learning or fine-tune the existing universal model, so as to complete the target task efficiently and conveniently. In transfer learning, the field where the existing knowledge is located is called the source domain, and the field where the new knowledge to be learned is the target domain. The core of transfer learning research is how to maximize the use of similar fields to transfer knowledge when the target domain and task are given.

[0005] In summary, in order to improve the scene classification performance of small sample high-resolution remote sensing images, the present invention proposes a small sample remote sensing scene classification method based on deep feature extraction and sample weighted distribution adaptive migration. The deep features of the scene image are extracted by an improved deep learning network, and the target classification task is completed by effectively migrating the features of sample images in different fields. Summary of the invention

[0006] Purpose of the invention: Aiming at the problem of small sample high spatial resolution remote sensing image scene classification, a small sample remote sensing scene classification method based on deep feature extraction and sample weighted distribution adaptation migration is proposed. First, the source domain and target domain training sample sets are prepared. An appropriate amount of images are selected from different remote sensing scene data sets to construct a large number of source training sample sets and a small number of target domain training sample sets, respectively, to simulate the situation of small sample classification; secondly, AlexNet is selected as the main DCNN network, and the deep convolution features of the sample images are extracted in combination with the FPN network layer fusion strategy; finally, a single sample weighted distribution adaptation algorithm WS-BDA is improved to realize the feature distribution migration from the source domain to the target domain, and the target classifier is trained using the migrated feature data to complete the classification of remote sensing scene images.

[0007] Technical solution: The present invention proposes a small sample high-resolution remote sensing image scene classification method, which includes the following steps:

[0008] (1) First, the source domain and target domain training sample sets are prepared. Appropriate amounts of images are selected from different remote sensing scene datasets to construct the source training sample set and the target domain training sample set, respectively, to simulate the situation of small sample classification.

[0009] (2) AlexNet is selected as the main DCNN network, and the FPN network layer fusion strategy is combined to extract deep convolution features of sample images;

[0010] (3) A single sample weighted distribution adaptation algorithm WS-BDA is proposed to achieve feature distribution migration from the source domain to the target domain, and the migrated feature data is used to train the target classifier to complete the classification of remote sensing scene images.

[0011] Furthermore, in step (2), AlexNet is selected as the main DCNN network, and the deep convolution features of the sample image are extracted by combining the FPN network layer fusion strategy. The specific method is as follows:

[0012] First, the image is input into AlexNet, and three convolution feature maps {C1, C2, C3} of different scales are obtained from the bottom up; starting from the highest layer feature map P3 = C3, P3 is upsampled, and C2 is reduced in dimension using a 1×1 convolution kernel. The two results of P3 upsampling and C2 dimension reduction are added together through lateral connection, and then a 3×3 convolution operation is performed to obtain P2 of the next layer; then, P2 is upsampled, and C1 is reduced in dimension using a 1×1 convolution kernel. The two results of P2 upsampling and C1 dimension reduction are added together through lateral connection, and then a 3×3 convolution operation is performed to obtain P1 of the next layer; finally, P1 is directly input into the fully connected layer of AlexNet, and according to the parameter settings of the network layer, the output of Alene's fc7 layer is selected as the deep feature representation of the image;

[0013] Since the dimension of the output feature of the fc7 layer of AlexNet is as high as 4096, PCA whitening is performed on the output feature of the fc7 layer to reduce the dimension. Let the output feature matrix of the fc7 layer be Among them, n represents the number of eigenvectors contained in the feature matrix of the fc7 layer, and d represents the dimension of each eigenvector. The process of implementing PCA whitening is as follows:

[0014] First, decentralize the feature dataset: Among them, mean(·) is the mean function;

[0015] Second, calculate the covariance matrix of F: Perform singular value decomposition on ∑ and obtain the eigenvalue λ i and the corresponding eigenvector v i , select the first k eigenvectors with the largest eigenvalues ​​to form the whitening transformation matrix W;

[0016] Third, use W to transform the feature data: F_pw = W T *F, reduce the correlation between data;

[0017] Fourth, use the scaling operator 1 / (λ i +ζ), where ζ is a parameter set to ensure that the denominator is not 0 and is set to 10 -5 , the variance of the reduced dimension data F_pw is normalized, that is, the variance of the normalized data is 1, and the normalized F_pw feature set is represented as fea.

[0018] Furthermore, in step (3), a distribution adaptation algorithm WS-BDA based on single sample weighting is proposed to achieve feature distribution migration from the source domain to the target domain, and the target classifier is trained using the migrated feature data to complete the classification of remote sensing scene images. The specific method is as follows:

[0019] Assume that the source domain sample feature data The corresponding label set is Target domain sample data And the corresponding label set Among them, m<<n, 1≤i≤m,1≤j≤n, C is the number of categories of sample labels, X=[X s ;X t ] is the total sample set, X s With X t From different data sets, the marginal probability distribution P of the two s (x s )≠P t (x t ), conditional probability distribution P s (y s |x s )≠P t (y t |x t );

[0020] The balanced distribution adaptation algorithm is used to estimate the source domain D s and the target domain D t The distribution distances between fields are as follows:

[0021] Distance(D s ,D t )=(1-μ)Distance(P(x s ),P(xt ))+μDistance(P(y s |x s ),P(y t |x t )) (1)

[0022] Among them, μ∈[0,1] represents the balance factor between the two data distributions. When μ→1, it indicates that the conditional distribution adaptation has a greater impact than the preset condition; when μ→0, it indicates that the marginal distribution adaptation meets the preset condition. s ) and P(y s |x s ) represent the marginal probability distribution and conditional probability distribution of the source domain sample space, P(x t ) and P(y t |x t ) represent the marginal probability distribution and conditional probability distribution of the target domain sample space respectively. The BDA algorithm uses the maximum mean difference to quantify the distribution distance between data domains, which is specifically expressed as formula (2):

[0023]

[0024] Among them, H represents the reproducible Hilbert space, C represents the total number of categories, u and v represent the number of samples in the source domain and the target domain respectively. Respectively represent the sample subsets of category c in the source domain and the target domain, u c ,v c is the corresponding number of samples, transforming formula (2) into the optimization problem shown in formula (3):

[0025]

[0026] Among them, X is the total set of samples, A is the feature transformation matrix to be found, λ is the regularization factor, I represents the identity matrix, E represents the matrix of all ones, tr(·) represents the matrix trace, ||·|| F represents the F norm, M0 and M c They are respectively MMD matrices, which are defined as:

[0027]

[0028]

[0029] Use the Lagrange multiplier method to transform the solution of A into the following eigenvalue decomposition problem:

[0030]

[0031] Where Φ represents the Lagrange multiplier;

[0032] The above BDA algorithm is improved by the cosine similarity measurement method. The similarity between a single sample and the global mean point of the sample space is first calculated, and then the distribution weight within the sample space is calculated based on this. Specifically, for the preprocessed feature data set fea, its mean vector is fea_mean, and the weight matrix corresponding to the training sample set is calculated by the following formula:

[0033]

[0034] The obtained weight matrix β is used to optimize the representation of the MMD distance, and the new target expression to be optimized is expressed as:

[0035]

[0036] Among them, A is the required feature transformation matrix, λ is the regularization factor, I represents the identity matrix, E represents the matrix of all ones, tr(·) represents the matrix trace, ||·|| F represents the F norm, M'0 and M c ' are the optimized MMD matrices, which are defined as:

[0037]

[0038]

[0039] Among them, in M c 'middle, fea (c) with fea_mean (c) They are the sample subsets with category label c in the sample feature set and their mean vectors; (M'0) ij is the i-th row and j-th column element of the M'0 matrix, (M c ') ij M c 'The element in the i-th row and j-th column of the matrix, β s(i) is the weight corresponding to the i-th sample in the source domain, β t(i) is the weight corresponding to the i-th sample in the target domain, is the weight corresponding to the i-th sample belonging to the c-th class in the source domain, is the weight corresponding to the i-th sample belonging to the c-th class in the target domain; x i and x j Represents (M'0) ij Corresponding to the i-th and j-th samples, X s represents the source domain sample set, X t represents the target domain sample set, and Indicates that (M c ')ij The corresponding i-th and j-th samples of the c-th class are, represents the source domain sample set of the cth category, represents the target domain sample set of the cth class;

[0040] For the optimization problem of formula (8), the characteristic migration matrix A is solved by the Lagrange multiplier method:

[0041]

[0042] Where Φ represents the Lagrange multiplier. The partial derivative of A in equation (11) is calculated and set to 0 to obtain:

[0043]

[0044] The optimization problem is finally transformed into:

[0045]

[0046] At this time, the optimal feature transfer matrix A can be obtained through generalized eigenvalue decomposition. After obtaining A, the feature data in the source domain and the target domain are respectively mapped to the shared feature subspace, so as to realize the migration and adaptation of the feature distribution from the source domain to the target domain, and use the migrated feature data to train the target classifier, and finally complete the classification of the remote sensing scene image.

[0047] Beneficial effects: Compared with the prior art, the above technical solution adopted by the present invention has the following beneficial effects:

[0048] (1) At the image feature extraction level, the present invention applies the FPN network layer fusion strategy to the scene image classification problem, and fuses it with AlexNet to extract the deep features of the image so that it contains sufficient detail information and semantic information.

[0049] (2) In terms of transfer learning algorithm design, the present invention improves the current best distribution adaptation algorithm, considers the distribution differences between single samples and assigns appropriate weights in the quantitative calculation of sample space differences, and directly uses the real labels of the target domain training samples in the classifier training process, which improves the performance of feature distribution adaptation and the efficiency of the algorithm. Experiments show that the method proposed in the present invention performs well in remote sensing scene classification problems under small sample conditions, and is effective and robust. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a framework diagram of an embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of the horizontal connection during the fusion of the AlexNet network and the FPN of the present invention;

[0052] Figure 3 The confusion matrix diagram of the classification algorithm proposed by the present invention when the proportion of target domain training samples is 10%;

[0053] Figure 4 This is a comparison result diagram of different feature classification algorithms when the target domain training sample ratio of the present invention is 10%;

[0054] Figure 5 This is a comparison result diagram of different classifiers when the target domain training sample ratio of the present invention is 10%. DETAILED DESCRIPTION

[0055] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0056] like Figure 1 As shown, the present invention proposes a small sample high-resolution remote sensing image scene classification method, the method comprising the following steps:

[0057] The first step is to prepare the source domain and target domain training sample sets. Select appropriate amounts of images from different remote sensing scene data sets to construct the source training sample set and the target domain training sample set, respectively, to simulate the situation of small sample classification. The operation process is as follows:

[0058] The famous UC-Merced remote sensing scene dataset was selected, which contains 21 different categories of remote sensing scene images, including airports, farms, residential areas, etc. There are 100 images in each category. The spatial resolution of all samples in this image set is 30 cm and the size is 256×256 pixels.

[0059] The second dataset used in the experiment is the NWPU-RESISC45 remote sensing scene dataset, which covers 45 different remote sensing scenes, including airports, bushes, beaches, etc., with 700 sample images in each category. The images in this dataset are all 256×256 pixels in size, with spatial resolution ranging from 20 cm to 30 meters.

[0060] Since the scene categories in the two data sets are not completely corresponding, the present invention selects 19 common image categories from the UC-Merced and NWPU-RESISC45 data sets to construct the target domain and source domain sample sets respectively. The scenes involved include: airports, baseball fields, beaches, bushes, dense residential areas, forests, highways, golf courses, ports, intersections, medium-sized residential areas, mobile housing areas, overpasses, parking lots, rivers, runways, sparse residential areas, oil tanks and tennis courts. There are 700 images for each type of scene in the NWPU-RESISC data set. The present invention randomly selects 100 images for the involved categories to construct the final source domain sample set. Finally, there are 1900 NWPU-RESISC images in the source domain and 1900 UC-Merced images in the target domain.

[0061] In the second step, AlexNet is selected as the main DCNN network, and the FPN network layer fusion strategy is combined to extract deep convolution features from the sample image. The operation process is as follows:

[0062] First, the image is fed into AlexNet (see Figure 1 ), and obtain three convolution feature maps {C1, C2, C3} of different scales from bottom to top; then, starting from the highest layer feature map P3 (P3 = C3), upsample P3 and use a 1×1 convolution kernel to reduce the dimension of C2. The above two results are added together through horizontal connection (see Figure 2 ) and then perform a 3×3 convolution operation to obtain P2 of the next layer. Subsequently, P1 is obtained in the same way; finally, the output of P1 is input into the fully connected layer of AlexNet, and according to the parameter setting of the network layer, the output of the fc7 layer of AlexNet is selected as the deep feature representation of the image.

[0063] Since the dimension of the output feature of the fc7 layer of AlexNet is as high as 4096 dimensions, the present invention performs PCA whitening and dimensionality reduction on the sample feature data set. Assume that the feature data matrix to be processed is The process of implementing PCA whitening is as follows:

[0064] First, decentralize the feature dataset: Where mean(·) is the mean function;

[0065] Second, calculate the covariance matrix of F: Perform singular value decomposition on ∑ and obtain the eigenvalue λ i and the corresponding eigenvector v i , select the feature vectors to form the whitening transformation matrix W according to the dimensionality reduction requirements;

[0066] Third, use W to transform the feature data: F_pw = WT *F, reduce the correlation between data;

[0067] Fourth, use the scaling operator 1 / (λ i +eps), where eps is a parameter set to ensure that the denominator is not 0, normalizes the variance of the reduced-dimensional data, further eliminates redundancy between feature data, and standardizes the feature data form. The present invention represents the final feature set as fea.

[0068] The third step is to improve a single sample weighted distribution adaptation algorithm WS-BDA to achieve feature distribution migration from the source domain to the target domain, and use the migrated feature data to train the target classifier to complete the classification of remote sensing scene images. The operation process is as follows:

[0069] The research of this paper is based on remote sensing scene classification under small sample conditions. First, the problem is defined as follows: Suppose there is sample data in the source domain The corresponding label set is Target domain sample data And the corresponding tag set Where m<<n, 1≤i≤m,1≤j≤n, C is the number of categories of sample labels, X=[X s ;X t ] is the total sample set. s With X t They come from different data sets, so there are obvious distribution differences, that is, the marginal probability distributions P s (x s )≠P t (x t ), conditional probability distribution P s (y s |x s )≠P t (y t |x t ).

[0070] There are two main types of feature space distribution: marginal probability distribution and conditional probability distribution. The Balanced Distribution Adaptation (BDA) algorithm considers both data distribution forms at the same time, adjusts the weights of the two distributions in the overall distribution of the sample space according to the difference between the domains, and finally optimizes the distribution distance between the domains to perform feature adaptation and obtain good migration effects. Formula (1) shows that the BDA algorithm estimates the source domain D s and the target domain D t The expression of the distribution distance between domains is:

[0071] Distance(D s,D t )≈(1-μ)Distance(P(x s ),P(x t ))+μDistance(P(y s |x s ),P(y t |x t )) (1)

[0072] Among them, μ∈[0,1] represents the balance factor between the two data distributions. When μ→1, it indicates that the conditional distribution adaptation has a greater impact than the preset condition; when μ→0, it indicates that the marginal distribution adaptation meets the preset condition. s ) and P(y s |x s ) represent the marginal probability distribution and conditional probability distribution of the source domain sample space, P(x t ) and P(y t |x t ) represent the marginal probability distribution and conditional probability distribution of the target domain sample space respectively. The BDA algorithm uses the maximum mean difference to quantify the distribution distance between data domains, which is specifically expressed as formula (2):

[0073]

[0074] Among them, H represents the reproducible Hilbert space, C represents the total number of categories, u and v represent the number of samples in the source domain and the target domain respectively. Respectively represent the sample subsets of category c in the source domain and the target domain, u c ,v c is the corresponding number of samples, transforming formula (2) into the optimization problem shown in formula (3):

[0075]

[0076] Among them, X is the total set of samples, A is the feature transformation matrix to be found, λ is the regularization factor, I represents the identity matrix, E represents the matrix of all ones, tr(·) represents the matrix trace, ||·|| F represents the F norm, M0 and M c They are respectively MMD matrices, which are defined as:

[0077]

[0078]

[0079] Use the Lagrange multiplier method to transform the solution of A into the following eigenvalue decomposition problem:

[0080]

[0081] Here, Φ represents the Lagrange multiplier.

[0082] When the BDA algorithm measures the distribution distance between domains with the help of the Maximum Mean Discrepancy (MMD), it assumes that the impact of a single sample in a domain on the sample space is exactly the same. However, in fact, there must be distribution differences of varying sizes between different samples in the same domain, and the contribution of independent samples to the overall distribution of the sample space should be distinguished. Therefore, the present invention takes the distribution differences between single samples into account in the calculation of the MMD distance of the sample space distribution, thereby more accurately quantifying the distribution distance between domains and further improving the effect of feature transfer learning.

[0083] In addition, the BDA algorithm makes full use of the training samples in the source domain during feature transfer and classifier training, while the target domain training samples are represented by pseudo-labels, so continuous iteration is required to obtain a classification model that fits the training samples. In fact, the target domain training samples themselves carry real labels. If they are applied in the classifier learning process, not only can the accuracy of the target prediction model be improved, but also the iterative optimization process required for assigning pseudo-labels to the target domain training samples is eliminated, thereby improving the efficiency of transfer learning. Therefore, in the process of classifier learning, the present invention directly uses the real labels of the target domain training samples, optimizing the overall performance of feature distribution adaptation.

[0084] Finally, the present invention uses the cosine similarity measurement method to calculate the similarity between a single sample and the global mean point of the sample space, and uses this as its distribution weight within the sample space. For the preprocessed feature data set fea, its mean vector fea_mean, then the weight matrix corresponding to the training sample set will be calculated by the following formula:

[0085]

[0086] Using the obtained weight matrix β to optimize the representation of the MMD distance, the target expression to be optimized of the algorithm of the present invention can be expressed as follows:

[0087]

[0088] Among them, A is the feature transformation matrix to be found, λ is the regularization factor, I represents the identity matrix, E represents the matrix of all ones, tr(·) represents the matrix trace, ||·|| F represents the F norm, M'0 and M c ' are the optimized MMD matrices, which are defined as:

[0089]

[0090]

[0091] Among them, in M c 'middle, fea (c) with fea_mean (c) They are the sample subsets with category label c in the sample feature set and their mean vectors; (M'0) ij is the i-th row and j-th column element of the M'0 matrix, (M c ') ij M c 'The element in the i-th row and j-th column of the matrix, β s(i) is the weight corresponding to the i-th sample in the source domain, β t(i) is the weight corresponding to the i-th sample in the target domain, is the weight corresponding to the i-th sample belonging to the c-th class in the source domain, is the weight corresponding to the i-th sample belonging to the c-th class in the target domain. i and x j Represents (M'0) ij Corresponding to the i-th and j-th samples, X s represents the source domain sample set, X t represents the target domain sample set, and Indicates that (M c ') ij The corresponding i-th and j-th samples of the c-th class are, represents the source domain sample set of the cth category, represents the target domain sample set of class c.

[0092] For the optimization problem of formula (8), the characteristic migration matrix A can be solved by the Lagrange multiplier method:

[0093]

[0094] Where Φ represents the Lagrange multiplier. Taking the partial derivative of equation (11) with respect to A and setting it to 0, we get:

[0095]

[0096] The optimization problem is finally transformed into:

[0097]

[0098] At this time, the optimal feature transfer matrix A can be obtained by generalized eigenvalue decomposition. After obtaining A, the feature data in the source domain and the target domain can be mapped to the shared feature subspace respectively, so that the feature distribution migration and adaptation from the source domain to the target domain can be realized, and the target classifier can be trained using the migrated feature data to finally complete the classification of the remote sensing scene image. The improved algorithm is recorded as WS-BDA in the present invention, and the following Table 1 describes the algorithm in detail:

[0099] Table 1 WS-BDA algorithm description

[0100]

[0101] The algorithm proposed by the present invention is evaluated by the following experiments:

[0102] Firstly, the performance of the classification algorithm proposed in the present invention is verified. Figure 3 The confusion matrix of the algorithm proposed in the present invention when the target domain training sample ratio is 10% is shown. It can be seen that the features designed in the present invention are very accurate in classifying most scenes, and some categories such as airports, bushes, parking lots, etc. have reached more than 95%.

[0103] Secondly, the deep features (denoted as Proposed) obtained by the fusion of AlexNet and FPN strategy proposed in the present invention are compared with some other excellent features, including the deep features extracted only using the AlexNet network (denoted as AlexNet), mature features HOG, CLBP and the serial fusion feature HOG+CLBP of HOG and CLBP. HOG and CLBP features can respectively characterize the original shape and texture information of the remote sensing scene image. HOG+CLBP improves the diversity of features by effectively fusing the two features. In addition, the BoVW feature is selected to represent the encoding features of the image for comparison. In order to fairly analyze the classification performance of all features, the classification algorithms used are the WS-BDA algorithm proposed in the present invention. The experiment was carried out under the condition that the ratio of target domain training samples to test samples was 10%:90%. At this time, the number of samples participating in the training in the target domain is much different from the number of training samples in the source domain, which can fully simulate the small sample environment. Figure 4 The producer accuracy PA curve of the classification algorithm based on different features is given ( Figure 4 (a)), user accuracy UA curve ( Figure 4 (b)), OE curve of omission error ( Figure 4 (c)) and the misclassification error CE curve ( Figure 4 (d)) Comparison chart. Figure 4It can be seen that: whether it is the PA or UA indicator, the features of the present invention have the best classification results in almost all categories and are very stable. In contrast, other features may be more accurate than the method of the present invention when classifying a certain category of scenes, but the overall stability cannot be guaranteed. For the OE and AE indicators, the features proposed by the present invention also maintain the lowest error value in most categories.

[0104] Secondly, the improved WS-BDA algorithm (denoted as Proposed) of the present invention is compared with some other excellent classification algorithms, including the excellent transfer learning algorithms BDA and JDA, and the good performance of random forest (RF) and KNN in non-transfer learning classifiers. Among them, JDA is the first distribution adaptation algorithm for the joint distribution of marginal probability and conditional probability between domains, and the BDA algorithm is actually an extension of the JDA algorithm. Random forest performs well in high-dimensional data classification, and the KNN algorithm has excellent performance in nonlinear classification applications. They are often used to solve remote sensing scene classification problems. In order to fairly analyze the performance of each algorithm, all classifiers are experimented with the deep features proposed by the present invention. Similarly, the experiment will be conducted when the proportion of training samples in the target domain is 10%. Figure 5 The producer accuracy PA curve based on the classifier is given ( Figure 5 (a)), User Accuracy UA Polyline ( Figure 5 (b)), OE curve of omission error ( Figure 5 (c)) and the misclassification error CE curve ( Figure 5 (d)) Comparison chart. Figure 5 It can be seen that: in all cases, for the PA and UA indicators, the classification results of the algorithm of the present invention are all located at the top of the line graph, indicating that for each type of scene, the present invention can obtain the best classification accuracy and is very stable. Correspondingly, the OE and AE curves obtained by the algorithm of the present invention are always at the bottom of the line graph, with the lowest error rate. In comparison, the test results of other classification algorithms are obviously inferior to the improved algorithm of the present invention.

Claims

1. A small sample high-resolution remote sensing image scene classification method, characterized in that: The method comprises the following steps: (1) First, the source domain and target domain training sample sets are prepared. Appropriate amounts of images are selected from different remote sensing scene datasets to construct the source training sample set and the target domain training sample set, respectively, to simulate the situation of small sample classification. (2) AlexNet is selected as the main DCNN network, and the FPN network layer fusion strategy is combined to extract deep convolution features of sample images; (3) Design a single sample weighted distribution adaptation algorithm WS-BDA to achieve feature distribution migration from the source domain to the target domain, and use the migrated feature data to train the target classifier to complete the classification of remote sensing scene images; In step (3), a distribution adaptation algorithm WS-BDA based on single sample weighting is designed to achieve feature distribution migration from the source domain to the target domain, and the target classifier is trained using the migrated feature data to complete the classification of remote sensing scene images. The specific method is as follows: Assume that the source domain sample feature data The corresponding label set is Target domain sample data And the corresponding tag set Among them, m<<n, C is the number of categories of sample labels, X = [X s ;X t ] is the total sample set, X s With X t From different data sets, the marginal probability distribution P of the two s (x s )≠P t (x t ), conditional probability distribution P s (y s |x s )≠P t (y t |x t ); The balanced distribution adaptation algorithm is used to estimate the source domain D s and the target domain D t The distribution distances between fields are as follows: Distance(D s ,D t )=(1-μ)Distance(P(x s ),P(x t ))+μDistance(P(y s |x s ),P(y t |x t )) (1) Among them, μ∈[0,1] represents the balance factor between the two data distributions. When μ→1, it indicates that the conditional distribution adaptation has a greater impact than the preset condition; when μ→0, it indicates that the marginal distribution adaptation meets the preset condition. s ) and P(y s |x s ) represent the marginal probability distribution and conditional probability distribution of the source domain sample space, P(x t ) and P(y t |x t ) represent the marginal probability distribution and conditional probability distribution of the target domain sample space respectively. The BDA algorithm uses the maximum mean difference to quantify the distribution distance between data domains, which is specifically expressed as formula (2): Among them, H represents the reproducible Hilbert space, C represents the total number of categories, u and v represent the number of samples in the source domain and the target domain respectively. Respectively represent the sample subsets of category c in the source domain and the target domain, u c ,v c is the corresponding number of samples, transforming formula (2) into the optimization problem shown in formula (3): Among them, X is the total set of samples, A is the feature transformation matrix to be found, λ is the regularization factor, I represents the identity matrix, E represents the matrix of all ones, tr(·) represents the matrix trace, ||·|| F represents the F norm, M0 and M c They are respectively MMD matrices, which are defined as: Use the Lagrange multiplier method to transform the solution of A into the following eigenvalue decomposition problem: Where Φ represents the Lagrange multiplier; The above BDA algorithm is improved by the cosine similarity measurement method. The similarity between a single sample and the global mean point of the sample space is first calculated, and then the distribution weight within the sample space is calculated based on this. Specifically, for the preprocessed feature data set fea, its mean vector is fea_mean, and the weight matrix corresponding to the training sample set is calculated by the following formula: The obtained weight matrix β is used to optimize the representation of the MMD distance, and the new target expression to be optimized is expressed as: Among them, A is the required feature transformation matrix, λ is the regularization factor, I represents the identity matrix, E represents the matrix of all ones, tr(·) represents the matrix trace, ||·|| F represents the F norm, M'0 and M c ' are the optimized MMD matrices, which are defined as: Among them, in M c 'middle, fea (c) with fea_mean (c) They are the sample subsets with category label c in the sample feature set and their mean vectors; (M'0) ij is the i-th row and j-th column element of the M'0 matrix, (M c ') ij M c 'The element in the i-th row and j-th column of the matrix, β s(i) is the weight corresponding to the i-th sample in the source domain, β t(i) is the weight corresponding to the i-th sample in the target domain, is the weight corresponding to the i-th sample belonging to the c-th class in the source domain, is the weight corresponding to the i-th sample belonging to the c-th class in the target domain; x i and x j Represents and (M'0) ij Corresponding to the i-th and j-th samples, X s represents the source domain sample set, X t represents the target domain sample set, and Indicates that (M c ') ij The corresponding i-th and j-th samples of the c-th class are, represents the source domain sample set of the cth category, represents the target domain sample set of the cth category; For the optimization problem of formula (8), the characteristic migration matrix A is solved by the Lagrange multiplier method: Where Φ represents the Lagrange multiplier. The partial derivative of A in equation (11) is calculated and set to 0 to obtain: The optimization problem is finally transformed into: At this time, the optimal feature transfer matrix A can be obtained through generalized eigenvalue decomposition. After obtaining A, the feature data in the source domain and the target domain are respectively mapped to the shared feature subspace, so as to realize the migration and adaptation of the feature distribution from the source domain to the target domain, and use the migrated feature data to train the target classifier, and finally complete the classification of the remote sensing scene image.

2. The small sample high-resolution remote sensing image scene classification method according to claim 1 is characterized in that: In step (2), AlexNet is selected as the main DCNN network, and the deep convolution features of the sample image are extracted by combining the FPN network layer fusion strategy. The specific method is as follows: First, the image is input into AlexNet, and three convolution feature maps {C1, C2, C3} of different scales are obtained from the bottom up; starting from the highest layer feature map P3 = C3, P3 is upsampled, and C2 is reduced in dimension using a 1×1 convolution kernel. The two results of P3 upsampling and C2 dimension reduction are added together through lateral connection, and then a 3×3 convolution operation is performed to obtain P2 of the next layer; then, P2 is upsampled, and C1 is reduced in dimension using a 1×1 convolution kernel. The two results of P2 upsampling and C1 dimension reduction are added together through lateral connection, and then a 3×3 convolution operation is performed to obtain P1 of the next layer; finally, P1 is directly input into the fully connected layer of AlexNet, and according to the parameter settings of the network layer, the output of Alene's fc7 layer is selected as the deep feature representation of the image; Since the dimension of the output feature of the fc7 layer of AlexNet is as high as 4096, PCA whitening is performed on the output feature of the fc7 layer to reduce the dimension. Let the output feature matrix of the fc7 layer be Among them, n represents the number of eigenvectors contained in the feature matrix of the fc7 layer, and d represents the dimension of each eigenvector. The process of implementing PCA whitening is as follows: First, decentralize the feature dataset: Among them, mean(·) is the mean function; Second, calculate the covariance matrix of F: Perform singular value decomposition on ∑ and obtain the eigenvalue λ i and the corresponding eigenvector v i , select the first k eigenvectors with the largest eigenvalues ​​to form the whitening transformation matrix W; Third, use W to transform the feature data: F_pw = W T *F, reduce the correlation between data; Fourth, use the scaling operator 1 / (λ i +ζ), where ζ is a parameter set to ensure that the denominator is not 0 and is set to 10 -5 , the variance of the reduced dimension data F_pw is normalized, that is, the variance of the normalized data is 1, and the normalized F_pw feature set is represented as fea.

Images