A subspace network-based few-shot remote sensing scene classification method

Abstract

The present application relates to a kind of based on subspace network's few sample remote sensing scene classification method comprising: according to the category of remote sensing scene image in remote sensing scene image data set, remote sensing scene image data set is divided into training set, verification set and test set;According to the training set, verification set and test set divided, training task, verification task and test task are constructed;Establish the few sample remote sensing scene classification model based on subspace network, according to the training task set loss function constructed;According to the loss function set, using the meta-learning training strategy based on task to the few sample remote sensing scene classification model based on subspace network is meta-trained, while training, through verification task, the few sample remote sensing scene classification model based on subspace network is meta-verified, the model of best performance is saved, after training and verification process are completed, the model of best performance saved is meta-tested by test task;The present application has better classification performance and higher stability.

Description

Technical Field

[0001] This invention belongs to the field of intelligent analysis of remote sensing images, and in particular relates to a few-sample remote sensing scene classification method based on subspace networks. Background Technology

[0002] Remote sensing scene classification aims to distinguish different scene categories by understanding the content of remote sensing images, and is widely used in fields such as natural disaster detection, urban planning, and land cover classification. In recent years, deep learning methods have achieved good results in remote sensing scene classification; however, these methods rely on a large number of labeled samples for training and cannot classify new categories of remote sensing scene images, exhibiting significant limitations. In contrast, few-shot learning methods only require a small number of labeled samples to classify images of new categories, which precisely compensates for the shortcomings of deep learning methods.

[0003] Currently, the few-shot learning approach has been widely applied in natural image classification. However, since remote sensing scene images and natural images are two different types of images, directly applying few-shot natural image classification methods to remote sensing images does not achieve good classification results. Compared with natural images, remote sensing scene images have the following characteristics: (1) Complex background: Remote sensing scene images are mostly taken at high altitudes and contain many ground features unrelated to the category, presenting a complex background. (2) High inter-class similarity: The similarity of objects in shape and texture makes remote sensing scene images of different categories appear similar. (3) Large intra-class differences: Due to the diversity of ground features and differences in lighting, scale, and angle, scene images of the same category have significant visual differences. These characteristics make the few-shot remote sensing scene classification task more challenging.

[0004] Few-shot remote sensing scene classification methods mainly involve the design of feature extraction and classification metric modules. Existing few-shot remote sensing scene classification methods focus primarily on feature extraction, introducing attention mechanisms or multi-scale feature extraction modules to extract distinctive image features, neglecting the guiding role of remote sensing scene semantics in feature extraction. Simultaneously, these methods also neglect the design of the classification metric module, often employing prototype-based classifiers. However, the prototype obtained by simply weighting the image features of each category retains a significant amount of background information in the image that is irrelevant to the category semantics, insufficient to represent the semantic commonalities of the categories. In other words, prototype-based classifiers are significantly negatively affected by irrelevant background information in the image, resulting in poor classification performance. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention proposes a few-sample remote sensing scene classification method based on subspace networks to effectively mitigate the negative impact of irrelevant background information and improve the classification effect of remote sensing scene images. The method includes the following steps:

[0006] S1: Divide the remote sensing scene image dataset into training set, validation set and test set according to the category of remote sensing scene images in the dataset;

[0007] S2: Construct training tasks, validation tasks, and test tasks based on the predefined training set, validation set, and test set;

[0008] S3: Establish a few-shot remote sensing scene classification model based on subspace networks, and set the loss function according to the constructed training task;

[0009] S4: Based on the set loss function, use the task-based meta-learning training strategy to perform meta-training on the few-shot remote sensing scene classification model based on subspace network. During training, perform meta-validation on the few-shot remote sensing scene classification model based on subspace network through validation tasks, and save the model with the best performance.

[0010] S5: After the training and validation process is completed, perform meta-testing on the best-performing model through a test task.

[0011] Furthermore, the division of the remote sensing scene image dataset into training, validation, and test sets includes:

[0012] Set the total categories as C total The remote sensing scene image dataset D is divided into training set D according to the category of remote sensing scene images. train Validation set D val and test set D test Training set D train Validation set D val and test set D test The categories of remote sensing scene images are C train C val and C test ,in, And C train ∪C val ∪C test =C total .

[0013] Furthermore, the construction of the training task, validation task, and testing task includes:

[0014] In training set D train Validation set D val and test set D testThe training, validation, and testing tasks are constructed in the same way. The training task is constructed by starting with the class set C of the training set. train N categories are randomly selected, and K+M remote sensing scene images are randomly selected from each category. The K remote sensing scene images from each category are used as the support set, and the remaining M remote sensing scene images are used as the query set. Similarly, the verification task and the test task are constructed.

[0015] Furthermore, the few-shot remote sensing scene classification model based on subspace networks includes: an image feature extraction module, a scene semantic guidance module, and a classifier module;

[0016] The image feature extraction module uses ResNet12 as its network structure to extract features from the input remote sensing scene image to obtain feature map F. m ;

[0017] The scene semantic guidance module includes: upper and lower branches and a feature aggregation module. The upper branch is used to aggregate features based on feature map F. m Obtain the scene embedding u′; the lower branch is used to represent the feature map F. m Aligning with the manifold shared with the scene embedding yields a new feature map F. m The feature aggregation module is used to embed the scene into u′ and the feature map F. m The semantic relation graph r is obtained by calculating spatial correlation; based on the semantic relation graph r, the feature graph F is... m The feature maps are then weighted and flattened to obtain the feature embedding z.

[0018] The classifier module is used to construct a standard subspace classifier based on the feature embeddings of samples in the query set and support set for each task, and to classify the samples in the query set using the standard subspace classifier.

[0019] Furthermore, in the upper branch of the scene semantic guidance module, the feature map F is first processed. m A global average pooling operation is performed to obtain a global feature aggregation representation u. Then, a learnable 1×1 convolutional layer is used to convolve the global feature aggregation representation u to obtain the scene embedding u′ of the remote sensing scene image. The scene embedding u′ can be expressed by the formula:

[0020] u′=σ α (GAP(F m ))

[0021] Where GAP represents global average pooling, σ α (·) represents a convolutional layer, and α is the corresponding parameter;

[0022] In the lower branch of the scene semantic guidance module, the feature map F mThe feature map F is obtained by performing a convolution operation. m The convolutional module used consists of 1×1 convolutional layers, batch normalization layers, and ReLU activation function layers, with feature map F. m ′ is represented as:

[0023] F m ′=σ β (F m )

[0024] Where, σ β (·) denotes the convolution module of the lower branch, and β is the corresponding parameter;

[0025] In the feature aggregation module of the scene semantic guidance module, the scene embedding u′ and feature map F of the remote sensing scene image are first used as the basis for the feature aggregation module. m The correlation between the features at each spatial location i and the scene semantic embedding u′ is calculated using a pointwise inner product method, and then normalization is performed to obtain the scene semantic relationship graph r. The correlation value at location i in the scene semantic relationship graph r is r. i Calculated using the following formula:

[0026]

[0027] in, Representing feature map F m The vector corresponding to position i in the spatial domain, H×W represents the feature map F. m The total number of spatial locations;

[0028] Using semantic relation graph r to analyze the original feature graph F m After performing a weighted update and flattening it into a feature vector, the feature embedding z of the remote sensing scene image is obtained, as follows:

[0029] z = Flatten(rF) m )

[0030] Flatten refers to the flattening operation.

[0031] Furthermore, the construction of the standard subspace classifier includes:

[0032] Step 101: Calculate the average feature embedding of the support set samples corresponding to each class in each task as the initial prototype for that class. For class c, its initial prototype μ c Represented as:

[0033]

[0034] Where K represents the number of samples supporting class c in the support set. The feature embedding of the i-th sample of class c in the support set of the task is represented.

[0035] Step 102: Subtract the feature embeddings of the support set samples for each category in the task from the initial prototype of that category to obtain the initial feature space of that category. For category c, its initial feature space is represented as:

[0036]

[0037] in, Represents the initial feature space of category c;

[0038] Step 103: Perform singular value decomposition on the initial feature space of each category in the task to obtain the left singular matrix. For category c, this operation is specifically represented as follows:

[0039]

[0040] Among them, the left singular matrix B c =[a1,a2,a3,a4,…,a n ] is an orthogonal matrix; for the left singular matrix B c The initial feature subspace of this category is obtained by truncation. For category c, its initial feature subspace is represented as:

[0041] P c =[a1,a2,a3,a4]

[0042] Among them, P c Let c represent the initial feature subspace of category c;

[0043] Step 104: Calculate the projected distance between the feature embeddings of samples in the query set of the task and the initial feature subspace of each class, which is expressed as:

[0044] d c (q i )=||(IM c (z) que,i -μ c )||2,q i ∈que

[0045] Where, d c (q i ) represents the sample q in the query set of the task. i The initial feature subspace P of category c c The projected distance, where que represents the query set of the task. ||·||2 represents the L2 norm, z que,i The sample q in the query set represents the task. i Feature embedding;

[0046] Step 105: Based on the projected distance of the feature embeddings of samples in the task's query set into the initial feature subspace of each category, calculate the probability that the samples in the task's query set belong to each category, which is expressed as:

[0047]

[0048] Where c∈{1,2,…,N}, N represents the total number of categories in the support set of the task, and m c (q i ) represents the sample q in the query set of the task. i The probability of belonging to category c;

[0049] Step 106: Based on the probability that samples in the query set belong to each category, fine-tune the initial prototypes for each category to obtain fine-tuned prototypes for each category. For category c, its fine-tuned prototype is represented as:

[0050]

[0051] Where, μ′ c This represents the fine-tuned prototype of category c;

[0052] Step 107: Subtract the feature embeddings of the support set samples for each class in the task from the fine-tuned prototype of that class to obtain the corrected feature space for that class. For class c, its corrected feature space is represented as:

[0053]

[0054] in, Represents the modified feature space of category c;

[0055] Step 108: Perform singular value decomposition on the modified feature space of each category in the task to obtain the left singular matrix. For category c, this operation is specifically represented as follows:

[0056]

[0057] The left singular matrix B c ′=[a1′,a2′,a3′,a4′,…,a n '] is an orthogonal matrix. For the left singular matrix B... c If we truncate the subspace to obtain the modified feature space of category c, then the modified feature space of category c is represented as follows:

[0058] P rc =[a1′,a2′,a3′,a4′]

[0059] Among them, P rc Represents the modified feature subspace of category c;

[0060] Step 109: Modify the subspace P for each category rc The initial values ​​for the feature subspace are used to automatically learn the standard subspace set P′ for all categories using the objective function:

[0061]

[0062] in, Let D(x) be the set of standard subspaces for each category. i,j ,P′ i P' represents the subspace P' of the i-th class in the support set of the task, from the j-th support sample of the i-th class. i The projection distance, D(x) i,j ,P′ l P' represents the subspace P' from the j-th support sample of the i-th class to the l-th class in the support set of the task. l The projection distance;

[0063] Step 110: Construct a standard subspace classifier using the standard subspace of each category as a metric to classify the samples in the query set of the task. For category c, it is represented as:

[0064] d c ′(q i )=||(IM c ′)(z que,i -μ C )||2

[0065]

[0066] Among them, M′ c =P′ c P′ c T , P′ c The standard subspace representing category c, d c ′(q i ) represents the sample q in the query set of the task. i To the standard subspace P′ of category c c The projected distance, p′(y=c|q i ) represents the sample q in the query set of the task. i The probability of belonging to category c is compared with the query set image q. i The probability value belonging to each category is used to classify the category corresponding to the highest probability.

[0067] Preferably, the loss function includes:

[0068] L = L cls +ωL inter +γL intra

[0069]

[0070]

[0071]

[0072] Where L represents the loss function, ω and γ represent the weight coefficients; N represents the total number of classes supporting the set of samples in a training task, M represents the number of query samples in each class in the training task, and p c,q P represents the predicted probability that the query sample q belongs to its true class c. i and P j Let Z represent the standard subspace of any two categories, M represent the number of query samples in each category, and Z represent the standard subspace of any two categories. i Let y represent the feature embedding of the i-th query sample. i Indicate the true category of the i-th query sample; for I [x] The value of I is determined when the expression x is true. [x] The value is 1 if the condition is met and 0 otherwise. cosine(a,b) represents the cosine similarity between vectors a and b.

[0073] Preferably, the validation task for the few-shot remote sensing scene classification model based on subspace networks, and the saving of the best-performing model, includes:

[0074] A task-based approach is used to train a few-shot remote sensing scene classification model based on subspace networks. During training, a certain number of iterations are set, and each iteration selects a training task for forward propagation. The loss is calculated, and the Adam algorithm is used to continuously optimize and update the model parameters. After a certain number of training iterations, a model validation is performed. A certain number of validation tasks are selected, and the current model is used to classify and predict the query set for each task. The average classification accuracy of all validation tasks is calculated as the result of this model validation. Based on the validation results, the best model up to the current training time is saved.

[0075] Preferably, the step of testing the best-performing model through test tasks includes: randomly selecting 600 test tasks, using the best-performing model obtained from training to perform classification prediction on the query set images in each test task, and calculating the average classification accuracy of these 600 test tasks as the result of model testing.

[0076] The present invention has at least the following beneficial effects

[0077] This invention employs a scene semantic guidance module to enhance regional features in remote sensing scene images that are highly correlated with category semantics; it introduces query set image information to construct a standard subspace classifier, improving the accuracy and reliability of the category metric benchmark, and the constructed standard subspace classifier is less affected by complex background information; at the same time, the loss function designed in this network can constrain the model to correctly understand category semantics, alleviate the classification confusion caused by large intra-class differences and inter-class similarities in remote sensing images, and improve the classification effect of remote sensing scene images. Attached Figure Description

[0078] Figure 1 This is an overall flowchart of the present invention;

[0079] Figure 2 This is a network framework diagram of the present invention;

[0080] Figure 3 This is a schematic diagram of the scene semantic guidance module of the present invention. Detailed Implementation

[0081] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0082] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0083] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0084] Please see Figure 1 This invention provides a few-shot remote sensing scene classification method based on subspace networks, comprising:

[0085] S1: Divide the remote sensing scene image dataset into training set, validation set and test set according to the category of remote sensing scene images in the dataset;

[0086] S2: Construct training tasks, validation tasks, and test tasks based on the predefined training set, validation set, and test set;

[0087] S3: Establish a few-shot remote sensing scene classification model based on subspace networks, and set the loss function according to the constructed training task;

[0088] S4: Based on the set loss function, use the task-based meta-learning training strategy to perform meta-training on the few-shot remote sensing scene classification model based on subspace network. During training, perform meta-validation on the few-shot remote sensing scene classification model based on subspace network through validation tasks, and save the model with the best performance.

[0089] S5: After the training and validation process is completed, perform meta-testing on the best-performing model through a test task.

[0090] Furthermore, the division of the remote sensing scene image dataset into training, validation, and test sets includes:

[0091] Set the total categories as C total The remote sensing scene image dataset D is divided into training set D according to the category of remote sensing scene images. train Validation set D cal and test set D test Training set D train Validation set D val and test set D test The categories of remote sensing scene images are C train C val and C test ,in, And C train ∪C val ∪C test =C total In this embodiment, the categories do not overlap.

[0092] Furthermore, the construction of the training task, validation task, and testing task includes:

[0093] In training set D train Validation set D val and test set D testThe training, validation, and testing tasks are constructed in the same way. Taking the construction of an N-way K-shot M-query training task as an example, the construction of the training task includes: starting from the category set C of the training set... train N categories are randomly selected, and K+M remote sensing scene images are randomly selected from each category. The K remote sensing scene images from each category form the support set, and the remaining M remote sensing scene images form the query set. It is guaranteed that no duplicate images appear in the support set and query set for each task. Similarly, verification and test tasks are constructed. In summary, each task consists of a support set and a query set. Support set: S = {(x 1,1 ,y 1,1 ),(x 1,2 ,y 1,2 ),…,(x N,K ,y N,K )},x i,j Let y represent the j-th image from category i. i,j ∈{0,1,2,…,N-1} represents the image x i,j True category labels, query set: Q = {q1, q2, ..., q n×M}, q l This represents the l-th image in the query set. For the training, validation, and testing tasks, the class labels of the images in the support set are known, while the class labels of the images in the query set are unknown and need to be predicted.

[0094] like Figure 2 As shown, the few-shot remote sensing scene classification model based on subspace networks further includes: an image feature extraction module, a scene semantic guidance module, and a classifier module;

[0095] The image feature extraction module uses ResNet12 as its network structure to extract features from the input remote sensing scene image to obtain feature map F. m The input is the image x after resizing the original image to 128×128. i The output is a C×H×W feature map F. m (C, H, and W represent the number of channels, height, and width of the feature map, respectively), which can be expressed as:

[0096]

[0097] In the formula, These are the parameters for a standard image feature extraction module;

[0098] The scene semantic guidance module includes: upper and lower branches and a feature aggregation module. The upper branch is used to aggregate features based on feature map F. m Obtain the scene embedding u′; the lower branch is used to represent the feature map F.m Aligning with the manifold shared with the scene embedding yields a new feature map F. m The feature aggregation module is used to embed the scene into u′ and the feature map F. m The semantic relation graph r is obtained by calculating spatial correlation; based on the semantic relation graph r, the feature graph F is... m The feature map is then weighted and flattened to obtain the feature embedding z. In this embodiment, the scene semantic guidance module highlights the features of regions in the image that are highly correlated with the semantics of the category.

[0099] The classifier module is used to construct a standard subspace classifier based on the feature embeddings of samples in the query set and support set for each task, and to classify the samples in the query set using the standard subspace classifier.

[0100] like Figure 3 As shown, further, in the upper branch of the scene semantic guidance module, the feature map F is first processed. m A global average pooling operation is performed to obtain a global feature aggregation representation u. Then, a learnable 1×1 convolutional layer is used to convolve the global feature aggregation representation u to obtain the scene embedding u′ of the remote sensing scene image. The scene embedding u′ can be expressed by the formula:

[0101] u′=σ α (GAP(F m ))

[0102] Where GAP represents global average pooling, σ α (·) represents a convolutional layer, and α is the corresponding parameter;

[0103] In the lower branch of the scene semantic guidance module, the feature map F m The feature map F is obtained by performing a convolution operation. m The convolutional module used consists of 1×1 convolutional layers, batch normalization layers, and ReLU activation function layers, with feature map F. m ′ is represented as:

[0104] F m ′=σ β (F m )

[0105] Where, σ β (·) denotes the convolution module of the lower branch, and β is the corresponding parameter;

[0106] In the feature aggregation module of the scene semantic guidance module, the scene embedding u′ and feature map F of the remote sensing scene image are first used as the basis for the feature aggregation module. mThe correlation between the features at each spatial location i and the scene semantic embedding u′ is calculated using a pointwise inner product method, and then normalization is performed to obtain the scene semantic relationship graph r. The correlation value at location i in the scene semantic relationship graph r is r. i Calculated using the following formula:

[0107]

[0108] Among them, F m ′ i Representing feature map F m The vector corresponding to position i in the spatial domain, H×W represents the feature map F. m The total number of spatial locations;

[0109] Using semantic relation graph r to analyze the original feature graph F m After performing a weighted update and flattening it into a feature vector, the feature embedding z of the remote sensing scene image is obtained, as follows:

[0110] z = Flatten(rF) m )

[0111] Here, Flatten represents the flattening operation. In this embodiment, the original feature map F is processed by the semantic relation graph r. m A weighted update is performed, and the data is flattened into feature vectors to obtain the feature embedding of the remote sensing scene image. For samples of category c in the support set of the task as input, a support embedding is generated. For the samples in the task query set, generate query embedding features z = z que,i ;

[0112] Furthermore, the construction of the standard subspace classifier includes:

[0113] Step 101: Calculate the average feature embedding of the support set samples corresponding to each class in each task as the initial prototype for that class. For class c, its initial prototype μ c Represented as:

[0114]

[0115] Where K represents the number of samples supporting class c in the support set. The feature embedding of the i-th sample of class c in the support set of the task is represented.

[0116] Step 102: Subtract the feature embeddings of the support set samples for each category in the task from the initial prototype of that category to obtain the initial feature space of that category. For category c, its initial feature space is represented as:

[0117]

[0118] in, Represents the initial feature space of category c;

[0119] Step 103: Perform singular value decomposition on the initial feature space of each category in the task to obtain the left singular matrix. For category c, this operation is specifically represented as follows:

[0120]

[0121] Among them, the left singular matrix B c =[a1,a2,a3,a4,…,a n ] is an orthogonal matrix; for the left singular matrix B c The initial feature subspace of this category is obtained by truncation. For category c, its initial feature subspace is represented as:

[0122] P c =[a1,a2,a3,a4]

[0123] Among them, P c Let c represent the initial feature subspace of category c;

[0124] Step 104: Calculate the projected distance between the feature embeddings of samples in the query set of the task and the initial feature subspace of each class, which is expressed as:

[0125] d c (q i )=||(IM c (z) que,i -μ c )||2,q i ∈que

[0126] Where, d c (q i ) represents the sample q in the query set of the task. i The initial feature subspace P of category c c The projected distance, where que represents the query set of the task. ||·||2 represents the L2 norm, z que,i The sample q in the query set represents the task. i Feature embedding;

[0127] Step 105: Based on the projected distance of the feature embeddings of samples in the task's query set into the initial feature subspace of each category, calculate the probability that the samples in the task's query set belong to each category, which is expressed as:

[0128]

[0129] Where c∈{1,2,…,N}, N represents the total number of categories in the support set of the task, and mc (q i ) represents the sample q in the query set of the task. i The probability of belonging to category c;

[0130] Step 106: Based on the probability that samples in the query set belong to each category, fine-tune the initial prototypes for each category to obtain fine-tuned prototypes for each category. For category c, its fine-tuned prototype is represented as:

[0131]

[0132] Where, μ′ c This represents the fine-tuned prototype of category c;

[0133] Step 107: Subtract the feature embeddings of the support set samples for each class in the task from the fine-tuned prototype of that class to obtain the corrected feature space for that class. For class c, its corrected feature space is represented as:

[0134]

[0135] in, Represents the modified feature space of category c;

[0136] Step 108: Perform singular value decomposition on the modified feature space of each category in the task to obtain the left singular matrix. For category c, this operation is specifically represented as follows:

[0137]

[0138] The left singular matrix B c ′=[a1′,a2′,a3′,a4′,…,a n '] is an orthogonal matrix. For the left singular matrix B... c If we truncate the subspace to obtain the modified feature space of category c, then the modified feature space of category c is represented as follows:

[0139] P rc =[a1′,a2′,a3′,a4′]

[0140] Among them, P rc Represents the modified feature subspace of category c;

[0141] Step 109: Modify the subspace P for each category rc The initial values ​​for the feature subspace are used to automatically learn the standard subspace set P′ for all categories using the objective function:

[0142]

[0143] in, Let D(x) be the set of standard subspaces for each category.i,j ,P i ′) represents the subspace P of the i-th class and the j-th support sample in the support set of the task to the i-th class. i The projected distance of ′, D(x) i,j ,P l ′) represents the subspace P of the i-th class and j-th support sample in the support set of the task to the l-th class. l The projected distance of ′;

[0144] Step 110: Construct a standard subspace classifier using the standard subspace of each category as a metric to classify the samples in the query set of the task. For category c, it is represented as:

[0145] d c ′(q i )=||(IM′ c (z) que,i -μ c )||2

[0146]

[0147] Among them, M′ c =P′ c P′ c T , P′ c The standard subspace representing category c, d c ′(q i ) represents the sample q in the query set of the task. i To the standard subspace P′ of category c c The projected distance, p′(y=c|q i ) represents the sample q in the query set of the task. i The probability of belonging to category c is compared with the query set image q. i The probability values ​​belonging to each category are classified into the category corresponding to the highest probability. In this embodiment, the processing of training tasks, validation tasks, and testing tasks is as described in steps 101-110 above, and the same principle is used to process training tasks, validation tasks, and testing tasks.

[0148] Preferably, the loss function includes:

[0149] L = L cls +ωL inter +γL intra

[0150]

[0151]

[0152]

[0153] Where L represents the loss function, ω and γ represent the weight coefficients; N represents the total number of classes supporting the set of samples in a training task, M represents the number of query samples in each class in the training task, and p c,q P represents the predicted probability that the query sample q belongs to its true class c. i and P j Let Z represent the standard subspace of any two categories, M represent the number of query samples in each category, and Z represent the standard subspace of any two categories. i Let y represent the feature embedding of the i-th query sample. i Indicate the true category of the i-th query sample; for I [x] The value of I is determined when the expression x is true. [x] The value is 1 if the condition is met and 0 otherwise. cosine(a,b) represents the cosine similarity between vectors a and b.

[0154] Preferably, the validation task for the few-shot remote sensing scene classification model based on subspace networks, and the saving of the best-performing model, includes:

[0155] A task-based approach is used to train a few-shot remote sensing scene classification model based on subspace networks. During training, a certain number of iterations are set, and each iteration selects a training task for forward propagation. The loss is calculated, and the Adam algorithm is used to continuously optimize and update the model parameters. After a certain number of training iterations, a model validation is performed. A certain number of validation tasks are selected, and the current model is used to classify and predict the query set for each task. The average classification accuracy of all validation tasks is calculated as the result of this model validation. Based on the validation results, the best model up to the current training time is saved.

[0156] Preferably, the step of testing the best-performing model through test tasks includes: randomly selecting 600 test tasks, using the best-performing model obtained from training to perform classification prediction on the query set images in each test task, and calculating the average classification accuracy of these 600 test tasks as the result of model testing.

[0157] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0158] In summary, this invention employs a scene semantic guidance module to enhance regional features in remote sensing scene images that are highly correlated with category semantics; it introduces query set image information to construct a standard subspace classifier, improving the accuracy and reliability of the category metric benchmark, and the constructed standard subspace classifier is less affected by complex background information; at the same time, the loss function designed in this network can constrain the model to correctly understand category semantics, alleviate the classification confusion caused by large intra-class differences and inter-class similarities in remote sensing images, and improve the classification effect of remote sensing scene images.

[0159] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A few-shot remote sensing scene classification method based on subspace networks, characterized in that, include: S1: Divide the remote sensing scene image dataset into training set, validation set and test set according to the category of remote sensing scene images in the dataset; S2: Construct training tasks, validation tasks, and test tasks based on the predefined training set, validation set, and test set; S3: Establish a few-shot remote sensing scene classification model based on subspace networks, and set the loss function according to the constructed training task; The few-shot remote sensing scene classification model based on subspace networks includes: an image feature extraction module, a scene semantic guidance module, and a classifier module; The image feature extraction module uses ResNet12 as its network structure to extract feature maps from the input remote sensing scene images. ; The scene semantic guidance module includes: two branches (upper and lower) and a feature aggregation module. The upper branch is used to aggregate features based on feature maps. Get Scene Embedding The lower branch is used to represent the feature map. Aligning to a manifold shared with the scene embedding yields a new feature map. The feature aggregation module is used to embed the scene. and feature map Semantic relation graph is obtained by spatial correlation calculation. According to the semantic relationship graph For feature maps Weighting is performed, and then the resulting feature map is flattened to obtain the feature embedding. ; The classifier module is used to construct a standard subspace classifier based on the feature embeddings of samples in the query set and support set for each task, and to classify the samples in the query set using the standard subspace classifier. The loss function includes: in, This indicates the loss function that is set. and The weighting coefficients are represented by these coefficients. This represents the total number of classes that support the set of samples in a training task. This indicates the number of query samples in each category for the training task. Indicates query sample Its true category The predicted probability, and The standard subspace representing any two categories, Indicates the first Feature embedding of each query sample Indicates the first The true category of each query sample; for The value is obtained when the expression... When true, The value is 1 if it is not 0 otherwise. Representing vectors and Cosine similarity; S4: Based on the set loss function, use the task-based meta-learning training strategy to perform meta-training on the few-shot remote sensing scene classification model based on subspace network. During training, perform meta-validation on the few-shot remote sensing scene classification model based on subspace network through validation tasks, and save the model with the best performance. S5: After the training and validation process is completed, perform meta-testing on the best-performing model through a test task.

2. The few-shot remote sensing scene classification method based on subspace networks according to claim 1, characterized in that, The process of dividing the remote sensing scene image dataset into training, validation, and test sets includes: Set the total categories as Remote sensing scene image dataset The training set is divided according to the category of remote sensing scene images. Validation set and test set training set Validation set and test set The categories of remote sensing scene images are respectively , and ,in, , , and .

3. The few-shot remote sensing scene classification method based on subspace networks according to claim 2, characterized in that, The construction of training, validation, and testing tasks includes: In the training set Validation set and test set The training, validation, and testing tasks are constructed in the same way. The training task is constructed by: extracting data from the class set of the training set... N categories are randomly selected, and K+M remote sensing scene images are randomly selected from each category. The K remote sensing scene images from each category are used as the support set, and the remaining M remote sensing scene images are used as the query set. Similarly, the verification task and the test task are constructed.

4. The few-shot remote sensing scene classification method based on subspace networks according to claim 3, characterized in that, In the upper branch of the scene semantic guidance module, the feature map is first processed. Perform global average pooling to obtain the global feature aggregation representation. , and then utilize the learnable 1 1. Convolutional layer for global feature aggregation representation Perform convolution operations to obtain scene embeddings from remote sensing scene images. Scene embedding Expressed as a formula: in, Indicates global average pooling. Indicates a convolutional layer. For the corresponding parameters; In the lower branch of the scene semantic guidance module, the feature map is paired. Convolution operation is performed to obtain feature maps The convolutional modules used consist of 1 It consists of a convolutional layer, a batch normalization layer, and a ReLU activation function layer, and the feature map... Represented as: in, This represents the convolutional module of the lower branch. For the corresponding parameters; In the feature aggregation module of the scene semantic guidance module, the scene embedding of the remote sensing scene image is first used as the basis for the feature aggregation module. and feature map Calculate each spatial location using the point-by-point inner product method. Corresponding feature and scene semantic embedding The correlation is then normalized to obtain the scene semantic relationship graph. Scene semantic relationship graph Upper position Correlation values ​​at the location Calculated using the following formula: in, Representation of feature map Upper spatial position The corresponding vector at that location, Representation of feature map The total number of spatial locations; Using semantic relationship graphs For the original feature map Perform a weighted update and flatten it into feature vectors to obtain the feature embedding of the remote sensing scene image. , is represented as: in, This indicates a flattening operation.

5. The few-shot remote sensing scene classification method based on subspace networks according to claim 4, characterized in that, The construction of the standard subspace classifier includes: Step 101: Calculate the average feature embedding of the support set samples corresponding to each category in each task as the initial prototype for that category. Its initial prototype Represented as: in, Indicates support for centralized categories The number of samples, Indicates the support set category for the task The Feature embedding of each sample; Step 102: Subtract the feature embeddings of the support set samples for each category in the task from the initial prototype of that category to obtain the initial feature space for that category. Its initial feature space is represented as: in, Indicate category The initial feature space; Step 103: Perform singular value decomposition on the initial feature space of each category in the task to obtain the left singular matrix. For category c, this operation is specifically represented as follows: Among them, the left singular matrix ] is an orthogonal matrix; for left singular matrices The initial feature subspace of this category is obtained by truncation. For category c, its initial feature subspace is represented as: ] in, Let c represent the initial feature subspace of category c; Step 104: Calculate the projected distance between the feature embeddings of samples in the query set of the task and the initial feature subspace of each class, which is expressed as: in, Samples in the query set representing the task To Category initial feature subspace The projection distance, The query set representing the task. , Describing the L2 norm, Samples in the query set representing the task Feature embedding; Step 105: Based on the projected distance of the feature embeddings of samples in the task's query set into the initial feature subspace of each category, calculate the probability that the samples in the task's query set belong to each category, which is expressed as: in, N represents the total number of categories in the support set of the task. Samples in the query set representing the task Category The probability of; Step 106: Based on the probability that samples in the query set belong to each category, fine-tune the initial prototypes for each category to obtain fine-tuned prototypes for each category. For category c, its fine-tuned prototype is represented as: in, This represents the fine-tuned prototype of category c; Step 107: Subtract the feature embeddings of the support set samples for each class in the task from the fine-tuned prototype of that class to obtain the corrected feature space for that class. Its modified feature space is represented as: in, Indicate category The modified feature space; Step 108: Perform singular value decomposition on the modified feature space of each category in the task to obtain the left singular matrix. For category c, this operation is specifically represented as follows: Among them, the left singular matrix ] is an orthogonal matrix; for left singular matrices After truncating the subspace, the modified feature space of the category is obtained, and then the category... The modified feature subspace is represented as: in, Represents the modified feature subspace of category c; Step 109: Correct subspaces for each category The initial values ​​for the feature subspace are used to automatically learn the standard subspace set for all categories using the objective function. : in, For the set of standard subspaces of each category, The first in the support set for the task The first category The number of supporting samples to the first Subspaces of each category The projection distance, The first in the support set for the task The first category The number of supporting samples to the first Subspaces of each category The projection distance; Step 110: Construct a standard subspace classifier using the standard subspace of each category as a metric to classify the samples in the query set of the task. For category c, it is represented as: in, , Indicate category standard subspace Samples in the query set representing the task To Category standard subspace The projection distance, Samples in the query set representing the task Category The probability of comparing the query set images The probability value belonging to each category is used to classify it into the category corresponding to the highest probability.

6. The few-shot remote sensing scene classification method based on subspace networks according to claim 5, characterized in that, The validation task verifies the few-shot remote sensing scene classification model based on subspace networks, and the model with the best performance is saved as follows: A task-based approach is used to train a few-shot remote sensing scene classification model based on subspace networks. During training, the number of iterations is set, and each iteration selects a training task for forward propagation. The loss is calculated, and the Adam algorithm is used to continuously optimize and update the model parameters. After a preset number of training iterations, a model validation is performed. A preset number of validation tasks are selected, and the current model is used to classify and predict the query set for each task. The average classification accuracy of all validation tasks is calculated as the result of this model validation. Based on the validation results, the best model up to the current training time is saved.

7. The few-shot remote sensing scene classification method based on subspace networks according to claim 6, characterized in that, The process of testing the best-performing model through test tasks includes: randomly selecting 600 test tasks, using the best-performing model trained to classify and predict the query set images in each test task, and calculating the average classification accuracy of these 600 test tasks as the result of the model test.

Images