A small sample classification method based on collaborative metrics

Abstract

The present invention discloses a small-sample classification method based on collaborative metrics, comprising the following steps: obtaining global spatial semantic relationships and local spatial semantic relationships of different granularities based on support samples and query samples; fusing the global spatial semantic relationships and local spatial semantic relationships of each granularity to obtain a global-local fused spatial semantic relationship of each granularity; performing collaborative metrics decisions on the multi-granularity spatial semantic relationship set using a minimum sum feature set distance metric to obtain a predicted probability distribution; optimizing a multi-granularity collaborative metrics model using labels to obtain a multi-granularity collaborative metrics model for classification; and inputting an image to be classified into the multi-granularity collaborative metrics model for classification to obtain a classification result for the image to be classified. The present invention can effectively capture potential spatial discriminative semantic information, improve image classification performance under small-sample conditions, and address the technical issues of model collapse and overfitting under limited sample conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning classification, and in particular to a small sample classification method using collaborative metrics. Background Art

[0002] The rise of large deep learning models has attracted widespread attention and research. These models, with their powerful representation and learning capabilities, have achieved remarkable results in various fields, including image recognition, natural language processing, and speech recognition. However, their remarkable success is compounded by the enormous demand for labeled data that these large models require. To achieve good performance, these models typically require massive amounts of labeled data for training to capture the complex patterns and regularities within the data. Obtaining large amounts of labeled data is a time-consuming, costly, and difficult task, particularly in fields requiring high expertise or manual intervention, such as medicine and remote sensing. Furthermore, manual labeling is subject to empirical errors, resulting in noisy and inconsistent datasets. These issues further limit the application and performance of large models. To address these issues and promote the development of next-generation artificial intelligence, a growing number of researchers are turning their attention to the problem of using small-sample, lightweight models to handle complex tasks. Specifically, they aim to enable models to perform well in real-world processing scenarios even when data is scarce. Small-sample algorithms have become a core research direction in this emerging field.

[0003] The basic idea of ​​few-shot learning is to transfer knowledge or experience from seen tasks (e.g., tasks sampled from base class data) to unseen tasks (e.g., tasks sampled from novel class data). Following this fundamental theory, transfer learning methods were first used to address the few-shot classification problem. Specifically, an initial model for image classification is trained using a cross-entropy loss on all samples from the base class. The model is then fine-tuned using a small number of samples from the novel class. However, since the novel class only has a few samples, the model often struggles to learn discriminative feature representations for the novel class, resulting in poor classification performance. To overcome these shortcomings, a meta-learning few-shot classification framework has been proposed. This framework uses data samples from the base class during training to mimic the few-shot classification task used during testing, completing the same task and facilitating the transfer of knowledge from the base class to the novel class. Although various meta-learning-based methods employ complex algorithms and network architectures, they still struggle to learn robust features using only a small number of samples. This results in poor performance in real-world applications and a failure to meet user needs. Therefore, it is urgent to explore the learning mechanism of classification algorithms under limited sample annotation conditions and to construct robust feature representation methods to further improve the data classification performance under small sample conditions. Summary of the Invention

[0004] In order to solve the above problems, the present invention provides a small sample classification method with collaborative measurement, which can complete the classification task under the conditions of limited labeled samples, and can realize efficient and high-precision small sample image classification, greatly improving the classification accuracy and effect of existing small sample image classification.

[0005] In order to achieve the purpose of the present invention, the following technical solutions are adopted:

[0006] The present invention provides a small sample classification method based on collaborative metrics, comprising the following steps: 1. A small sample classification method based on collaborative metrics, characterized in that it comprises the following steps: S101: constructing a first number of tasks on a labeled optical remote sensing image, each of the tasks comprising a support sample and a query sample; wherein the number of support samples does not exceed 10; S102: inputting the support samples and the query samples into a multi-granularity collaborative metrics model to obtain global spatial semantic relationships of different granularities and local spatial semantic relationships of different granularities; S103: fusing the global spatial semantic relationship and the local spatial semantic relationship at each granularity to obtain a global spatial semantic relationship at each granularity; S104: fusing the global spatial semantic relationship and the local spatial semantic relationship at each granularity to obtain a global spatial semantic relationship at each granularity; S105: fusing the global spatial semantic relationship and the local spatial semantic relationship at each granularity to obtain a global spatial semantic relationship at each granularity; S106: fusing the global spatial semantic relationship and the local spatial semantic relationship at each granularity to obtain a global spatial semantic relationship at each granularity; S107: fusing the global spatial semantic relationship and the local spatial semantic relationship at each granularity to obtain a global spatial semantic relationship at each granularity; S108: fusing the global spatial semantic relationship and the local spatial semantic relationship at each granularity to obtain a global spatial semantic relationship at each granularity; S109: fusing the global spatial semantic relationship and the local spatial semantic relationship at each granularity to obtain a global spatial semantic relationship at each granularity; S110: fusing the global spatial semantic relationship and the local spatial semantic relationship at each granularity to obtain a global spatial semantic relationship at each granularity; S111: fusing the global spatial semantic relationship and the local spatial semantic relationship at each granularity to obtain a global spatial semantic relationship at each granularity; S112: fusing the global spatial semantic relationship and the local spatial semantic relationship at each granularity to obtain a global spatial semantic relationship at each The method comprises the following steps: step S104: combining the global-local fusion spatial semantic relations of different granularities to obtain a multi-granularity spatial semantic relationship set; step S105: performing collaborative measurement decision on the multi-granularity spatial semantic relationship set using the minimum sum feature set distance measurement method to obtain a predicted probability distribution; step S106: optimizing the multi-granularity collaborative measurement model using labels to obtain a multi-granularity collaborative measurement model for classification; step S107: inputting the image to be classified into the multi-granularity collaborative measurement model for classification to obtain a classification result of the image to be classified, wherein the classification result indicates the category information of the image to be classified.

[0007] Furthermore, the method also includes: the global spatial semantic relationships of different granularities indicate the overall summary information of the labeled optical remote sensing image, including the global spatial semantic relationships of the supporting samples and the global spatial semantic relationships of the query samples; the local spatial semantic relationships of different granularities indicate the local detail information of the labeled optical remote sensing image, including the local spatial semantic relationships of the supporting samples and the local spatial semantic relationships of the query samples; the global-local fusion spatial semantic relationships include the global-local fusion spatial semantic relationships of the supporting samples and the global-local fusion spatial semantic relationships of the query samples; the multi-granularity spatial semantic relationship set includes the multi-granularity spatial semantic relationship set of the supporting samples and the multi-granularity spatial semantic relationship set of the query samples.

[0008] Furthermore, the collaborative measurement decision-making on the multi-granularity spatial semantic relationship set using the minimum sum feature set distance measurement method includes: S105.1: using the multi-granularity spatial semantic relationship set of the support sample to take an average to obtain the prototype spatial semantic relationship set; S105.2: using the minimum sum feature set distance measurement method to perform collaborative measurement decision-making between multi-granularity spatial semantic relationship sets, and calculating the distance between the multi-granularity spatial semantic relationship set of the query sample and the prototype spatial semantic relationship set.

[0009] Furthermore, fusing the global spatial semantic relationship and the local spatial semantic relationship at each granularity to obtain a global-local fused spatial semantic relationship at each granularity includes: S103.1: aggregating the local spatial semantic relationships through pooling to obtain a pooled local spatial semantic relationship;

[0010] S103.2: The global spatial semantic relationship at each granularity and the local spatial semantic relationship after pooling Add to obtain global-local fusion spatial semantic relationship The expression is as follows:

[0011]

[0012] Among them, i is a positive integer, representing the i-th granularity, N i represents the number of converter layers corresponding to the i-th granularity, and M represents the number of granularities.

[0013] Furthermore, the global-local fusion spatial semantic relations of different granularities are combined to obtain a multi-granularity spatial semantic relationship set including:

[0014] The global-local fusion spatial semantic relations of different granularities are all collected to obtain a multi-granularity spatial semantic relationship set Set, which is expressed as follows:

[0015]

[0016] Represents the Nth M Global-local fusion spatial semantic relations of layers.

[0017] Furthermore, averaging the multi-granularity spatial semantic relationship set of the support samples to obtain the prototype spatial semantic relationship set includes:

[0018] Based on the multi-granularity spatial semantic relationship set of the supporting samples, the prototype spatial semantic relationship set corresponding to each category is calculated respectively. For the prototype spatial relationship set C of category k k The expression is as follows:

[0019]

[0020] Among them, S k Represents the number of support samples for category k, Set k Represents the set of multi-granularity prototype spatial relations of category k, represents the Nth M Global-local fusion spatial semantic relationship of layers; represents the Nth M The mean of global-local fusion spatial semantic relations at the layer level.

[0021] Furthermore, the minimum sum feature set distance measurement method is used to make collaborative measurement decisions between multi-granularity spatial semantic relationship sets. The distance between the multi-granularity spatial semantic relationship set of the query sample and the prototype spatial semantic relationship set is calculated, including:

[0022] The query sample x q Multi-granularity spatial semantic relationship set Expressed as:

[0023]

[0024] The distance measurement between sets adopts the minimum sum feature set distance measurement method, which is expressed as:

[0025]

[0026] in, Represents query sample x q Nth M The global-local fusion spatial semantic relationship of the layer, D() represents the set distance metric function, summin() represents the minimum sum feature set distance metric function, and the minimum sum feature set distance metric function is expressed as:

[0027]

[0028] represents the Nth M The mean of the global-local fusion spatial semantic relationship of the layer, Represents query sample x q Nth M The global-local fusion spatial semantic relationship of the layer, min() represents the minimum function, d() represents the distance measurement function; i, j are positive integers, and M represents the total number of different granularities.

[0029] Furthermore, optimizing the multi-granularity collaborative measurement model based on the label to obtain the multi-granularity collaborative measurement model for classification includes:

[0030] S106.1: Obtain the predicted probability distribution based on the distance between the multi-granularity spatial semantic relationship set of the query sample and the prototype spatial semantic relationship set

[0031]

[0032] in, Represents the predicted category, x q Represents the query sample, p represents the predicted probability, D() represents the set distance metric function, Represents query sample x q The multi-granularity spatial semantic relationship set obtained by the multi-granularity collaborative measurement model, Set k Representing a set of semantic relationships in the prototype space of category k; the predicted probability distribution indicates the likelihood of the output prediction result;

[0033] S106.2: Construct a loss function L using the predicted probability distribution:

[0034]

[0035]

[0036] N represents the number of classification categories for each task, Q represents the number of query samples in each task, i and n are both natural numbers not equal to 0, I() represents the conditional function, and y represents the true category;

[0037] S106.3: Supervise the multi-granularity collaborative measurement model by optimizing the loss function L to obtain the multi-granularity collaborative measurement model for classification.

[0038] Furthermore, the image to be classified is input into the multi-granularity collaborative measurement model for classification to obtain the classification result of the image to be classified, including: S107.1: inputting the image to be classified into the multi-granularity collaborative measurement model for classification to obtain global spatial semantic relationships and local spatial semantic relationships, which are fused to form a multi-granularity global-local spatial semantic relationship set; S107.2: obtaining the prototype spatial semantic relationship set based on the multi-granularity global-local spatial semantic relationship set; S107.3: performing minimum sum feature set distance measurement based on the prototype spatial semantic relationship set to obtain category probability, and the category with the largest probability is the classification result, and the classification result indicates the category of the image to be classified.

[0039] It should be understood that the contents described in the summary of the present invention are not intended to limit the key features or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The above and other features, advantages and aspects of the embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0041] Figure 1 Schematic diagram of the method flow of the present invention;

[0042] Figure 2 Schematic diagram of the multi-granularity collaborative measurement model of the present invention;

[0043] Figure 3 Schematic diagram of the minimum sum feature set distance measurement method of the present invention. DETAILED DESCRIPTION

[0044] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0045] In the description of the embodiments of the present invention, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to." The term "based on" should be understood as "based, at least in part, on." The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment." The term "some embodiments" should be understood as "at least some embodiments." Other explicit and implicit definitions may be included below.

[0046] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and description are to be regarded as illustrative in nature and not restrictive. There is no requirement for the order in which the method steps are described; any order that can be implemented is within the scope of the present invention.

[0047] Few-shot learning is to give a set of data D, which contains a test set D with a small number of labels. novel , and a training set D with all labels base , and satisfies D base With D novel The label space categories of are disjoint, that is Where c(·) is the set of dataset categories. The goal of small sample learning is to construct a mapping function f that uses D baseThe learned prior knowledge can be used to transform D novel The samples in the test set are mapped to the correct categories, thereby achieving classification of the test set samples.

[0048] In the meta-learning paradigm, a task T is constructed on data D. Task T includes at least two subsets: a support set S and a query set Q. Both subsets are sampled from data D and have the same label space. The support set S consists of samples with labeled information, and the query set Q consists of samples whose labels need to be predicted. For example, if there are N categories in the support set S, each category includes K different labeled support samples. The value of K is small, typically less than 10, such as 1 or 5. This is merely an example and should not be construed as a limitation of the present invention. The query set includes the same N categories as the support set, and each category includes X different query samples. This is called an N-WayK-ShotXquery task.

[0049] The present invention uses the ViT (vision transformer) structure as a multi-granularity collaborative measurement model to obtain global and local spatial semantic relationships at different granularities, and then fuses the global and local spatial semantic relationships at each granularity to obtain a global-local fused spatial semantic relationship. The global-local fused spatial semantic relationship has excellent representation capabilities and is more separable. Next, the global-local fused spatial semantic relationships at different granularities are combined to obtain a multi-granularity spatial semantic relationship set. Collaborative measurement decisions are made on the multi-granularity spatial semantic relationship set to form a more accurate category representation space when samples are limited. The minimum sum feature set distance measurement method is used to further improve the accuracy of small sample classification, and finally the category probability is used as the classification result.

[0050] Below is the attached figure Figure 1-Figure 3 The present invention is described in detail with reference to the embodiments.

[0051] like Figure 1 As shown, the present invention provides a small sample classification method based on collaborative metrics, comprising:

[0052] S101: Constructing a first number of tasks on labeled optical remote sensing images, each of the tasks including a support sample and a query sample.

[0053] The first number is an integer not less than 0, and the specific value can be flexibly selected according to actual conditions.

[0054] Both support samples and query samples are labeled optical remote sensing images, with the labels indicating the corresponding image category. The number of support samples should not exceed 10, and can be 1, 5, or 10, for example. Support samples are a small number of labeled samples (no more than 10) used in the simulated small sample size task. There is no specific requirement for the number of query samples.

[0055] S102: Input the support samples and query samples into a multi-granularity collaborative measurement model to obtain global spatial semantic relationships of different granularities and local spatial semantic relationships of different granularities.

[0056] Global spatial semantic relationships of different granularities indicate the overall summary information of labeled optical remote sensing images, including the global spatial semantic relationships of support samples and the global spatial semantic relationships of query samples.

[0057] Local spatial semantic relations of different granularities indicate local detail information of labeled optical remote sensing images, including local spatial semantic relations of support samples and local spatial semantic relations of query samples.

[0058] The following combination Figure 2 The multi-granularity collaborative measurement model is described in detail.

[0059] like Figure 2 As shown in the figure, the multi-granularity collaborative measurement model uses the ViT structure as the backbone network to capture spatial semantic relationships, and obtains global-local spatial semantic relationships of different granularities through the global-local spatial semantic relationship fusion module. The specific global-local spatial semantic relationship module structure is shown in Figure 2 As shown on the right, the global-local spatial semantic relationships of different granularities are fused to obtain a multi-granularity semantic relationship set, and finally a decision metric is performed on the multi-granularity semantic relationship set.

[0060] The multi-granularity collaborative measurement model adopts a ViT (vision transformer) structure. The ViT structure can adopt existing methods such as ViT-Base, ViT-Small, ViT-Tiny, etc., which are not limited here.

[0061] The ViT structure consists of multiple transformer layers, with the number of layers ranging from 6 to 20. Different transformer layers obtain spatial semantic relationships of different granularities. The output of each transformer layer consists of a category token and multiple general visual tokens. The category token contains the extracted global category information as the global spatial semantic relationship. Generally, visual tokens contain information of different local blocks extracted as local spatial semantic relations. In order to obtain global spatial semantic relationships and local spatial semantic relationships of different granularities, the outputs of M transformer layers with better feature generalization are taken, which can be specifically expressed as:

[0062]

[0063]

[0064]

[0065] N i ∈[1,…,L]; i=1,2,…,M

[0066] 2≤M≤L

[0067] represents the output of the transformer layer, represents a category token, Represents a general visual token, N i represents the specific number of transformer layers, i is an integer not equal to zero, representing the i-th granularity. M represents the total number of different granularities. In order to obtain the spatial semantic relationship of different granularities, M is greater than or equal to 2. L represents the number of transformer layers in the ViT structure, and hw represents the total number of general visual tokens.

[0068] Considering that hw is usually large and the computational overhead is too high, the local spatial semantic relationships are aggregated into a smaller number through pooling. The local spatial semantic relationships after pooling are expressed as:

[0069]

[0070] P is a hyperparameter, representing the number of horizontal / vertical local blocks after pooling, pool() represents the pooling function, and the total number of general visual tokens after pooling is p 2 In this embodiment, p is selected as 4. The local spatial semantic relationship supplements the detail information lost by the global spatial semantic relationship, and the global spatial semantic relationship provides the overall summary information that the local spatial semantic relationship lacks.

[0071] S103: Fusing the global spatial semantic relationship and the local spatial semantic relationship at each granularity to obtain a global-local fused spatial semantic relationship at each granularity;

[0072] The global-local fusion spatial semantic relationship has stronger representation capabilities, including the global-local fusion spatial semantic relationship of the support sample and the global-local fusion spatial semantic relationship of the query sample. S101.3 can be implemented in the following ways:

[0073] S103.1: Aggregate the local spatial semantic relations through pooling to obtain pooled local spatial semantic relations.

[0074] S103.2: The global spatial semantic relationship at each granularity and the local spatial semantic relationship after pooling Fusion / addition to obtain global-local fusion spatial semantic relationship The specific calculation method is as follows:

[0075]

[0076] Among them, i is a positive integer, representing the i-th granularity, N i represents the number of converter layers corresponding to the i-th granularity, and M represents the number of granularities.

[0077] Global-local fusion of spatial semantic relationships combines the advantages of global and local spatial semantic relationships, incorporating both global category information and discriminative details of local regions, resulting in stronger representational capabilities. In the case of small samples, global-local fusion of spatial semantic relationships forms a more accurate category representation.

[0078] S104: Combining the global-local fusion spatial semantic relations of different granularities to obtain a multi-granularity spatial semantic relationship set.

[0079] The multi-granularity spatial semantic relationship set includes the multi-granularity spatial semantic relationship set of the support sample and the multi-granularity spatial semantic relationship set of the query sample.

[0080] The multi-granularity spatial semantic relationship set corresponds one-to-one to the labeled optical remote sensing image. That is, each image obtains a multi-granularity spatial semantic relationship set Set after passing the multi-granularity collaborative measurement model. The multi-granularity spatial semantic relationship set Set is obtained by gathering all the global-local fusion spatial semantic relationships of different granularities. The expression is as follows:

[0081]

[0082] in, Represents the Nth M Global-local fusion spatial semantic relations of layers.

[0083] Different transformer layers capture spatial semantic relationships of different granularities. The shallower layers capture primary and simple spatial semantic relationships, while the deeper layers capture high-level spatial semantic relationships with strong generalization. The global-local spatial semantic relationships at different semantic levels complement each other. The fusion at the semantic level further enhances the transferability of spatial semantic relationships and performs well in small-sample classification tasks.

[0084] S105: Using a minimum sum feature set distance measurement method to perform collaborative measurement decision on the multi-granularity spatial semantic relationship set to obtain a predicted probability distribution.

[0085] This step can be implemented in the following ways:

[0086] S105.1: averaging the multi-granularity spatial semantic relationship sets of the support samples to obtain a prototype spatial semantic relationship set. The prototype spatial relationship set is used to indicate category information and calculate category prediction results.

[0087] Based on the multi-granularity spatial semantic relationship set of the supporting samples, the prototype spatial semantic relationship set corresponding to each category is calculated respectively. For the prototype spatial relationship set C of category k k The expression is as follows:

[0088]

[0089] Among them, S k Represents the number of support samples for category k, Set k Represents the set of multi-granularity prototype spatial relations of category k, represents the Nth M Global-local fusion spatial semantic relationship of layers; represents the Nth M The mean of global-local fusion spatial semantic relations at the layer level.

[0090] S105.2: Use the minimum sum feature set distance measurement method to perform collaborative measurement decision between multi-granularity spatial semantic relationship sets, and calculate the distance between the multi-granularity spatial semantic relationship set of the query sample and the prototype spatial semantic relationship set.

[0091] The query sample x q Multi-granularity spatial semantic relationship set Expressed as:

[0092]

[0093] When measuring the distance, since the prototype space semantic relationship set and the multi-granularity space semantic relationship set of the query sample are both sets, the distance measurement between the sets adopts the minimum sum feature set distance measurement method, which is expressed as:

[0094]

[0095] in, Represents query sample x q Nth MThe global-local fusion spatial semantic relationship of the layer, D() represents the set distance metric function, summin() represents the minimum sum feature set distance metric function, and the minimum sum feature set distance metric function is expressed as:

[0096]

[0097] represents the Nth M The mean of the global-local fusion spatial semantic relationship of the layer, Represents query sample x q Nth M The global-local fusion spatial semantic relationship of the layer, min() represents the minimum function, d() represents the distance metric function. In practice, the negative cosine similarity function is used. i and j are positive integers, and M represents the total number of different granularities. The minimum sum feature set distance metric function provides a new inter-set distance metric method for collaborative metric decision-making between sets, making it more suitable for small-sample classification tasks.

[0098] S106: Optimizing the multi-granularity collaborative measurement model using the labels to obtain a multi-granularity collaborative measurement model for classification.

[0099] This step can be implemented in the following ways:

[0100] S106.1: Obtain the predicted probability distribution based on the distance between the multi-granularity spatial semantic relationship set of the query sample and the prototype spatial semantic relationship set

[0101]

[0102] in, Represents the predicted category, x q Represents the query sample, p represents the predicted probability, D() represents the set distance metric function, Represents query sample x q The multi-granularity spatial semantic relationship set obtained by the multi-granularity collaborative measurement model, Set k Represents the set of semantic relations in the prototype space of category k.

[0103] The predicted probability distribution indicates the probability of the output prediction result, such as the probability of river is 0.8, the probability of street is 0.05, the probability of palace is 0.1, and the probability of farmland is 0.05. The overall probability of each predicted category is called a probability distribution.

[0104] S106.2: Construct a loss function L using the predicted probability distribution:

[0105]

[0106]

[0107] N represents the number of classification categories for each task, Q represents the number of query samples in each task, i and n are both natural numbers not equal to 0, I() represents the conditional function, and y represents the true category;

[0108] S106.3: Supervise the multi-granularity collaborative measurement model by optimizing the loss function L to obtain the multi-granularity collaborative measurement model for classification.

[0109] S107: Inputting the image to be classified into the multi-granularity collaborative measurement model for classification to obtain a classification result of the image to be classified, where the classification result indicates category information of the image to be classified.

[0110] This step can be implemented in the following ways:

[0111] S107.1: Input the image to be classified into the multi-granularity collaborative measurement model for classification to obtain global spatial semantic relationships and local spatial semantic relationships, which are then fused to form a multi-granularity global-local spatial semantic relationship set;

[0112] S107.2: Obtain the prototype spatial semantic relationship set based on the multi-granularity global-local spatial semantic relationship set;

[0113] S107.3: Performing a minimum sum feature set distance measurement based on the prototype space semantic relationship set to obtain a category probability. The category with the largest category probability is the classification result, and the classification result indicates the category of the image to be classified.

[0114] The beneficial technical effects of the present invention are as follows:

[0115] 1. The present invention proposes a multi-granularity collaborative measurement model to complete small-sample image classification tasks, uses spatial semantic relationships to construct a category representation with strong representation capabilities, and collaboratively measures decisions to determine categories. The present invention can obtain more accurate category representations under limited sample conditions, is more robust, and demonstrates excellent small-sample classification performance.

[0116] 2. The present invention proposes a method for fusing global spatial semantic relations and local spatial semantic relations to integrate information obtained from both.

[0117] 3. This invention proposes a minimum sum feature set distance measurement method to perform collaborative measurement decisions between multi-granularity spatial semantic relationship sets. The spatial semantic relationship set forms a more accurate representation space, and the collaborative measurement decision between sets is more reasonable.

[0118] 4. The present invention can significantly improve the ability to classify small sample images, and can achieve excellent classification performance in both natural scenes and remote sensing scenes, and has good practical application value.

[0119] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or perform equivalent replacements on some of the technical features thereof. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A small sample classification method based on collaborative metrics, characterized by: The following steps are involved: S101: constructing a first number of tasks on a labeled optical remote sensing image, each task including a support sample and a query sample; Among them, the number of supporting samples does not exceed 10; S102: Inputting the support sample and the query sample into a multi-granularity collaborative measurement model to obtain global spatial semantic relationships of different granularities and local spatial semantic relationships of different granularities; S103: Fusing the global spatial semantic relationship and the local spatial semantic relationship at each granularity to obtain a global-local fused spatial semantic relationship at each granularity; S104: combining global-local fusion spatial semantic relations of different granularities to obtain a multi-granularity spatial semantic relationship set; S105: using a minimum sum feature set distance measurement method to perform collaborative measurement decision on the multi-granularity spatial semantic relationship set to obtain a predicted probability distribution; S106: Optimizing the multi-granularity collaborative measurement model using the labels to obtain a multi-granularity collaborative measurement model for classification; S107: Inputting the image to be classified into the multi-granularity collaborative measurement model for classification to obtain a classification result of the image to be classified, where the classification result indicates category information of the image to be classified.

2. The method according to claim 1, wherein The method further comprises: The global spatial semantic relationships of different granularities indicate overall summary information of the labeled optical remote sensing image, including the global spatial semantic relationships of the supporting samples and the global spatial semantic relationships of the query samples; the local spatial semantic relationships of different granularities indicate local detail information of the labeled optical remote sensing image, including the local spatial semantic relationships of the supporting samples and the local spatial semantic relationships of the query samples; The global-local fusion spatial semantic relationship includes the global-local fusion spatial semantic relationship of the support sample and the global-local fusion spatial semantic relationship of the query sample; The multi-granularity spatial semantic relationship set includes the multi-granularity spatial semantic relationship set of the support sample and the multi-granularity spatial semantic relationship set of the query sample.

3. The method according to claim 1, wherein The collaborative measurement decision-making on the multi-granularity spatial semantic relationship set using the minimum sum feature set distance measurement method includes: S105.1: Averaging the multi-granularity spatial semantic relationship sets of the support samples to obtain a prototype spatial semantic relationship set; S105.2: Use the minimum sum feature set distance measurement method to perform collaborative measurement decision between multi-granularity spatial semantic relationship sets, and calculate the distance between the multi-granularity spatial semantic relationship set of the query sample and the prototype spatial semantic relationship set.

4. The method according to claim 1, wherein The fusing of the global spatial semantic relationship and the local spatial semantic relationship at each granularity to obtain the global-local fused spatial semantic relationship at each granularity includes: S103.1: Aggregate the local spatial semantic relations through pooling to obtain pooled local spatial semantic relations; S103.2: The global spatial semantic relationship at each granularity and the local spatial semantic relationship after pooling Add to obtain global-local fusion spatial semantic relationship The expression is as follows: Among them, i is a positive integer, representing the i-th granularity, N i represents the number of converter layers corresponding to the i-th granularity, and M represents the number of granularities.

5. The method according to claim 1, wherein Combining global-local fusion spatial semantic relations of different granularities to obtain a multi-granularity spatial semantic relationship set includes: The global-local fusion spatial semantic relations of different granularities are all collected to obtain a multi-granularity spatial semantic relationship set Set, which is expressed as follows: Represents the Nth M Global-local fusion spatial semantic relations of layers.

6. The method according to claim 3, wherein Averaging the multi-granularity spatial semantic relationship set of the support samples to obtain a prototype spatial semantic relationship set includes: Based on the multi-granularity spatial semantic relationship set of the supporting samples, the prototype spatial semantic relationship set corresponding to each category is calculated respectively. For the prototype spatial relationship set C of category k k The expression is as follows: Among them, S k Represents the number of support samples for category k, Set k Represents the set of multi-granularity prototype spatial relations of category k, represents the Nth M Global-local fusion spatial semantic relationship of layers; represents the Nth M The mean of global-local fusion spatial semantic relations at the layer level.

7. The method according to claim 3, wherein The minimum sum feature set distance measurement method is used to make collaborative measurement decisions between multi-granularity spatial semantic relationship sets. The distance between the multi-granularity spatial semantic relationship set of the query sample and the prototype spatial semantic relationship set is calculated, including: The query sample x q Multi-granularity spatial semantic relationship set Expressed as: The distance measurement between sets adopts the minimum sum feature set distance measurement method, which is expressed as: in, Represents query sample x q Nth M The global-local fusion spatial semantic relationship of the layer, D() represents the set distance metric function, summin() represents the minimum sum feature set distance metric function, and the minimum sum feature set distance metric function is expressed as: represents the Nth M The mean of the global-local fusion spatial semantic relationship of the layer, Represents query sample x q Nth M The global-local fusion spatial semantic relationship of the layer, min() represents the minimum function, d() represents the distance measurement function; i, j are positive integers, and M represents the total number of different granularities.

8. The method according to claim 2, wherein Optimizing the multi-granularity collaborative measurement model based on the label to obtain the multi-granularity collaborative measurement model for classification includes: S106.1: Obtain the predicted probability distribution based on the distance between the multi-granularity spatial semantic relationship set of the query sample and the prototype spatial semantic relationship set in, Represents the predicted category, x q Represents the query sample, p represents the predicted probability, D() represents the set distance metric function, Represents query sample x q The multi-granularity spatial semantic relationship set obtained by the multi-granularity collaborative measurement model, Set k Representing a set of semantic relationships in the prototype space of category k; the predicted probability distribution indicates the likelihood of the output prediction result; S106.2: Construct a loss function L using the predicted probability distribution: N represents the number of classification categories for each task, Q represents the number of query samples in each task, i and n are both natural numbers not equal to 0, I() represents the conditional function, and y represents the true category; S106.3: Supervise the multi-granularity collaborative measurement model by optimizing the loss function L to obtain the multi-granularity collaborative measurement model for classification.

9. The method according to claim 1, characterized in that Inputting the image to be classified into the multi-granularity collaborative measurement model for classification to obtain a classification result of the image to be classified includes: S107.1: Input the image to be classified into the multi-granularity collaborative measurement model for classification to obtain global spatial semantic relationships and local spatial semantic relationships, which are then fused to form a multi-granularity global-local spatial semantic relationship set; S107.2: Obtain a prototype spatial semantic relationship set based on the multi-granularity global-local spatial semantic relationship set; S107.3: Performing a minimum sum feature set distance measurement based on the prototype space semantic relationship set to obtain a category probability. The category with the largest category probability is the classification result, and the classification result indicates the category of the image to be classified.

Images