Small-sample learning method and system based on cross-relationship network measurement

Abstract

A few-shot learning method and system based on cross-relationship network measurement. The method includes: a feature embedding extraction step; a class feature synthesis step; a relationship feature acquisition step; a new feature acquisition step; a similarity measurement step; and a probability prediction step. When extracting features, the present invention considers the relationship between the query sample to be measured and each category of support samples, highlights the important information between the support-query pairs, and extracts more representative features. When measuring similarity, the present invention considers the relative differences within each category of the support set, reduces the distance between the query set samples and the support set samples of the same category, and increases the distance between the query set samples and the support set samples of different categories, thereby more effectively measuring the similarity between the sample to be measured and the support categories, and finally improving the accuracy of the few-shot learning image classification task.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and recognition, and particularly relates to a few-shot learning method and system based on cross-relation network measurement. Background Art

[0002] In recent years, deep learning has achieved great success in many aspects, such as image classification, text analysis, and speech recognition. Among them, the image classification task, as the basis for subsequent image object detection, segmentation, and semantic analysis, has always been a core issue in the field of computer vision. However, the great success of the current deep learning image classification task largely owes to a large number of labeled samples and huge computing power support. When the number of samples is lacking, problems such as overfitting are likely to occur, and it is difficult to generalize to new test tasks. In contrast, humans can accurately identify the samples to be tested with only a small number of samples. Therefore, studying the performance of deep learning in the case of few samples is closer to true artificial intelligence. Moreover, in many cases, the samples themselves are difficult to obtain or the labeling cost is huge, such as some precious plants and animals or medical images. Therefore, studying the performance of deep learning in the case of few samples has important practical significance.

[0003] Few-shot learning aims to achieve the correct classification of unlabeled samples to be tested given a small number of labeled samples. Generally, the given small number of labeled samples are called the support set, and the samples to be tested are called the query set. To improve the generalization ability of the deep learning model and make the training environment closer to the real test task, the current few-shot learning draws on the idea of meta-learning and adopts a scenario-based training method to train the model. The specific method is to construct a large number of tasks with the same settings as the task to be tested on the auxiliary training set according to the settings of the task to be tested, and train the model through a large number of tasks, so as to learn transferable knowledge and achieve good results in the target task to be tested.

[0004] Matching networks, first proposed by Vinyals et al. in 2016, were the first to use a context-based training mechanism. They use a feature embedding module to extract features from the test and support samples. A memory network then learns the attention weights for the test sample on each support category. Correct classification is achieved by calculating the cosine similarity between the query sample and the support category features. In 2017, Snell et al. proposed prototype networks. These networks eliminate the memory network-based attention process and instead use a mean vector approach to obtain prototypes for each support category. Correct classification is achieved by calculating the Euclidean distance between the test sample and each prototype. In addition to using cosine and Euclidean distances to measure the similarity between the test sample and each support category, Sung et al. proposed a nonlinear metric in 2018, replacing the traditional fixed metric function. These networks, called Relation Networks, employ deep neural networks to learn the similarity between the test sample and each support category, thereby achieving correct classification.

[0005] The above methods all belong to metric-based small sample learning, and their processes can be divided into a feature embedding module and a similarity measurement module. The feature embedding module usually extracts features of the support sample and the query sample through a convolutional neural network, and the similarity measurement module uses its own measurement method to compare the similarity between the features to achieve correct classification. When extracting features, the current small sample learning method uses the same or different feature embedding modules to extract their own features for the support set and the query set, without considering the relationship between the two. Therefore, the extracted features are not discriminative and difficult to distinguish when measured. When measuring similarity, the above metrics emphasize more on the absolute distance between the query set features and the features of each support category, and do not consider the differences between each support category. Therefore, it cannot effectively measure the relationship between the features of the query sample to be tested and the features of each support category.

[0006] In 2019, in order to more effectively measure the relationship between a sample to be measured and support classes, Li et al. proposed few-shot learning based on covariance measurement (CovaMNet), which classifies by measuring the second-order distribution consistency between samples and classes. In the same year, Li et al. also designed a deep nearest neighbor network (DN4) to measure the relationship between a query image and samples based on local descriptors. In 2020, in order to enable the feature embedding module to extract more representative features, Liu et al. proposed Prototype Rectification Networks, which reduce the intra-class bias and inter-class bias through a bias reduction module to obtain more accurate prototypes. In 2021, Xu et al. added a constellation module and an attention mechanism to the feature extraction network to model the relationship between features, obtaining more robust feature vectors. The above several methods are improved methods made in recent years on matching networks, prototype networks and relation networks. Although the effect has been improved to a certain extent, the learning cost and overall complexity have also increased accordingly.

[0007] In summary, the existing few-shot learning methods based on measurement have the following disadvantages:

[0008] 1. When extracting features, more emphasis is placed on the features of each support class and query sample respectively, while the relationship between them is ignored, and the extracted features are not representative;

[0009] 2. When measuring similarity, more emphasis is placed on the absolute measurement relationship between classes, while the relative measurement relationship between classes is ignored, and the similarity between the query sample to be measured and the support samples cannot be effectively measured;

[0010] 3. An additional complex network model or measurement algorithm is used for measurement, which increases the cost of few-shot learning to a certain extent. Summary of the Invention

[0011] To solve the above problems, the present invention proposes a few-shot learning method based on cross-relationship network measurement. When extracting features, the relationship between the query sample to be measured and each class of support samples is considered, highlighting the important information between the support-query pairs, and more representative features are extracted. When measuring similarity, the relative differences within each class of the support set are considered, reducing the distance of the support set that is the same class as the query set and increasing the distance of the support set that is different from the query set, so as to more effectively measure the similarity between the sample to be measured and the support classes, and finally improve the accuracy of the few-shot learning image classification task.

[0012] The present invention adopts the following technical solutions:

[0013] In a first aspect, a few-shot learning method based on cross-relationship network measurement includes:

[0014] Feature embedding extraction step: Use a feature embedding network to extract the feature maps of the support set and the query set respectively; the query set includes a number of query samples to be measured.

[0015] Class feature synthesis step: Mean fuse the feature maps of samples of the same class in the support set to obtain the class feature maps of each support set class.

[0016] Relationship feature acquisition step: Concatenate the feature maps of each class in the support set and the feature maps of the query samples to be measured on the channel, and then input them into a cross-relationship network to obtain their relationship feature maps.

[0017] New feature acquisition step: Make residual connections between the obtained relationship feature maps and the class feature maps of the original support set and the feature maps of the query samples to be measured in the query set respectively, to obtain new features in pairs of the query samples to be measured and each support set class.

[0018] Similarity measurement step: Calculate the distances between the query samples to be measured and each support set class through distance scaling to perform similarity measurement.

[0019] Probability prediction step: Convert the distances between the query samples to be measured and each support set class into probability scores to obtain the final predicted labels of the query samples to be measured.

[0020] Preferably, the feature embedding extraction step specifically includes:

[0021] Let the number of support set classes be N, and the number of samples in each support set class be K. represents the kth sample of the nth class in the support set. represents the ith query sample to be measured in the query set, and its label is unknown.

[0022] Represent the support set as where y n ∈ {y1, y2,..., y N} is the label of the support set, and the feature map obtained after passing through the feature embedding network is represented as c, h, and w respectively represent the channels, height, and width of the output feature map obtained by the support set samples passing through the feature embedding network.

[0023] Represent the query set as The feature map obtained by the query set passing through the feature embedding network is represented as c, h, and w respectively represent the number of channels, height, and width of the output feature map obtained by the feature embedding network for the query sample to be measured, that is, the number of channels, height, and width of the output feature map obtained by the feature embedding network for the query sample to be measured are the same as those of the output feature map obtained by the feature embedding network for the support set samples.

[0024] Preferably, in the class feature synthesis step, the class feature map of the nth class support set is represented by the following formula:

[0025]

[0026] where

[0027] Preferably, in the relationship feature acquisition step, the relationship feature map between the class feature map of the nth class support set and the query sample to be measured is represented by the following formula: is represented by the following formula:

[0028]

[0029] where g θ is the cross-relationship network, and θ is the parameter of the cross-relationship network; the input of the cross-relationship network is indicating concatenating the class feature map of the support set and the feature map of the query sample to be measured on the channel; the relationship feature map

[0030] Preferably, in the new feature acquisition step, the new feature acquisition formula is as follows:

[0031]

[0032]

[0033] where represents the new feature obtained by the nth class support set through the cross-relationship network, represents the new feature obtained by the i-th query set sample through the cross-relationship network,

[0034] After inputting the class feature of each class support set and the query sample to be measured into the cross-relationship network, the new features in pairs of the query sample to be measured and each category are obtained where Performing the same operation on each query sample to be measured in the query set as the i-th query sample to be measured, the new features in pairs of each query sample to be measured and each support set category are obtained where C S-Q ,

[0035] Preferably, in the similarity measurement step, the query sample to be tested and the nth class support set S n distance The distance is expressed as follows:

[0036]

[0037] where d(·,·) represents the normalized Euclidean distance between two vectors; assume that the features and extracted from the nth class support set and the ith query sample to be tested after the feature embedding network and the cross-relationship network are M-dimensional vectors, then the normalized Euclidean distance between the two features is expressed as:

[0038]

[0039] where and represent the values of the normalized features on the mth dimension, and are the norms of the feature vectors:

[0040]

[0041]

[0042] Regarding any class support set as a positive sample and the remaining support set classes as negative samples, the new distance between the query sample to be tested and each support set class can be obtained Performing the above operations on all query samples to be tested to obtain the new distance d' S-Q of the query set and the support set, d' S-Q ∈R I×N .

[0043] Preferably, the probability prediction step specifically includes:

[0044] Converting the distances between the query sample to be tested and each support set class into probability scores, which is implemented using the softmax function, as follows:

[0045]

[0046] where represents the predicted label of the query sample to be tested , represents the probability that the query sample to be tested belongs to the nth class of the support set, represents the distance between the query sample to be tested and the nth class support set;

[0047] Set the class with the highest class probability score P as the final predicted label of the query sample to be tested.

[0048] In a second aspect, a few-shot learning system based on cross-relationship network metric includes:

[0049] A feature embedding extraction module, configured to use a feature embedding network to extract feature maps of a support set and a query set respectively; the query set includes several query samples to be tested;

[0050] A class feature synthesis module, configured to fuse the feature maps of samples of the same class in the support set by averaging to obtain class feature maps of each support set class;

[0051] A relationship feature acquisition module, configured to splice the feature maps of each class in the support set and the feature map of the query sample to be tested on the channel, and then input them into a cross-relationship network to obtain a relationship feature map of the two;

[0052] A new feature acquisition module, configured to perform residual connection on the obtained relationship feature map with the class feature map of the original support set and the feature map of the query sample to be tested in the query set respectively to obtain paired new features of the query sample to be tested and each support set class;

[0053] A similarity metric module, configured to perform similarity metric by calculating the distance between the query sample to be tested and each support set class through distance scaling;

[0054] A probability prediction module, configured to convert the distance between the query sample to be tested and each support set class into a probability score to obtain the final predicted label of the query sample to be tested.

[0055] In a third aspect, an image classification method includes the few-shot learning method based on cross-relationship network metric as described above, and further includes:

[0056] A classification result acquisition step, taking the final predicted label of the query sample to be tested obtained as the classification result of the query sample to be tested.

[0057] In a fourth aspect, an image classification system includes the few-shot learning system based on cross-relationship network metric as described above, and further includes:

[0058] A classification result acquisition module, configured to take the final predicted label of the query sample to be tested obtained as the classification result of the query sample to be tested.

[0059] The present invention has the following beneficial effects:

[0060] (1) Based on the feature embedding extraction, the present invention further learns the relationship between the support set categories and the features of the samples in the query set to be measured, obtains the corresponding relationship features between them, and uses them to enhance the original features of the support set categories and the samples in the query set to be measured. Compared with the feature extraction of the prior art, the finally obtained support set and query set features in the present invention are in one-to-one correspondence, so the discrimination ability and differentiation degree are better;

[0061] (2) The present invention calculates the distance between the query sample to be measured and each support set category through distance scaling for similarity measurement. It not only considers the absolute distance between the query sample to be measured and the features of each support set category, but also reduces the distance between the features of the same category as the query sample to be measured according to the differences of each support set category, and increases the distance from the features of different categories from the query sample to be measured, thus more effectively comparing the similarity between the query sample to be measured and each support set category;

[0062] (3) Compared with the existing few-shot learning methods, the present invention only adds a 1×1 convolutional network to extract the cross-relationship between the query sample to be measured and each support set category. The learning cost is low and the effect is good. The whole process can adopt an end-to-end training method.

[0063] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments, but the present invention is not limited to the embodiments. Description of the Drawings

[0064] Figure 1 It is a flowchart of the few-shot learning method based on cross-relationship network measurement according to the embodiment of the present invention;

[0065] Figure 2 It is a schematic diagram of the few-shot learning method based on cross-relationship network measurement according to the embodiment of the present invention;

[0066] Figure 3 It is a specific schematic diagram of the feature embedding extraction step and the class feature synthesis step according to the embodiment of the present invention;

[0067] Figure 4 It is a specific schematic diagram of the relationship feature acquisition step and the new feature acquisition step according to the embodiment of the present invention;

[0068] Figure 5 It is a structural block diagram of the few-shot learning system based on cross-relationship network measurement according to the embodiment of the present invention;

[0069] Figure 6 It is a flowchart of the image classification method according to the embodiment of the present invention;

[0070] Figure 7 It is a structural block diagram of the image classification system according to the embodiment of the present invention.

[0071] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. DETAILED DESCRIPTION

[0072] The present invention is further described below through specific implementation modes.

[0073] See also Figure 1 and Figure 2 As shown, this embodiment provides a small sample learning method based on cross-relationship network metrics, including:

[0074] In a feature embedding extraction step S101, a feature embedding network is used to extract feature graphs of a support set and a query set respectively; the query set includes a number of query samples to be tested;

[0075] Class feature synthesis step S102, fusion of feature maps of samples of the same category in the support set by mean, to obtain class feature maps of each support set category;

[0076] In step S103 of acquiring relationship features, the feature graphs of each category in the support set and the feature graph of the query sample to be tested are spliced on the channel, and then input into the cross-relationship network to obtain the relationship feature graph between the two.

[0077] New feature acquisition step S104: residual connection is performed on the obtained relationship feature graph with the class feature graph of the original support set and the feature graph of the query sample to be tested in the query set, to obtain new features of the query sample to be tested and each support set class pair;

[0078] Similarity measurement step S105, performing similarity measurement by calculating the distance between the query sample to be tested and each support set category through distance scaling;

[0079] In the probability prediction step S106 , the distance between the query sample to be tested and each support set category is converted into a probability score to obtain the final predicted label of the query sample to be tested.

[0080] In this embodiment, the support set S and the query set Q are first extracted from their respective feature maps through the feature embedding network and represented as a three-dimensional tensor. The feature maps in each category of the support set are fused by means of mean to obtain the class feature map of this category, which is used to represent the features of this category. The query feature map and the class feature map of each support set are then input into the cross-relationship network, and the relationship between the query sample to be tested and each support category is learned and the relationship feature map of the two is obtained. Then, the class feature map of the support set and the feature map of the query sample are respectively connected with the relationship feature map by residual connection to obtain new feature maps. Finally, the similarity between each feature is compared using a similarity metric, and the probability that the query sample to be tested belongs to each support category is obtained after passing through the softmax probability function.

[0081] The specific processing procedure of the feature embedding extraction step S101 is as follows.

[0082] Input the samples of the support set S and the query set Q into the feature embedding network to obtain the feature maps of the support set and the query set respectively. Similar to most metric-based few-shot learning, the feature embedding network in this embodiment uses a deep convolutional neural network (CNN) to extract features. Here, is denoted as, representing the network parameters.

[0083] Assume that the number of support set categories is N, and the number of samples in each support set category is K. denotes the k-th sample of the n-th category in the support set, denotes the i-th test query sample in the query set, whose label is unknown.

[0084] The support set can be expressed as where y n ∈{y1, y2,..., y N} is the label of the support set. The feature map obtained after passing through the feature embedding network can be expressed as c, h, and w are the channels, height, and width of the output feature map obtained by the support set samples after passing through the feature embedding network respectively.

[0085] The query set can be expressed as The feature map obtained by the query set after passing through the feature embedding network can be expressed as c, h, and w are the channels, height, and width of the output feature map obtained by the test query samples after passing through the feature embedding network respectively, that is, the channels, height, and width of the output feature map obtained by the test query samples after passing through the feature embedding network are the same as those of the output feature map obtained by the support set samples after passing through the feature embedding network.

[0086] The specific processing procedure of the class feature synthesis step S102 is as follows.

[0087] Perform mean fusion on the feature maps obtained by the samples of the same category in the support set after passing through the feature embedding network to obtain the class feature map of this category. The calculation formula for the class feature map of the n-th class support set is as follows:

[0088]

[0089] For the specific schematic diagrams of the feature embedding extraction step and the class feature synthesis step, please refer to Figure 3 as shown.

[0090] The specific processing procedure of the relationship feature acquisition step S103 is as follows.

[0091] The class feature maps of various types in the support set and the feature map of the query sample to be tested are input into the cross-relationship network, and the relationship feature map between the two is obtained through the cross-relationship network. Through a large number of scenario-based trainings, the relationship feature map obtained by this network contains both the features of the support set category and the features of the query sample to be tested, and some features that are the same between the two are strengthened, while some unimportant information is ignored. The class feature of the nth type of support set and the query sample to be tested of the relationship feature map can be obtained by the following formula:

[0092]

[0093] where g θ is the cross-relationship network of this embodiment, θ is the parameter of this network. After multiple experiments, this embodiment finally adopts a 1×1 convolutional neural network as the cross-relationship feature extraction layer. The input of the cross-relationship network is indicating that the class features of the support set and the features of the query sample to be tested are concatenated on the channel. In order to keep the shape of the relationship feature map consistent with the original class feature map of the support set and the feature map of the query sample to be tested, the finally obtained relationship feature map

[0094] The specific processing process of the new feature acquisition step S104 is as follows.

[0095] In order to ensure that the obtained relationship features still do not lose the original features of the support set and the query set, the obtained relationship feature map is respectively subjected to residual connection with the class feature map of the original support set and the feature map of the query sample to be tested in the query set, and new, more discriminative features are obtained. Taking the class feature map of the nth type of support set and the i-th query sample to be tested as an example, the new feature acquisition formula is as follows:

[0096]

[0097]

[0098] where is the new feature obtained by the nth type of support set through the cross-relationship network, is the new feature obtained by the i-th query set sample through the cross-relationship network,

[0099] And so on, after inputting the class features of each type of support set and the query sample to be tested into the cross-relationship network, the new features paired with each support set category for the query sample to be tested are obtained where Perform the same operations on each sample in the query set as the i-th query sample to be tested, and obtain new features in pairs for each query sample to be tested and each support category where C S-Q , The specific process can be represented by the following pseudocode

[0100] for(i = 1; i <= I; i = i + 1)

[0101] for(n = 1; n <= N; n = n + 1)

[0102]

[0103]

[0104] Obtain where C S-Q ,

[0105] Obtain C S-Q , where C S-Q ,

[0106] In this way, for the support set, the new features contain both the original features extracted by the feature embedding module, and the acquisition of this feature is only related to the samples within this category itself; and it also contains the features including the query samples to be tested obtained through the cross-relationship network. Moreover, compared with the fact that the original support set features are the same for all query samples to be tested, the new features are in one-to-one correspondence with each query sample to be tested, including the relationship between pairs, so they are more discriminative and distinguishable. Similarly, the same is true for the query set samples

[0107] For the specific principles of the relationship feature acquisition step and the new feature acquisition step, please refer to Figure 4 as shown

[0108] The specific processing process of the similarity measurement step S105 is as follows

[0109] After being processed by the feature embedding network and the cross-relationship network, the samples in the support set and the query set are both mapped into a feature space. In this feature space, the closer the distance, the greater the similarity. Since only one class in the support set is of the same class as the query sample to be tested, it is called the positive sample class, and the other classes are of different classes from the query sample to be tested, so they are called negative sample classes. In actual measurement, it is desired to reduce the distance of the support set belonging to the same class as the query sample to be tested and increase the distance of the remaining support set belonging to different classes from the query sample to be tested, which is called distance scaling. Based on this idea, this embodiment proposes a new distance measurement method, with the query sample to be tested For example, its distance from the n-th class support set S n is The distance can be expressed as:

[0110]

[0111] where represents the normalized Euclidean distance; and The normalized Euclidean distance between them is as follows:

[0112]

[0113] In the formula, d(·,·) represents the normalized Euclidean distance between two vectors. Assume that the features and extracted from the n-th class support set and the i-th query sample to be tested after passing through the feature embedding network and the cross-relationship network are M-dimensional vectors. The normalized Euclidean distance between these two features can be expressed as:

[0114]

[0115] where and represent the values of the normalized features in the m-th dimension, and are the norms of the feature vectors:

[0116]

[0117]

[0118] Regarding any class support set as a positive sample and the remaining support set classes as negative samples, the new distances between the query samples to be tested and each support set class can be obtained Performing the above operations on all query samples to be tested, the new distance d' between the query set and the support set can be obtained S-Q , d' S-Q ∈R I×N . The specific process can be described by the following pseudocode.

[0119] for(i = 1; i <= I; i = i + 1)

[0120] for(n = 1; n <= N; n = n + 1)

[0121]

[0122] Obtain

[0123] Obtain d S-Q , d S-Q ∈RI×N

[0124] It can be seen that compared with the original distance, the new distance not only considers the absolute distance between the query sample to be measured and each support set category, but also considers the relative distance within the support set category, shortening the distance between the query sample to be measured and the same category within the support set category, while increasing the distance from the query sample to be measured to different categories, and can more effectively compare the similarity between the query sample to be measured and each support set category.

[0125] The specific processing process of the probability prediction step S106 is as follows.

[0126] Convert the distance between the query sample to be measured and each support set category into a probability score, which is implemented using the softmax function.

[0127]

[0128] Among them, represents the predicted label of the query sample to be measured ; is the probability that the query sample to be measured belongs to the nth category of the support set, is the distance between the query sample to be measured and the nth category support set, which can be obtained from step 105.

[0129] The model parameters of the few-shot learning method based on the cross-relationship network metric in this embodiment are mainly divided into two parts. One part comes from the feature embedding network is the network parameter of this network; the other part comes from the cross-relationship network g θ , θ is the network parameter of this network. Therefore, the model can be trained using cross-entropy loss:

[0130]

[0131] Among them, represents the true label of the i-th query sample to be measured ; I(·) is an indicator function used to reflect whether the prediction is correct, that is, when the prediction is correct, I(·) = 0, otherwise I(·) = 1. The final predicted label of the query sample to be measured is the class with the highest class probability score P.

[0132] By randomly extracting a large number of tasks from the auxiliary training set for scenario training, continuously optimizing and iterating the model, and finding the optimal parameters to minimize the loss to obtain the final feature embedding network model and cross-relationship network model.

[0133] See Figure 5 As shown, according to another aspect of the present invention, this embodiment also discloses a few-shot learning system based on the cross-relationship network metric, including:

[0134] A feature embedding extraction module 501 is configured to use a feature embedding network to extract feature graphs of a support set and a query set, respectively; the query set includes a plurality of query samples to be tested;

[0135] Class feature synthesis module 502, used to fuse the feature maps of samples of the same category in the support set by means of the mean to obtain the class feature maps of each support set category;

[0136] The relationship feature acquisition module 503 is used to splice the feature graphs of each category in the support set and the feature graph of the query sample to be tested on the channel, and then input them into the cross-relationship network to obtain the relationship feature graph between the two;

[0137] New feature acquisition module 504 is used to perform residual connection on the obtained relationship feature graph with the class feature graph of the original support set and the feature graph of the query sample to be tested in the query set, so as to obtain new features of the query sample to be tested and each support set class pair;

[0138] A similarity measurement module 505 is configured to perform similarity measurement by calculating the distance between the query sample to be tested and each support set category through distance scaling;

[0139] The probability prediction module 506 is used to convert the distance between the query sample to be tested and each support set category into a probability score to obtain the final predicted label of the query sample to be tested.

[0140] Specific implementation of a small sample learning system based on cross-relationship network metric The same small sample learning method based on cross-relationship network metric is not repeated in this embodiment.

[0141] For further information, see Figure 6 As shown, this embodiment also discloses an image classification method, including:

[0142] In a feature embedding extraction step S601, a feature embedding network is used to extract feature graphs of a support set and a query set respectively; the query set includes a number of query samples to be tested;

[0143] Class feature synthesis step S602, fusion of feature maps of samples of the same category in the support set by mean, to obtain class feature maps of each support set category;

[0144] Relationship feature acquisition step S603: The feature graphs of each category in the support set and the feature graph of the query sample to be tested are spliced on the channel, and then input into the cross-relationship network to obtain the relationship feature graph between the two;

[0145] New feature acquisition step S604: Residual connections are made between the obtained relational feature maps and the class feature maps of the original support set and the feature maps of the to-be-tested query samples in the query set respectively, to obtain new features paired between the to-be-tested query samples and each support set category.

[0146] Similarity measurement step S605: Similarity measurement is performed by calculating the distances between the to-be-tested query samples and each support set category through distance scaling.

[0147] Probability prediction step S606: The distances between the to-be-tested query samples and each support set category are converted into probability scores to obtain the final predicted labels of the to-be-tested query samples.

[0148] Classification result acquisition step 607: The final predicted labels of the to-be-tested query samples obtained are used as the classification results of the to-be-tested query samples.

[0149] The image classification method of this embodiment can improve the accuracy of the small-sample learning image classification task.

[0150] In addition, as shown in Figure 7 This embodiment also discloses an image classification system, including:

[0151] Feature embedding extraction module 701, configured to use a feature embedding network to extract the feature maps of the support set and the query set respectively; several to-be-tested query samples are included in the query set.

[0152] Class feature synthesis module 702, configured to obtain the class feature maps of each support set category by mean fusion of the feature maps of samples of the same category in the support set.

[0153] Relational feature acquisition module 703, configured to splice the feature maps of each class in the support set and the feature maps of the to-be-tested query samples on the channel, and then input them into a cross-relational network to obtain the relational feature maps of the two.

[0154] New feature acquisition module 704, configured to make residual connections between the obtained relational feature maps and the class feature maps of the original support set and the feature maps of the to-be-tested query samples in the query set respectively, to obtain new features paired between the to-be-tested query samples and each support set category.

[0155] Similarity measurement module 705, configured to perform similarity measurement by calculating the distances between the to-be-tested query samples and each support set category through distance scaling.

[0156] Probability prediction module 706, configured to convert the distances between the to-be-tested query samples and each support set category into probability scores to obtain the final predicted labels of the to-be-tested query samples.

[0157] The classification result acquisition module 707 is configured to use the final predicted label of the query sample to be measured as the classification result of the query sample to be measured.

[0158] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A small-sample learning method based on cross-relationship network measurement, characterized in that include: A feature embedding extraction step, using a feature embedding network to extract feature graphs of a support set and a query set respectively; the query set includes a number of query samples to be tested; In the class feature synthesis step, the feature maps of samples of the same category in the support set are fused by mean to obtain the class feature maps of each support set category; In the relational feature acquisition step, the feature graphs of each type in the support set and the feature graph of the query sample to be tested are spliced on the channel, and then input into the cross-relationship network to obtain the relational feature graph of the two; In the new feature acquisition step, the obtained relationship feature graph is residually connected with the class feature graph of the original support set and the feature graph of the query sample to be tested in the query set, thereby obtaining new features that are paired with the query sample to be tested and each support set category; Similarity measurement step, which calculates the distance between the query sample to be tested and each support set category through distance scaling to perform similarity measurement; The probability prediction step converts the distance between the query sample to be tested and each support set category into a probability score to obtain the final predicted label of the query sample to be tested; In the similarity measurement step, the query sample to be tested and the support set S of the nth class n distance The distance is expressed as follows: Where d(·,·) represents the normalized Euclidean distance between two vectors; Represents the new features obtained by the cross-relationship network of the n-th support set, Represents the new features obtained by the cross-relationship network of the i-th query set sample, N represents the number of support set categories; assuming that the n-th support set and the i-th query sample are extracted after the feature embedding network and the cross-relationship network and is an M-dimensional vector, then the normalized Euclidean distance between two features is expressed as: Among them, and represent the values of the normalized features on the m-th dimension, and is the norm of the feature vector: Regarding any type of support set as positive samples and the remaining support set categories as negative samples, the new distances between the query samples to be measured and each support set category can be obtained. Performing the above operations on all query samples to be measured, the new distance d' between the query set and the support set is obtained. S-Q , d' S-Q ∈R I×N ; The probability prediction step specifically includes: Convert the distance between the query sample to be tested and each support set category into a probability score using the softmax function as follows: Among them, represents the predicted label of the query sample to be tested , represents the probability that the query sample to be tested belongs to the nth class of the support set represents the distance between the query sample to be tested and the nth class support set; The class with the highest class probability score P is set as the final predicted label of the query sample to be tested.

2. The small-sample learning method based on cross-relationship network measurement according to claim 1, wherein The feature embedding and extraction step specifically includes: Let the number of support set classes be N, and the number of samples in each support set class be K. denotes the k-th sample of the n-th class in the support set. denotes the i-th query sample to be tested in the query set, whose label is unknown. Represent the support set as where y n ∈{y1, y2,..., y N} is the label of the support set, and the feature map obtained after passing through the feature embedding network is represented as c, h, and w respectively represent the number of channels, height, and width of the output feature map obtained by the support set samples through the feature embedding network; Represent the query set as Represent the feature map obtained after the query set passes through the feature embedding network as Let c, h, and w represent the number of channels, height, and width of the output feature map obtained after the query sample to be tested passes through the feature embedding network, that is, the number of channels, height, and width of the output feature map obtained after the query sample to be tested passes through the feature embedding network are the same as those of the output feature map obtained after the support set sample passes through the feature embedding network.

3. The few-shot learning method based on cross-relation network metric according to claim 2, characterized in that In the class feature synthesis step, the class feature map of the n-th class support set is represented by the following formula: Among them, 4. The few-shot learning method based on cross-relationship network metric according to claim 3, wherein In the step of obtaining the relationship feature, the class feature map of the nth type of support set and the query sample to be tested The relationship feature map of is represented by the following formula: Among them, g θ is a cross-relation network, and θ is the parameter of the cross-relation network; the input of the cross-relation network is indicating concatenating the class feature map of the support set and the feature map of the query sample to be tested on the channel; the relation feature map 5. The few-shot learning method based on cross-relation network metric according to claim 4, wherein In the new feature acquisition step, the new feature acquisition formula is as follows: After inputting the class features of each type of support set and the query sample to be tested into the cross-relationship network, new features paired with each category for the query sample to be tested are obtained. Among them, Perform the same operation as the i-th query sample to be tested on each query sample to be tested in the query set, and new features paired with the categories of each support set for each query sample to be tested are obtained. Among them, 6. A few-shot learning system based on cross-relationship network measurement, characterized in that Based on the small sample learning method based on cross-relationship network metric according to any one of claims 1 to 5, the system includes: A feature embedding extraction module is used to extract feature graphs of a support set and a query set respectively using a feature embedding network; the query set includes a number of query samples to be tested; The class feature synthesis module is used to fuse the feature maps of samples of the same category in the support set through mean fusion to obtain the class feature maps of each support set category; The relationship feature acquisition module is used to splice the feature graphs of various types in the support set with the feature graphs of the query sample to be tested on the channel, and then input them into the cross-relationship network to obtain the relationship feature graph between the two; The new feature acquisition module is used to perform residual connections on the obtained relationship feature graph with the class feature graph of the original support set and the feature graph of the query sample to be tested in the query set, thereby obtaining new features that are paired with the query sample to be tested and each support set category; The similarity measurement module is used to calculate the distance between the query sample to be tested and each support set category through distance scaling to perform similarity measurement; The probability prediction module is used to convert the distance between the query sample to be tested and each support set category into a probability score to obtain the final predicted label of the query sample to be tested.

7. An image classification method, characterized in that, The method for learning a small sample size based on a cross-relationship network metric according to any one of claims 1 to 5 further comprises: The classification result acquisition step is used to use the obtained final predicted label of the query sample to be tested as the classification result of the query sample to be tested.

8. An image classification system, characterized in that, The small sample learning system based on cross-relationship network metrics according to claim 6 further includes: The classification result acquisition module is used to use the obtained final predicted label of the query sample to be tested as the classification result of the query sample to be tested.

Images