A small sample image classification method based on multi-level distributed propagation

Abstract

The application discloses a kind of small sample image classification methods of multistage distribution propagation, first, image dataset is acquired, contour segmentation is carried out to the image to intercept target area picture and do feature extraction, then the feature vector extracted is used as sample;GMDP module is constructed, and it is trained using training set sample, test set sample is input into the GMDP module trained, and the category of target object in corresponding image can be directly predicted and output.The MDPN network proposed in the application is optimized for the noise interference problem in small sample classification task and the multi-level feature extraction problem, through the target area positioning method based on instance segmentation, the influence of background and pseudo target on small sample classification is greatly reduced, and the multi-level distribution features of image are obtained by cascading GNN.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision, and particularly relates to a multi-level distributed propagation small sample image classification method. BACKGROUND

[0002] Few-shot learning (FSL) usually needs prior knowledge to construct a neural network, which can be generally divided into three ways: (1) using external memory; (2) introducing prior knowledge in model initialization parameters; and (3) using training data as prior knowledge.

[0003] The learning method using external memory is to store the image features of the training set into an external memory, and then compare the test sample with the features read from the external memory during the test, so as to predict the label of the test sample. The document [Santoro A, Bartunov S, Botvinick M, et al. Meta-learning with memory-augmented neural networks [C] / / International conference on machine learning. PMLR, 2016: 1842-1850] first proposed the idea of using external memory to solve the small sample learning task in 2016, and the memory-augmented neural network (MANN) proposed by them can solve the instability of LSTM. The document [Cai Q, Pan Y, Yao T, et al. Memory matching networks for one-shot image recognition [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 4080-4088] proposes a memory matching network, which extracts the support set features and stores them in the storage module together with the corresponding class labels to form a "key-value pair". The document [Kaiser L, Nachum O, Roy A, et al. One-shot learning with self-similarity networks [C] / / International conference on machine learning. PMLR, 2018: 2801-2810] proposes a self-similarity network, which uses the cosine similarity between the query sample and the support set to predict the label of the query sample. However, the additional storage space and computing resources will increase the cost of training. Nachum O, Roy A, et al. Learning to remember rare events [J]. arXiv preprint arXiv: 1703.03129, 2017] proposes a lifelong memory module, which uses the nearest neighbor idea to select the k samples closest to the query sample, and predicts the label of the sample accordingly. However, it should be noted that the additional storage space and computing resources will increase the cost of training.

[0004] The prior knowledge is introduced into the model initialization parameters, that is, the meta-learning method, which aims to enable the model to obtain learning ability and make it able to automatically learn some meta-knowledge (meta-knowledge is more basic knowledge that constitutes knowledge itself and knowledge that controls and regulates knowledge). A gradient-based method MAML proposed in 2017 [Finn C, Abbeel P, Levine S. Model-agnostic meta-learning for fast adaptation of deep networks [C] / / International conference on machine learning. PMLR, 2017: 1126-1135] designs a meta-learner as an optimizer that learns the model parameter updates in new classes with only a few optimization steps. Meta-SGD [Li Z, Zhou F, Chen F, et al. Meta-sgd: Learning to learn quickly for few-shot learning [J]. arXiv preprint arXiv: 1707.09835, 2017] developed based on MAML can learn the optimization direction and rate. In addition, Reptile [Nichol A, Achiam J, Schulman J. On first-order meta-learning algorithms [J]. arXiv preprint arXiv: 1803.02999, 2018] proposed by Alex et al. in 2018 avoids the calculation of two derivatives in MAML, greatly reducing the amount of calculation. MetaOptNet [Lee K, Maji S, Ravichandran A, et al. Meta-learning with differentiable convex optimization [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2019: 10657-10665] proposes to replace the nearest neighbor method with a linear classifier that can be optimized as convex optimization learning.

[0005] The way of using training data as prior knowledge is divided into fine-tuning-based method and metric-based method, the former idea is to pre-train the model using a large amount of data, and then fine-tune using the target small sample dataset; the latter idea is to extract the features of the samples through the network, and analyze the feature distance to obtain a network that can distinguish different categories. Many classic networks of small sample classification are based on the metric method, such as MatchingNet[Vinyals O, Blundell C, Lillicrap T, et al. Matching networks for one shot learning[J]. Advances in neural information processing systems, 2016, 29] generates a weighted nearest neighbor classifier by calculating the mapping distance between the support set and the query set; the ProtoNet proposed in the literature [Snell J, Swersky K, Zemel R. Prototypical networks for few-shot learning[J]. Advances in neural information processing systems, 2017, 30] extracts the prototype features from the samples of the same category, and then compares the Euclidean distance between the query sample features and the prototype features to predict the sample category. RelationNet[Sung F, Yang Y, Zhang L, et al. Learning to compare: Relation network for few-shot learning[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 1199-1208] uses an adaptive nonlinear classifier to measure the relationship between the features of the support set and the query set.

[0006] A graph is a data structure consisting of points and edges connecting these points, with the advantages of strong expressiveness and intuitive display, and its essence is information propagation, spreading messages from each node to surrounding nodes, also known as neighborhood aggregation. Although information integration is carried out between neighbor nodes, after several iterations, the information of each node will contain the topology of the entire graph structure, node information and the history of the entire dynamic process. Graph neural network (GNN) is a deep learning-based graph information processing model, and GNN expresses the function of information propagation between nodes in the graph by using neural networks. Due to its strong relationship expression ability, GNN has become a widely used graph analysis method [Zhou J, Cui G, Hu S, et al. Graph neural networks: A review of methods and applications [J]. AI open, 2020, 1: 57-81]. Moreover, GNN has many variants, such as graph convolutional network (GCN), gated graph neural network and graph attention network (GAN), etc.

[0007] In recent years, GNN has been widely used in the field of few-shot learning. The literature [Garcia V, Bruna J. Few-shot learning with graph neural networks[J]. arXiv preprint arXiv:1711.04043, 2017] first proposed to use GNN to solve the problem of few-shot image classification in 2018. In GNN, each sample is regarded as a node in the graph, and the embedding vector of each node is learned and updated through GNN, and then the edge vector is updated through the node vector. The transductive propagation network (TPN) proposed in the literature [Liu Y, Lee J, Park M, et al. Learning to propagate labels: Transductive propagation network for few-shot learning[J]. arXiv preprint arXiv:1805.10002, 2018] uses the entire query set for transductive inference to further utilize the intra-class similarity and inter-class difference. The edge-labeling graph neural network proposed in the literature [Kim J, Kim T, Kim S, et al. Edge-labeling graph neural network for few-shot learning[C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2019: 11-20] labels the edge. Two dimensions of edge features correspond to the intra-class similarity features and inter-class difference features of the two nodes connected by the edge, respectively. Finally, binary classification is performed on the edge to determine whether the two nodes belong to the same class. The distribution propagation graph network (DPGN) proposed in the literature [Yang L, Li L, Zhang Z, et al. Dpgn: Distribution propagation graph network for few-shot learning[C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2020: 13390-13399] constructs an explicit class distribution relationship. The literature [Gidaris S, Komodakis N.Generating classification weights with gnn denoisingautoencoders for few-shot learning[C] / / Proceedings of the IEEE / CVF conferenceon computer vision and pattern recognition.2019:21-30] added denoising autoencoders (DAE) in GNN to correct the weights of small sample classes. The powerful information propagation property and relationship expression ability of GNN are worth further exploring. SUMMARY

[0008] Due to the limited number of labeled samples, the small sample image classification task is often disturbed by noise interference and insufficient feature extraction. In view of this, the present application provides a multilevel distribution propagation small sample image classification method. The performance of the image classification model MDPN (Multilevel Distribution Propagation Network for Few-shot Image Classification) proposed by the present application exceeds that of most existing small sample classification methods under the same setting.

[0009] A multilevel distribution propagation small sample image classification method, comprising the following steps:

[0010] (1) Obtain an image data set, wherein each image is labeled with a class label of its target object, and divide all images in the data set into a training set and a test set;

[0011] (2) Extract a target region picture by contour segmentation of the image, and then extract a feature vector of the image by feature extraction of the target region picture;

[0012] (3) Construct a GMDP (graph-based multilevel distribution propagation) module, and train the training set images using the feature vectors of the training set images;

[0013] (4) Input the feature vectors of the test set images into the trained GMDP module, and directly predict and output the class of the target object in the corresponding image.

[0014] Furthermore, in step (2), the IOSL (instance-segmentation-based object localization) module is used to perform contour segmentation on the image to extract the target region image. First, the outer contour of the target object region in the image is accurately segmented using prior knowledge in the dataset. Then, a mask is used to cover the non-target object region. Finally, the target region image is obtained by cropping along the minimum bounding rectangle of the target object region.

[0015] Furthermore, in step (2), the Backbone module is used to feature the target region image. It uses ResNet12 as the feature extractor, and the last feature is processed by global average pooling and then passed through a fully connected layer with batch normalization to obtain a feature vector with a dimension of 128 as the output.

[0016] Furthermore, the GMDP module, based on a multi-level distributed propagation of a graph neural network, consists of multiple iterations, with each iteration consisting of an instance graph. Primary distribution map Secondary distribution map Level 3 Distribution Map The components are connected sequentially, where the subscript l is a natural number representing the iteration number; the instance feature V is used. l point Construction example diagram The nodes in the calculation edge features (its representative V) l point The distribution of and The fusion process yields the instance features V. l d ist_1 ; use V l dist_1 Construction distribution map The nodes in and according to The order of arrangement is then used to calculate... edge features That is, the first-order distribution (represented by V) l dist_1 The distribution), and Directly used as instance feature V l dist_2 ; use V l dist_2 Construction distribution map The nodes in and according to The order of arrangement is then used to calculate... edge features That is, a second-order distribution (represented by V). l dist_2 The distribution), and Directly used as instance feature V l dist_3 ; use V l dist_3 Construction distribution map The nodes in and according to The order of arrangement is then used to calculate... edge features That is, a three-level distribution (represented by V). l dist_3 The distribution of With V l point Instance features are obtained by fusion. Used to construct the instance graph for the next iteration. By connecting the nodes in the loop, one iteration is completed.

[0017] Furthermore, the instance features When l = 0, The expression is as follows:

[0018]

[0019] in: Represents the instance graph of the l-th iteration. The feature of the i-th node, x i For the i-th sample in a training episode, f emb () represents the backbone network, which uses ResNet12, i = 1, 2, ..., T, where T is the total number of samples in a training episode (including support samples and query samples).

[0020] Furthermore, the edge features When l = 0, The expression is as follows:

[0021]

[0022] When l > 0, The expression is as follows:

[0023]

[0024] in: Represents the instance graph of the l-th iteration. The edge characteristics between the i-th node and the j-th node. Represents the instance graph of the l-th iteration. The features of the j-th node, This is an encoding network that transforms a distribution into a deterministic metric. It consists of two convolutional modules and a sigmoid layer connected in sequence. The input of the convolutional module is processed by the convolutional layer, batch normalization and ReLU activation function in sequence before outputting, j = 1, 2, ..., T.

[0025] Furthermore, the instance features When l = 0, The expression is as follows:

[0026]

[0027] When l > 0, The expression is as follows:

[0028]

[0029] in: Represents the distribution diagram of the l-th iteration. The features of the i-th node, Represents the instance graph of the l-th iteration. The edge characteristics between the i-th node and the k-th node, where || represents the concatenation operator, δ() represents the Kronecker function, and x i For the i-th sample in a training episode, y i and y k Let N be the class labels of the i-th and k-th samples in a training episode, respectively. sup P2D() represents the number of support samples in a training episode. It first concatenates two sets of features, and then transforms the concatenated features through a fully connected layer and a ReLU activation function before outputting the result. i = 1, 2, ..., T, where T is the total number of samples in a training episode.

[0030] Furthermore, the edge features When l = 0, The expression is as follows:

[0031]

[0032] When l > 0, The expression is as follows:

[0033]

[0034] in: Represents the distribution diagram of the l-th iteration. The edge characteristics between the i-th node and the j-th node. Represents the distribution diagram of the l-th iteration. The features of the j-th node, This is an encoding network that transforms a distribution into a deterministic metric. It consists of two convolutional modules connected together. The input of each convolutional module is processed sequentially through a convolutional layer, batch normalization, and ReLU activation function before being output, where j = 1, 2, ..., T.

[0035] Furthermore, the edge features and The expression is as follows:

[0036]

[0037]

[0038] in: Represents the distribution diagram of the l-th iteration. The edge characteristics between the i-th node and the j-th node. Represents the distribution diagram of the l-th iteration. The edge characteristics between the i-th node and the j-th node. Represents the distribution diagram of the l-th iteration. The features of the i-th node, Represents the distribution diagram of the l-th iteration. The features of the i-th node, This is an encoding network that transforms a distribution into a deterministic metric. It consists of two convolutional modules connected together. The input of each convolutional module is processed sequentially through a convolutional layer, batch normalization, and ReLU activation function before outputting, i,j=1,…,T, where T is the number of samples in a training episode.

[0039] Furthermore, the instance features The expression is as follows:

[0040]

[0041]

[0042] Where: || represents the concatenation operator. Represents an instance diagram of the (l+1)th iteration. The features of the i-th node, Represents the instance graph of the l-th iteration. The features of the j-th node, Represents the distribution diagram of the l-th iteration. The edge characteristics between the i-th node and the j-th node. Represents the distribution diagram of the l-th iteration. The edge characteristics between the i-th node and the j-th node. Represents the distribution diagram of the l-th iteration. The edge features between the i-th node and the j-th node are given by D2P(), which represents the distribution-instance aggregation network. It consists of two convolutional modules connected together. The input of the convolutional module is processed by the convolutional layer, batch normalization and ReLU activation function in sequence before outputting. i,j=1,…,T, where T is the number of samples in a training episode.

[0043] Furthermore, the training process of the GMDP module in step (3) consists of multiple training episodes. Each training episode randomly samples multiple sets of image feature vectors from K categories as samples from the training set. Each category consists of N support samples and M query samples. The difference is that the label information of the support samples is used in the prediction process, but the label information of the query samples is not used. K, M, and N are all self-defined natural numbers. For any training episode, all samples of the training episode are input into the GMDP module. The GMDP module will predict the category of the target object in the image corresponding to the query sample and calculate the loss function between the prediction result and the real label. Then, based on the loss function, the parameters of the GMDP module are iteratively updated by backpropagation through gradient descent until the loss function converges and the training is completed.

[0044] The MDPN network proposed in this invention optimizes the noise interference problem and multi-level feature extraction problem in few-sample classification tasks. By using the target region localization method based on instance segmentation, it greatly reduces the impact of background and false targets on few-sample classification, and obtains the multi-level distribution features of the image by concatenating and stacking GNNs. Attached Figure Description

[0045] Figure 1 Legend for the CUB-200-2011 dataset.

[0046] Figure 2 This is a schematic diagram of the overall framework of the image classification model MDPN of the present invention.

[0047] Figure 3 This is a schematic diagram of the execution flow of the ISOL module.

[0048] Figure 4 This is a schematic diagram of the execution flow of the GMDP module. Detailed Implementation

[0049] To describe the present invention in more detail, the technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] This invention provides a small-sample image classification method based on multi-level distribution propagation, comprising the following steps:

[0051] (1) Bird images are used as input, and the labeled bird category tags are used as supervision signals, and both are sent to the network.

[0052] In this implementation, the images are primarily from the CUB-200-2011 dataset, which contains 11,788 images of 200 bird species. The dataset is divided into 100 base classes, 50 validation classes, and 50 new classes, as detailed below. Figure 1 As shown.

[0053] (2) The overall framework of the image classification model MDPN of this invention is as follows: Figure 2 As shown, the outline of the bird in the image is first segmented using the ISOL module, and non-object areas are covered with masks. Then, the output image of the ISOL module is cropped using the minimum bounding rectangle.

[0054] In this implementation, the IOSL module primarily uses prior knowledge from the dataset to accurately segment the outer contour of the object region, then covers the non-object regions with a mask, and finally extracts the output image of the IOSL module along the minimum bounding rectangle of the object region. The specific process is as follows: Figure 3 As shown.

[0055] (3) The segmented target region image is sent to the Backbone module to extract features.

[0056] In this implementation, the Backbone module uses ResNet12 as the feature extractor. After the last feature is processed by global average pooling, it passes through a fully connected layer with batch normalization to obtain a 128-dimensional instance feature as the final output of the backbone network.

[0057] (4) Using the bird tags in step (1) and the feature maps extracted in step (3) as input, the GMDP module is fed into the module for training.

[0058] The GMDP module is used for feature post-processing and consists of three cascaded graph networks. Its output is a three-level distribution with increasing levels of abstraction. Then, we use these three distributed features to update the original features sent to the GMDP module, and the updated features are then fed back into the GMDP module for the next iteration. The specific process is as follows: Figure 4 As shown. The purpose of iteration is to calculate the distribution multiple times so that the final output features contain information about the entire image, thereby achieving a good prediction result.

[0059] Specifically, the multi-level distribution propagation based on graph neural networks consists of l iterations, each iteration consisting of an instance graph. A primary distribution map A secondary distribution map A three-level distribution map The structure consists of a subscript l indicating the l-th iteration. First, all samples (support set samples and query set samples) have instance features V extracted by the backbone network. l point V l point As an example diagram The nodes in the graph are used to compute edge features. Secondly, we will and V is obtained by fusion l dist_1 V l dist_1 Used to construct a first-order distribution map The nodes in V, and V l dist_1 according to The order, and then the calculation edge features V represents l dist_1 The distribution of this is the first-order distribution; then, It was used directly to construct a secondary distribution map. node V in l dist_2 V l dist_2 according to Initialize in the order of... and then calculate... edge features Representative node V l dist_2 The distribution is a second-order distribution. Similarly, It was used directly to construct a three-level distribution map. node V in l dist_3 V l dist_3 according to Arrange them in order, and then calculate. edge features V represents l dist_3 The distribution is determined by the three-level distribution; finally, the characteristics of the obtained three-level distribution are calculated. Will be with instance feature V l point The fusion process yields new nodes. It will be used to construct the instance graph in the (l+1)th iteration. The nodes are then used to complete one iteration.

[0060] The iterative update based on multi-level distributed propagation in graph networks can be represented as: Vl point , V l dist_1 , V l dist_2 , V l dist_3 , The definition is as follows: Where i,j=1,…,T, N×K represents all samples in a training episode, and N×K represents the number of support samples. This indicates the number of samples queried.

[0061] Initial values ​​of nodes in the example graph The f is used emb The output features, for each sample x i have:

[0062]

[0063] in: m represents the dimension of the embedded feature.

[0064] Instance-Distributed Aggregation (P2D AGG): Edges in the instance graph Distribution representing instance features Initialization is as follows:

[0065]

[0066] in: It is a coding network that transforms a distribution into a deterministic metric. It consists of two Conv-BN-ReLU modules and a sigmoid layer.

[0067] When the number of iterations l > 0 The update is as follows:

[0068]

[0069] Finally, for Normalize.

[0070] The purpose of the instance-distributed aggregation module is to integrate the instance graph of this iteration. Distribution characteristics in The first-level distribution plot of the previous iteration Node features And generate the first-level distribution map for this iteration. Node features V l dist_1 . Each distribution in They are all N sup 3D eigenvectors The value of the j-th term represents the sample x i and x j Relationship, N sup This represents all support set samples in a task. Initialization is as follows:

[0071]

[0072] in: || is the concatenation operator, δ(·) is the Kronecker function, when y i =y j Output 1 when it is active, and 0 otherwise (y) i and y j (It's a tag).

[0073] When the iteration number l>0 The update is as follows:

[0074]

[0075] in: P2D stands for Instance-Distribution Aggregation Network. P2D first connects two features, and then transforms the connected features: This transformation consists of a fully connected layer and a ReLU.

[0076] Multi-level distribution propagation:

[0077] Edges of a Level 1 Distribution Graph The distribution representing the characteristics of the first-order distribution, when the iteration number l = 0, Initialization is as follows:

[0078]

[0079] in: Coding Network It consists of two Conv-BN-ReLU modules.

[0080] When the number of iterations l > 0 The update rules are as follows:

[0081]

[0082] Finally, for Perform regularization.

[0083] Due to the edge characteristics of the first-level distribution graph Instead of being aggregated with other features, it is directly used as a node feature in the secondary distribution map, therefore Similarly, the node characteristics of the three-level distribution map

[0084] Edges of a second-order distribution graph That is, the distribution with second-order distribution characteristics. The update rules are as follows:

[0085]

[0086] Edges of a Level 3 Distribution Map That is, the distribution with three-level distribution characteristics. The update rules are as follows:

[0087]

[0088] Distributed-Instance Aggregation (D2P AGG): The distribution characteristics of the three levels of encoding in the D2P module are related to... The instance features are aggregated and then flowed back to the instance graph after each iteration, serving as node features for the instance graph in the next iteration. The update is as follows:

[0089]

[0090]

[0091] in: || is the concatenation operator. It is a distributed-instance aggregation network, D2P, which consists of two Conv-BN-ReLU modules; after this step, It can inherit the distribution characteristics of multiple levels and prepare for the next iteration.

[0092] There are special training strategies for few-shot classification tasks. Specifically, in a standard few-shot image classification task, there are two datasets: the training set D. base ={(x i ,y i ),y i ∈C base} and test set D novel ={(x i ,y i ),y i ∈C novel}, where C base and C novel These represent the training category and the test category, respectively, and they do not overlap. Training and testing for few-shot image classification consists of many small task episodes. Each episode comprises K randomly sampled classes, with each class consisting of N labeled images and U unlabeled images, i.e., a K-way N-shot task setting. The labeled images form the support set. Unlabeled images form the query set The two do not overlap

[0093] Taking a 5-way 1-shot (i.e., K=5, N=1) task setting as an example:

[0094] 4.1 Divide the training set D by category base and test set D novel ;

[0095] 4.2 Training Set D base Five categories are sampled, and one sample from each category is used as a labeled image to form a support set S. U samples from these five categories are then sampled as unlabeled images to form a query set Q. U can be set according to your needs, such as 1, 15, etc. In the experiment, we take U=1.

[0096] The model learns features of images in the support set S and the query set Q, predicts the class labels of images in the query set Q, calculates the loss between the predicted labels and the true labels, and backpropagates the loss.

[0097] 4.3 Repeat step 4.2 until the set number of episodes is reached;

[0098] 4.4 Test Set D novel Five categories are sampled, and one sample from each category is used as a labeled image to form a support set S′. U′ samples from these five categories are then used as unlabeled images to form a query set Q′. The accuracy (Acc) of the model in predicting the query sample categories is tested. U′ can be set according to your needs, such as 1, 15, etc. In the experiment, we set U′ = 1.

[0099] 4.5 Repeat step 4.4 until the set number of episodes is reached;

[0100] 4.6 Repeat steps 4.2 to 4.5 until the set number of epochs is reached.

[0101] Our goal is to train a function that can be used based on a small support set of samples S∈D. novel The query set sample Q∈D novel A classifier that accurately maps to the corresponding label.

[0102] (5) By feeding the bird images into the trained frame, the corresponding bird category can be predicted.

[0103] The above description of the embodiments is provided to enable those skilled in the art to understand and apply the present invention. Those skilled in the art can readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made to the present invention by those skilled in the art based on the disclosure thereof should be within the scope of protection of the present invention.

Claims

1. A multi-level distributed propagation small sample image classification method, comprising the following steps: (1) obtaining an image dataset, wherein each image is labeled with a class label of its target object, and dividing all images in the dataset into a training set and a test set; (2) using an IOSL module to perform contour segmentation on the image to obtain a target region picture, and then performing feature extraction on the target region picture to obtain a feature vector of the image; the IOSL module first accurately segments the outer contour of the target object region in the image based on prior knowledge in the dataset, then covers the non-target object region with a mask, and finally crops the target region picture along the minimum bounding rectangle of the target object region; (3) constructing a GMDP module and training it using the feature vectors of the images in the training set; The GMDP module based on multi-level distribution propagation of graph neural network is composed of multiple iterations, and each iteration is composed of an instance graph A first-level distribution graph A second-level distribution graph A third-level distribution graph are sequentially connected, wherein the subscript l is a natural number representing the number of iterations; (4) inputting the feature vectors of the images in the test set into the trained GMDP module to directly predict and output the class of the target object in the corresponding image. Example feature V l point Constructing nodes in example graph and calculating edge feature of Fusing and to obtain example feature V l dist_1 ; using V l dist_1 Constructing nodes in distribution graph and arranging in order of , and further calculating edge feature of , i.e. first-level distribution, and directly taking as example feature V l dist_2 ; using V l dist_2 Constructing nodes in distribution graph and arranging in order of , and further calculating edge feature of , i.e. second-level distribution, and directly taking as example feature V l dist_3 ; using V l dist_3 Constructing nodes in distribution graph and arranging in order of , and further calculating edge feature of , i.e. third-level distribution, and fusing and V l point to obtain example feature for constructing nodes in example graph of next iteration, and thus completing one iteration process;​​​​ In step (2), the Backbone module is used to extract features from the target region picture, which uses ResNet12 as a feature extractor, processes the last feature with a global average pool, and then passes it through a fully connected layer with batch normalization to obtain a feature vector with a dimension of 128 as output.

2. The small sample size image classification method of claim 1, wherein: In step (3), the training process of the GMDP module consists of multiple training episodes. Each training episode randomly samples multiple groups of image feature vectors from K classes in the training set as samples, each class consisting of N support samples and M query samples. The difference lies in that the label information of the support samples is needed in the prediction process, while the label information of the query samples is not needed. K, M, and N are all natural numbers set by the user. For any training episode, all samples in the training episode are input into the GMDP module, which predicts the class of the target object in the image corresponding to the query sample, calculates the loss function between the prediction result and the true label, and then iteratively updates the parameters of the GMDP module according to the loss function through backpropagation and gradient descent method until the loss function converges, and the training is completed.

3. The small sample image classification method of claim 1, wherein: The example features When l = 0, The expression for l = 1 is as follows: where: represents the l-th iteration instance graph represents the feature of the i-th node in the graph, x i represents the i-th sample in a training episode, f emb () represents the backbone network, which adopts ResNet12, i = 1, 2, …, T, and T is the number of all samples in a training episode.

4. The small sample image classification method of claim 3, wherein: The edge feature When l = 0, The expression of the above is as follows: When l > 0, The expression is as follows: wherein: represents the lth iteration instance graph the edge feature between the ith node and the jth node in the graph, represents the lth iteration instance graph the feature of the jth node in the graph, is an encoding network for converting the distribution into a deterministic metric, which is composed of two convolution modules and a sigmoid layer connected in turn, and the input of the convolution module is processed by a convolution layer, a batch normalization and a ReLU activation function in turn and output, j = 1, 2, …, T.

5. The small sample image classification method of claim 1, wherein: The example features When l = 0, The expression for l = 1 is as follows: When l > 0, The expression is as follows: wherein: represents the lth iteration distribution graph represents the feature of the ith node in the graph represents the lth iteration instance graph represents the edge feature between the ith node and the kth node in the graph, || represents a splicing operator, δ() represents a Kronecker function, x i is the ith sample in a training episode, y i and y k are the class labels of the ith sample and the kth sample in a training episode, respectively, N sup is the number of support samples in a training episode, P2D() represents an instance-distribution aggregation network, which first connects the two sets of features, and then converts the connected features through a fully connected layer and a ReLU activation function to output, i=1, 2, …, T, T is the number of all samples in a training episode.

6. The small sample image classification method of claim 5, wherein: The edge feature When l = 0, The expression of the above is as follows: When l > 0, The expression is as follows: The edge feature And The expression is as follows: wherein: represents the lth iteration distribution graph represents the edge feature between the ith node and the jth node in the graph, represents the lth iteration distribution graph represents the edge feature between the ith node and the jth node in the graph, represents the lth iteration distribution graph represents the edge feature between the ith node and the jth node in the graph, represents the lth iteration distribution graph represents the feature of the jth node in the graph, represents the lth iteration distribution graph represents the feature of the ith node in the graph, represents the lth iteration distribution graph represents the feature of the ith node in the graph, is an encoding network for converting the distribution into a deterministic metric, which is composed of two convolution module connection groups, the input of the convolution module is processed by a convolution layer, batch normalization and a ReLU activation function in turn and then output, j = 1, 2, …, T.

7. The small sample image classification method of claim 1, wherein: The example features The expression is as follows: where || denotes concatenation operator, denotes the (l+1)-th iteration instance graph denotes the feature of the i-th node in the denotes the l-th iteration instance graph denotes the feature of the j-th node in the denotes the l-th iteration distribution graph denotes the edge feature between the i-th node and the j-th node in the denotes the l-th iteration distribution graph denotes the edge feature between the i-th node and the j-th node in the denotes the l-th iteration distribution graph denotes the edge feature between the i-th node and the j-th node in the D2P() denotes the distribution-instance aggregation network, which is composed of two convolution modules, the input of the convolution module is processed by convolution layer, batch normalization and ReLU activation function in turn and then output, i, j = 1, …, T, T is the number of all samples in an episode of training.

8. The small sample image classification method of claim 1, wherein: ​

Images