A small sample image recognition method, device and equipment using a hypergraph neural network

Through the small sample image recognition method of hypergraph neural network, the convolutional neural network extracts multiple semantic features and constructs hypergraphs. The hypergraph neural network aggregates information, solving the problem that binary relationships only come from label differences in the existing technology, and improving the problem that the two-part relationships in the existing technology are The accuracy of small sample image recognition is captured and rich semantic and distributed features are captured.

CN114463596BActive Publication Date: 2024-11-05SUZHOU UNIV OF SCI & TECH +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111633547.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-28
Publication Date
2024-11-05
Estimated Expiration
2041-12-28

AI Technical Summary

Technical Problem

In the existing graph-based small sample learning method, binary relationships only come from label differences, ignoring the rich semantic features in the sample, resulting in limited performance.

Method used

Using the small sample image recognition method of hypergraph neural network, multi-semantic features are extracted through convolutional neural networks, supernodes and hyper-edges are constructed, supernodes and hyper-edges are aggregated, super-edge information is used to iteratively update supernode features, and semantic embedded features are captured. Distribute features and perform image recognition.

Benefits of technology

The accuracy of small sample image recognition is improved, rich semantic features and distribution features are captured, the limitations of small sample learning based on graphs are overcome, and the recognition performance is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114463596B_ABST
    Figure CN114463596B_ABST
Patent Text Reader

Abstract

The present invention relates to a small sample image recognition method of a hypergraph neural network, which uses different receptive fields of hollow convolution in a convolutional neural network to obtain multi-semantic features of different structures and construct a multi-semantic hypergraph; uses a hypergraph neural network to aggregate all hyperedge information of supernodes in the multi-semantic hypergraph to update the initial supernode feature representation to obtain the supernode feature representation of a training set; obtains multi-semantic distribution information based on the supernode feature representation of the training set, and interacts the multi-semantic distribution information with the supernode feature representation of the training set, continuously optimizes the multi-semantic hypergraph through mutual iterative updates, obtains the target supernode feature representation of the training set, and calculates the classification result of the query set in the training set based on the target supernode feature representation of the training set. The present invention captures rich semantic embedding features and multiple semantic distribution features through image message transmission in a hypergraph structure, and realizes the recognition of small sample image classification.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a small sample image recognition method, device, equipment and computer-readable storage medium of a hypergraph neural network. Background Art

[0002] In recent years, the problem of learning with few samples has been widely studied, and many advanced methods have emerged to improve the performance of learning with few samples. The main related methods are currently based on graph neural networks: The core of graph neural network-based methods is to explore the relationship between query samples and support samples / classes. Inspired by the success of graph neural networks (GNNS) in modeling point-to-point relationships and information propagation, researchers have proposed many graph-based methods to perform few-sample learning tasks and have achieved great success. Transductive Propagation Network (TPN) uses the similarity between support and query examples to propagate labels through the similarity of pairwise node features. It uses a closed-form solution to perform iterative label propagation. Edge Labeling Graph Neural Network (EGNN) uses the similarity / dissimilarity between nodes to dynamically update edge labels. Distribution Propagation Graph Network (DPGN) constructs a dual graph to model the distribution level relationship of samples and outperforms most existing methods in classification tasks.

[0003] Traditional graph structure small sample learning, due to the small number of samples in small sample learning tasks, traditional small sample learning methods are prone to overfitting. Recent work has explored building graph structures between sample instances and using graph neural networks to enhance the recognition of small sample learning tasks. In general, a fully connected graph structure G = (V, E) is a convolutional neural network CNN that extracts sample features, where each node represents a sample and the edge represents the binary relationship between the two connected samples. and Represent the node set and edge set of the graph respectively. ij Represents node v i and v j If the relationship between ij =1 indicates node v i and v j belong to the same category. It can be seen that the binary relationship in graph-based small-sample learning only comes from label differences, while ignoring the rich semantic features contained in the samples. This may limit its performance. Therefore, it is necessary to design a small-sample image recognition method based on a hypergraph neural network. Summary of the invention

[0004] To this end, the technical problem to be solved by the present invention is to overcome the defect in the prior art that the binary relationship in small sample learning based on graph only comes from label differences, while ignoring the rich semantic features contained in the samples.

[0005] In order to solve the above technical problems, the present invention provides a small sample image recognition method of a hypergraph neural network, comprising:

[0006] S101: constructing a small sample image recognition neural network; wherein the small sample image recognition neural network includes a convolutional neural network and a hypergraph neural network;

[0007] S102: Inputting a training set into the small sample image recognition neural network for training; wherein the training set includes a support set and a query set;

[0008] S103: using different receptive fields of the atrous convolution in the convolutional neural network to obtain multi-semantic features of the training set;

[0009] S104: constructing super nodes according to the multi-semantic features of the training set, calculating the semantic similarity between every two super nodes, and constructing hyper edges according to the semantic similarity between every two super nodes to obtain a multi-semantic hypergraph;

[0010] S105: using the hypergraph neural network to aggregate all hyperedge information of supernodes in the multi-semantic hypergraph to update the initial supernode feature representation of the training set, to obtain the supernode feature representation of the training set;

[0011] S106: obtaining polysemantic distribution information according to the supernode feature representation of the training set, performing information interaction between the polysemantic distribution information and the supernode feature representation of the training set, updating the hyperedges of the polysemantic hypergraph, obtaining a new polysemantic hypergraph, and returning to execute step S105, so as to iteratively update the polysemantic hypergraph until the number of updates reaches a predetermined threshold, thereby obtaining the target supernode feature representation of the training set;

[0012] S107: Calculate the probability that the supernode feature representation of the query set belongs to the category of the support set according to the target supernode feature representation of the training set, and obtain the classification result of the query set.

[0013] In one embodiment of the present invention, obtaining the multi-semantic features of the training set by utilizing different receptive fields of the dilated convolution in the convolutional neural network includes:

[0014] Inputting the training set into the convolutional neural network CNN;

[0015] The multi-semantic features U of the training set are obtained by using the different receptive fields of the dilated convolution in the convolutional neural network. k =DilatedConv(CNN(X),γ k );

[0016] Where X is the training set, γk is the k-fold expansion rate of the dilated convolution, k∈{1,…,K}, and K is the number of the multi-semantic features.

[0017] In one embodiment of the present invention, constructing super nodes according to the multi-semantic features of the training set, calculating the semantic similarity between every two super nodes, constructing hyper edges according to the semantic similarity between every two super nodes, and obtaining a multi-semantic hypergraph includes:

[0018] constructing a supernode according to the multi-semantic features of the training set;

[0019] Through the relationship matrix A of the multi-semantic features k =f(U k ; W k ), calculate the similarity of multi-semantic features between every two super nodes

[0020] Among them, W k is the trainable parameter of the multi-semantic feature, is the multi-semantic feature of supernode i, is the multi-semantic feature of supernode j, σ is the activation function;

[0021] A hyperedge is constructed according to the semantic similarity between each two supernodes to obtain a multi-semantic hypergraph.

[0022] In one embodiment of the present invention, the step of aggregating all hyperedge information of supernodes in the multi-semantic hypergraph using the hypergraph neural network to update the initial supernode feature representation of the training set to obtain the supernode feature representation of the training set includes:

[0023] Inputting the multi-semantic hypergraph into the hypergraph neural network;

[0024] Using the hypergraph neural network U=[U,D -1 BDB -1 U]W, aggregating all hyperedge information of supernodes in the multi-semantic hypergraph to update the initial supernode feature representation of the training set to obtain the supernode feature representation of the training set;

[0025] Among them, U=[U k ,k≤K], U is the feature representation set of supernodes in the multi-semantic hypergraph, W is the learnable parameter of the hypergraph neural network, B is the influence of each supernode in the multi-semantic hypergraph on each hyperedge, and D is the influence of each hyperedge to each supernode in the multi-semantic hypergraph.

[0026] In one embodiment of the present invention, obtaining multi-semantic distribution information according to the supernode feature representation of the training set, performing information interaction between the multi-semantic distribution information and the supernode feature representation of the training set, updating the hyperedges of the multi-semantic hypergraph, and obtaining a new multi-semantic hypergraph includes:

[0027] Obtaining multi-semantic distribution information according to the super-node feature representation of the training set;

[0028] According to A V =f(V;W V ) and A u =f(U U ; W U ), interacting the multi-semantic distribution information with the super-node feature representation of the training set, updating the hyper-edges of the multi-semantic hypergraph, and obtaining a new multi-semantic hypergraph;

[0029] Among them, A V is the relationship matrix of the multi-semantic distribution information, A u is the relationship matrix of the super node, V is the multi-semantic distribution information, U U is the supernode feature representation of the training set, W V is the trainable parameter of the multi-semantic distribution information, W U is a trainable parameter representing the supernode feature of the training set.

[0030] In one embodiment of the present invention, the step of calculating the probability that the supernode feature representation of the query set belongs to the category of the support set based on the target supernode feature representation of the training set, and obtaining the classification result of the query set includes:

[0031] according to Calculating the probability that the supernode feature representation of the query set belongs to the support set category, and obtaining the classification result of the query set;

[0032] Among them, y j is the jth sample in the support set, A l (U,ij) is the relationship matrix represented by the target supernode feature, N represents the number of categories in the support set, and K represents the number of samples in each category of the support set.

[0033] In one embodiment of the present invention, the step of calculating the probability that the supernode feature representation of the query set belongs to the category of the support set based on the target supernode feature representation of the training set, and obtaining the classification result of the query set comprises:

[0034] Inputting the evaluation set into the small sample image recognition neural network to obtain the classification result of the query set in the evaluation set;

[0035] Calculate the loss function of the small sample image recognition neural network based on the classification results of the query set in the evaluation set evaluating the small sample image recognition neural network;

[0036] Among them, ce is the cross entropy loss function, is the relation loss function, λ1 is the first hyperparameter, λ2 is the second hyperparameter, p is the classification result of the query set in the evaluation set, and L is the number of iterative updates of the supernode feature representation corresponding to the evaluation set.

[0037] The present invention provides a small sample image recognition learning device of a hypergraph neural network, comprising:

[0038] A first building module is used to build a small sample image recognition neural network; wherein the small sample image recognition neural network includes a convolutional neural network and a hypergraph neural network;

[0039] An input module, used to input a training set into the small sample image recognition neural network for training; wherein the training set includes a support set and a query set;

[0040] A convolution module, used to obtain multi-semantic features of the training set by utilizing different receptive fields of the dilated convolution in the convolutional neural network;

[0041] A second construction module is used to construct a supernode according to the multi-semantic features of the training set, calculate the semantic similarity between every two supernodes, and construct a hyperedge according to the semantic similarity between every two supernodes to obtain a multi-semantic hypergraph;

[0042] A supernode feature updating module, used to update the initial supernode feature representation of the training set by aggregating all hyperedge information of supernodes in the multi-semantic hypergraph using the hypergraph neural network to obtain the supernode feature representation of the training set;

[0043] an iterative update module, configured to obtain polysemantic distribution information according to the supernode feature representation of the training set, perform information interaction between the polysemantic distribution information and the supernode feature representation of the training set, update the hyperedges of the polysemantic hypergraph, obtain a new polysemantic hypergraph, and return to execute the steps of the supernode feature update module so as to iteratively update the polysemantic hypergraph until the number of updates reaches a predetermined threshold, thereby obtaining the target supernode feature representation of the training set;

[0044] The calculation module is used to calculate the probability that the supernode feature representation of the query set belongs to the category of the support set according to the target supernode feature representation of the training set, so as to obtain the classification result of the query set.

[0045] The present invention provides a small sample image recognition device of a hypergraph neural network, comprising:

[0046] Memory for storing computer programs;

[0047] A processor is used to implement the steps of a small sample image recognition method of a hypergraph neural network as described above when executing the computer program.

[0048] The present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of a small sample image recognition method of a hypergraph neural network as described above are implemented.

[0049] The above technical solution of the present invention has the following advantages compared with the prior art:

[0050] The small sample image recognition method of the hypergraph neural network described in the present invention constructs a small sample image recognition neural network, inputs a training set into the small sample image recognition neural network for training, uses different receptive fields of hollow convolution in a convolutional neural network to obtain multi-semantic features of different structures, and constructs a multi-semantic hypergraph; uses the hypergraph neural network to aggregate all hyper-edge information of supernodes in the multi-semantic hypergraph to update the initial supernode feature representation, and obtains the supernode feature representation of the training set; obtains multi-semantic distribution information based on the supernode feature representation of the training set, interacts the multi-semantic distribution information with the supernode feature representation of the training set, continuously optimizes the multi-semantic hypergraph through mutual iterative updating, obtains the target supernode feature representation of the training set, and calculates the classification result of the query set in the training set based on the target supernode feature representation of the training set. The present invention constructs a multi-semantic hypergraph through a convolutional neural network, aggregates all hyper-edge information of supernodes in the multi-semantic hypergraph through a hypergraph neural network to update the initial super-node feature representation, introduces multi-semantic distribution information and the super-node feature representation of a training set to interact with each other through iterative updates to continuously optimize the multi-semantic hypergraph, and uses the constructed multi-semantic hypergraph to learn correct node representations through message transmission of the multi-semantic hypergraph, captures rich semantic embedding features and multiple semantic distribution features to guide the recognition of small sample image classification, and improves the accuracy of the small sample image recognition method. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0052] Figure 1 A flowchart of a first specific embodiment of the small sample image recognition method of the hypergraph neural network provided by the present invention;

[0053] Figure 2A flowchart of a second specific embodiment of the small sample image recognition method of the hypergraph neural network provided by the present invention;

[0054] Figure 3 A schematic diagram of a small sample image recognition method of a hypergraph neural network provided by the present invention;

[0055] Figure 4 Schematic diagram for constructing a multi-semantic hypergraph;

[0056] Figure 5 a is the structural diagram of the traditional graph;

[0057] Figure 5 b is the structure diagram of the hypergraph;

[0058] Figure 6 Schematic diagram of the update of the hypergraph neural network;

[0059] Figure 7 A structural block diagram of a small sample image recognition method of a hypergraph neural network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0061] Reference Figure 1 As shown, Figure 1 This is a flow chart of a first specific embodiment of the small sample image recognition method of the hypergraph neural network provided by the present invention; the specific operation steps are as follows:

[0062] Step S101: constructing a small sample image recognition neural network; wherein the small sample image recognition neural network includes a convolutional neural network and a hypergraph neural network;

[0063] Step S102: inputting a training set into the small sample image recognition neural network for training; wherein the training set includes a support set and a query set;

[0064] Step S103: using different receptive fields of the dilated convolution in the convolutional neural network to obtain multi-semantic features of the training set;

[0065] Step S104: constructing super nodes according to the multi-semantic features of the training set, calculating the semantic similarity between every two super nodes, and constructing hyper edges according to the semantic similarity between every two super nodes to obtain a multi-semantic hypergraph;

[0066] Step S105: using the hypergraph neural network to aggregate all hyperedge information of supernodes in the multi-semantic hypergraph, updating the initial supernode feature representation of the training set, to obtain the supernode feature representation of the training set;

[0067] Step S106: obtaining polysemantic distribution information according to the supernode feature representation of the training set, performing information interaction between the polysemantic distribution information and the supernode feature representation of the training set, updating the hyperedges of the polysemantic hypergraph, obtaining a new polysemantic hypergraph, returning to execute step S105, so as to iteratively update the polysemantic hypergraph until the number of updates reaches a predetermined threshold, and obtaining the target supernode feature representation of the training set;

[0068] Step S107: According to the target supernode feature representation of the training set, the probability that the supernode feature representation of the query set belongs to the category of the support set is calculated to obtain the classification result of the query set.

[0069] The method provided in this embodiment overcomes the limitations of graph-based small sample learning through a hypergraph-based structure, constructs a multi-semantic hypergraph through a convolutional neural network, and can construct high-order complex association relationships for different semantic features of samples. The hypergraph neural network aggregates all hyper-edge information of supernodes in the multi-semantic hypergraph to update the initial super-node feature representation to mine the complex potential semantic relationship between small examples in small sample learning. The multi-semantic hypergraph is continuously optimized by introducing multi-semantic distribution information and the final super-node feature representation of the training set for information interaction and mutual iterative updating. The correct node representation is learned with the constructed multi-semantic hypergraph through message passing of the multi-semantic hypergraph, and rich semantic embedding features and multiple semantic distribution features are captured to guide the recognition of small sample image classification.

[0070] Based on the above embodiments, this embodiment further illustrates the small sample image recognition method of the hypergraph neural network. Figure 2 and Figure 3 As shown, Figure 2 A flowchart of a second specific embodiment of the small sample image recognition method of the hypergraph neural network provided by the present invention; Figure 3 This is a schematic diagram of the small sample image recognition method of the hypergraph neural network provided by the present invention; the specific operation steps are as follows:

[0071] Step S201: constructing a small sample image recognition neural network; wherein the small sample image recognition neural network includes a convolutional neural network and a hypergraph neural network;

[0072] Step S202: inputting the training set into the small sample image recognition neural network for training; wherein the training set includes a support set and a query set, the support set is an unlabeled image, and the query set is a labeled image;

[0073] Step S203: using different receptive fields of the dilated convolution in the convolutional neural network to obtain multi-semantic features of the training set;

[0074] The different receptive fields of the atrous convolution in the convolutional neural network are used to obtain multi-semantic features of different structures. The formula is as follows:

[0075] U k =DilatedConv(CNN(X),γ k )

[0076] Among them, U k ∈R |V|×d is the feature representation of the kth semantics of the sample, d is the feature dimension, X is the training set, γ k is the k-fold expansion rate of the dilated convolution, k∈{1,…,K}. The multi-semantic features may be extracted by other methods such as dilated convolution or multi-head sample relationship, which is not limited in the embodiments of the present invention.

[0077] Step S204: constructing super nodes according to the multi-semantic features of the training set, calculating the semantic similarity between every two super nodes, and constructing hyper edges according to the semantic similarity between every two super nodes to obtain a multi-semantic hypergraph;

[0078] The multi-semantic supernode feature is represented as U∈R |V|×Kd , multi-semantic features are obtained by concatenating single semantic features. The specific formula is as follows:

[0079] U=[U k ,k≤K]

[0080] Where [·] is a cascade operation, and U is the feature representation set of the supernode in the multi-semantic hypergraph. Multi-semantic supernodes can make features with different semantics as diverse as possible, and dilated convolutions with different expansion rates can effectively capture multiple semantic features because different receptive fields pay enough attention to objects from small to large, from texture to the whole.

[0081] Construct specific hyperedges at the semantic level and calculate the semantic similarity between every two supernodes. The kth semantic relationship matrix is ​​A k ∈R |V|×|V| , the formula is as follows:

[0082] A k =f(U k ; W k )

[0083] The relationship formula between the two samples is Among them, W kis a trainable parameter and σ is an activation function. is the degree of similarity between sample i and sample j in semantic k. A hyperedge with a fixed number of nodes is set in the supernode, and the most similar m (m>1) nodes are selected to construct the hyperedge. For semantic k, the constructed hyperedge is In small sample learning, let |E k |=|V| to obtain more informative hyperedges. Then the formula for the entire multi-semantic hyperedge is as follows:

[0084] E=∪ k E k

[0085] Where |E|=K|V|. Construct the multi-semantic hypergraph as Figure 4 shown.

[0086] Step S205: using the hypergraph neural network to aggregate all hyperedge information of supernodes in the multi-semantic hypergraph, updating the initial supernode feature representation of the training set, to obtain the supernode feature representation of the training set;

[0087] Compared with ordinary graph neural networks, hypergraphs can connect more than two nodes with the same semantic information, such as Figure 5 After constructing the multi-semantic hypergraph, important information is obtained by updating the features of each supernode. In the constructed multi-semantic hypergraph, the relationship matrix H∈R |V|×|E| , the specific formula is as follows:

[0088]

[0089] H builds a bridge between multiple semantics, and supernodes with different semantics will be highlighted. This helps to understand the relationship between different semantics. Defining convolution operations on multi-semantic hypergraphs can effectively propagate the feature representations of multiple supernodes. The hypergraph convolutional neural network formula is as follows:

[0090] U=[U,D -1 BDB -1 U]W

[0091] Where W∈R 2Kd×Kd is a learnable parameter, B∈R |E|×|V| and D∈R |V|×|E| The influence of each supernode on each hyperedge and each hyperedge on the supernode. The specific formula is as follows:

[0092] B=H T , D=H⊙I

[0093] Among them, ⊙ is the product of the corresponding elements, I = [A K,k≤K], which is the distribution information of all semantics of the sample.

[0094] The hypergraph neural network obtains supernode information by aggregating supernode representations and hyperedges. -1 B assigns the importance weight of each connected supernode to each hyperedge, and then collects the semantic information of each node to obtain the semantic information represented by the hyperedge. In addition, other hyperedges associated with the hyperedge are aggregated to learn the feature representation of the hyperedge. DB -1 The final representation feature of the supernode is updated by aggregating all the hyperedge information of the supernode. The hypergraph neural network is updated as follows: Figure 6 shown.

[0095] Step S206: obtaining polysemantic distribution information according to the supernode feature representation of the training set, performing information interaction between the polysemantic distribution information and the supernode feature representation of the training set, updating the hyperedges of the polysemantic hypergraph, obtaining a new polysemantic hypergraph, returning to execute step S205, so as to iteratively update the polysemantic hypergraph until the number of updates reaches a predetermined threshold, and obtaining the target supernode feature representation of the training set;

[0096] Hypergraph neural networks connect nodes with the same semantics to obtain more information, which is conducive to effectively updating node representations. However, due to the small number of samples in small sample learning, the rationality of hypergraph construction greatly affects the final learning effect. Although multiple nodes are associated through multiple semantics, the rationality of hypergraph construction is still not convincing enough because the hyperedges we obtain cannot guarantee absolute reliability. Therefore, we introduce multi-semantic distributed information representation V:

[0097]

[0098] Among them, W d and b d is a learnable parameter. The polysemantic distribution information is equivalent to the polysemantic information similarity distribution calculated between any two supernodes, which may provide information not included in the visual content. The relationship matrix formula between polysemantic distribution information and supernodes is as follows:

[0099] A V =f(V;W V ),A u =f(U U ; W U )

[0100] Among them, A V is the relationship matrix of the multi-semantic distribution information, A u is the relationship matrix of the super node, V is the multi-semantic distribution information, UU is the supernode feature representation of the training set, W V is the trainable parameter of the multi-semantic distribution information, W U is a trainable parameter representing the supernode feature of the training set.

[0101] The new hypergraph structure is updated through these two relationship matrices, and then the hypergraph is learned to update the supernodes and multi-semantic distribution information. In this way, the supernode information and multi-semantic distribution information are propagated to each other through continuous iteration, and finally a continuously optimized supernode feature representation is obtained.

[0102] In the small-sample learning of multi-semantic hypergraphs, semantic distribution messages continuously optimize the hypergraph structure and super-node feature representation through mutual learning between multi-semantic features and distribution features.

[0103] Step S207: Calculate the probability that the supernode feature representation of the query set belongs to the category of the support set based on the target supernode feature representation of the training set, and obtain the classification result of the query set;

[0104] according to Calculating the probability that the supernode feature representation of the query set belongs to the support set category, and obtaining the classification result of the query set;

[0105] Among them, y j is the jth sample in the support set, A l (U,ij) is the relationship matrix represented by the target supernode feature, N represents the number of categories in the support set, and K represents the number of samples in each category of the support set.

[0106] Step S208: inputting the evaluation set into the small sample image recognition neural network to obtain the classification result of the query set in the evaluation set;

[0107] Step S209: calculating the loss function of the small sample image recognition neural network according to the classification results of the query set in the evaluation set, and evaluating the sample image recognition neural network;

[0108] The loss function of the small sample image recognition neural network is as follows:

[0109]

[0110] Wherein, λ1 is the first hyperparameter, λ2 is the second hyperparameter, L is the number of iterative updates of the supernode feature representation corresponding to the evaluation set, ζ ce is the cross entropy loss function, is the relationship loss function, and the formula is as follows:

[0111]

[0112] Among them, BCE is the binarization loss function, A l (U,ij) is the relationship similarity between the features of the l-th generation samples i and j, A l (V,ij) is the relationship similarity of the multi-semantic distribution information of the l-th generation sample i,j, α is the third hyperparameter, β is the third hyperparameter, α and β are used to balance the two loss functions, r ij is the relationship label for supervised learning. The formula is as follows:

[0113]

[0114] Step S210: using the evaluated small sample image recognition neural network to perform recognition and classification on the test set.

[0115] The method provided in this embodiment overcomes the limitations of graph-based small sample learning through a hypergraph-based structure, can construct high-order complex association relationships for different semantic features of samples, and mine the complex potential semantic relationships between small examples in small sample learning. It can improve the accuracy of small sample learning recognition, and efficiently and accurately detect and recognize targets in real time. With the help of a multi-semantic hypergraph neural network structure, it can learn more precise image feature representations, obtain multi-semantic information of image samples, improve problems such as image detail loss and background noise interference, and make the image feature representation more accurate, richer, and more natural.

[0116] Please refer to Figure 7 , Figure 7 A structural block diagram of a small sample image recognition method of a hypergraph neural network provided in an embodiment of the present invention; the specific device may include:

[0117] The first construction module 100 is used to construct a small sample image recognition neural network; wherein the small sample image recognition neural network includes a convolutional neural network and a hypergraph neural network;

[0118] An input module 200, used to input a training set into the small sample image recognition neural network for training; wherein the training set includes a support set and a query set;

[0119] A convolution module 300, used to obtain multi-semantic features of the training set by utilizing different receptive fields of the dilated convolution in the convolutional neural network;

[0120] The second construction module 400 is used to construct super nodes according to the multi-semantic features of the training set, calculate the semantic similarity between every two super nodes, and construct hyper edges according to the semantic similarity between every two super nodes to obtain a multi-semantic hypergraph;

[0121] A supernode feature updating module 500 is used to update the initial supernode feature representation of the training set by aggregating all hyperedge information of supernodes in the multi-semantic hypergraph using the hypergraph neural network to obtain the supernode feature representation of the training set;

[0122] Iterative update module 600, used for obtaining polysemantic distribution information according to the supernode feature representation of the training set, exchanging information between the polysemantic distribution information and the supernode feature representation of the training set, updating the hyperedges of the polysemantic hypergraph, obtaining a new polysemantic hypergraph, returning to execute step S105, so as to iteratively update the polysemantic hypergraph until the number of updates reaches a predetermined threshold, and obtaining the target supernode feature representation of the training set;

[0123] The calculation module 700 is used to calculate the probability that the supernode feature representation of the query set belongs to the category of the support set according to the target supernode feature representation of the training set, so as to obtain the classification result of the query set.

[0124] The small sample image recognition device of the hypergraph neural network of this embodiment is used to implement the small sample image recognition method of the hypergraph neural network mentioned above. Therefore, the specific implementation method of the small sample image recognition device of the hypergraph neural network can be seen in the embodiment part of the small sample image recognition method of the hypergraph neural network in the previous text. For example, 100, 200, 300, 400, 500, 600 and 700 are respectively used to implement steps S101, S102, S103, S104, S105, S106 and S107 in the small sample image recognition method of the hypergraph neural network mentioned above. Therefore, its specific implementation method can refer to the description of the corresponding each part of the embodiment, which will not be repeated here.

[0125] A specific embodiment of the present invention also provides a small sample image recognition device of a hypergraph neural network, including: a memory for storing a computer program; and a processor for implementing the steps of the above-mentioned small sample image recognition method of a hypergraph neural network when executing the computer program.

[0126] A specific embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned small sample image recognition method of a hypergraph neural network are implemented.

[0127] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0128] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0129] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0130] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0131] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A small sample image recognition method based on a hypergraph neural network, characterized in that: include: S101: constructing a small sample image recognition neural network; wherein the small sample image recognition neural network includes a convolutional neural network and a hypergraph neural network; S102: Inputting a training set into the small sample image recognition neural network for training; wherein the training set includes a support set and a query set; S103: using different receptive fields of the atrous convolution in the convolutional neural network to obtain multi-semantic features of the training set; S104: constructing super nodes according to the multi-semantic features of the training set, calculating the semantic similarity between every two super nodes, and constructing hyper edges according to the semantic similarity between every two super nodes to obtain a multi-semantic hypergraph, including: constructing a supernode according to the multi-semantic features of the training set; Through the relationship matrix A of the multi-semantic features k =f(U k ; W k ), calculate the similarity of multi-semantic features between every two super nodes Among them, W k is the trainable parameter of the multi-semantic feature, is the multi-semantic feature of supernode i, is the multi-semantic feature of supernode j, σ is the activation function; Constructing a hyperedge according to the semantic similarity between each two supernodes to obtain a multi-semantic hypergraph; S105: using the hypergraph neural network to aggregate all hyperedge information of supernodes in the multi-semantic hypergraph to update the initial supernode feature representation of the training set, to obtain the supernode feature representation of the training set; S106: obtaining polysemantic distribution information according to the supernode feature representation of the training set, performing information interaction between the polysemantic distribution information and the supernode feature representation of the training set, updating the hyperedges of the polysemantic hypergraph, obtaining a new polysemantic hypergraph, and returning to execute step S105, so as to iteratively update the polysemantic hypergraph until the number of updates reaches a predetermined threshold, thereby obtaining the target supernode feature representation of the training set; S107: According to the target supernode feature representation of the training set, according to Calculate the probability that the supernode feature representation of the query set belongs to the support set category, and obtain the classification result of the query set; where y j is the jth sample in the support set, A l (U,ij) is the relationship matrix represented by the target supernode feature, N represents the number of categories in the support set, and K represents the number of samples in each category of the support set.

2. The method according to claim 1, characterized in that: The method of obtaining the multi-semantic features of the training set by utilizing different receptive fields of the dilated convolution in the convolutional neural network includes: Inputting the training set into the convolutional neural network CNN; The multi-semantic features U of the training set are obtained by using the different receptive fields of the dilated convolution in the convolutional neural network. k =DilatedConv(CNN(X),γ k ); Where X is the training set, γ k is the k-fold expansion rate of the dilated convolution, k∈{1,…,K}, and K is the number of the multi-semantic features.

3. The method according to claim 1, characterized in that The step of aggregating all hyperedge information of supernodes in the multi-semantic hypergraph using the hypergraph neural network to update the initial supernode feature representation of the training set to obtain the supernode feature representation of the training set includes: Inputting the multi-semantic hypergraph into the hypergraph neural network; Using the hypergraph neural network U=[U,D -1 BDB -1 U]W, aggregating all hyperedge information of supernodes in the multi-semantic hypergraph to update the initial supernode feature representation of the training set to obtain the supernode feature representation of the training set; Among them, U=[U k ,k≤K], U is the feature representation set of supernodes in the multi-semantic hypergraph, W is the learnable parameter of the hypergraph neural network, B is the influence of each supernode in the multi-semantic hypergraph on each hyperedge, and D is the influence of each hyperedge to each supernode in the multi-semantic hypergraph.

4. The method according to claim 1, characterized in that: The step of obtaining multi-semantic distribution information according to the supernode feature representation of the training set, performing information interaction between the multi-semantic distribution information and the supernode feature representation of the training set, updating the hyperedges of the multi-semantic hypergraph, and obtaining a new multi-semantic hypergraph includes: Obtaining multi-semantic distribution information according to the super-node feature representation of the training set; According to A V =f(V;W V ) and A u =f(U U ; W U ), interacting the multi-semantic distribution information with the super-node feature representation of the training set, updating the hyper-edges of the multi-semantic hypergraph, and obtaining a new multi-semantic hypergraph; Among them, A V is the relationship matrix of the multi-semantic distribution information, A u is the relationship matrix of the super node, V is the multi-semantic distribution information, U U is the supernode feature representation of the training set, W V is the trainable parameter of the multi-semantic distribution information, W U is a trainable parameter representing the supernode feature of the training set.

5. The method according to claim 1, characterized in that The step of calculating the probability that the supernode feature representation of the query set belongs to the category of the support set according to the target supernode feature representation of the training set and obtaining the classification result of the query set comprises: Inputting the evaluation set into the small sample image recognition neural network to obtain the classification result of the query set in the evaluation set; Calculate the loss function of the small sample image recognition neural network based on the classification results of the query set in the evaluation set evaluating the small sample image recognition neural network; Among them, ce is the cross entropy loss function, is the relation loss function, λ1 is the first hyperparameter, λ2 is the second hyperparameter, p is the classification result of the query set in the evaluation set, and L is the number of iterative updates of the supernode feature representation corresponding to the evaluation set.

6. A small sample image recognition device based on a hypergraph neural network, characterized in that: include: A first building module is used to build a small sample image recognition neural network; wherein the small sample image recognition neural network includes a convolutional neural network and a hypergraph neural network; An input module, used to input a training set into the small sample image recognition neural network for training; wherein the training set includes a support set and a query set; A convolution module, used to obtain multi-semantic features of the training set by utilizing different receptive fields of the dilated convolution in the convolutional neural network; The second construction module is used to construct a supernode according to the multi-semantic features of the training set, calculate the semantic similarity between every two supernodes, and construct a hyperedge according to the semantic similarity between every two supernodes to obtain a multi-semantic hypergraph, including: constructing a supernode according to the multi-semantic features of the training set; Through the relationship matrix A of the multi-semantic features k =f(U k ; W k ), calculate the similarity of multi-semantic features between every two super nodes Among them, W k is the trainable parameter of the multi-semantic feature, is the multi-semantic feature of supernode i, is the multi-semantic feature of supernode j, σ is the activation function; Constructing a hyperedge according to the semantic similarity between each two supernodes to obtain a multi-semantic hypergraph; A supernode feature updating module, used to update the initial supernode feature representation of the training set by aggregating all hyperedge information of supernodes in the multi-semantic hypergraph using the hypergraph neural network to obtain the supernode feature representation of the training set; an iterative update module, configured to obtain polysemantic distribution information according to the supernode feature representation of the training set, perform information interaction between the polysemantic distribution information and the supernode feature representation of the training set, update the hyperedges of the polysemantic hypergraph, obtain a new polysemantic hypergraph, and return to execute the steps of the supernode feature update module so as to iteratively update the polysemantic hypergraph until the number of updates reaches a predetermined threshold, thereby obtaining the target supernode feature representation of the training set; A calculation module is used to represent the target super node feature of the training set according to Calculate the probability that the supernode feature representation of the query set belongs to the support set category, and obtain the classification result of the query set; where y j is the jth sample in the support set, A l (U,ij) is the relationship matrix represented by the target supernode feature, N represents the number of categories in the support set, and K represents the number of samples in each category of the support set.

7. A small sample image recognition device based on a hypergraph neural network, characterized in that: include: Memory for storing computer programs; A processor, used to implement the steps of a small sample image recognition method of a hypergraph neural network as described in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a small sample image recognition method of a hypergraph neural network as described in any one of claims 1 to 5.