Small sample medical image classification method based on relation and graph neural network
By constructing a relationship-based graph neural network, combining CNN to extract features and graph convolution operations, the problem of scarcity of data in small sample medical image classification is solved, and better generalization ability and classification effect are achieved.
Patent Information
- Application Number
- CN202510227235.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-27
- Publication Date
- 2025-06-27
AI Technical Summary
Traditional deep learning classifiers perform poorly in small sample medical image classification, especially when data is scarce, making it difficult to accurately analyze images and diagnose diseases.
A small sample medical image classification method based on relationship and graph neural network is adopted to extract features through CNN, build GNN, and use graph convolution operations and MLP to learn the weights between nodes to enhance the generalization ability of the model.
In the small sample learning scenario, disease-related patterns can be effectively identified, generalized ability to new categories, and enhanced model adaptability and classification effect.
Smart Images
Figure CN120219808A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to computer vision, and particularly to a few-shot medical image classification method based on relations and graph neural networks. Background Art
[0002] With the development of medical imaging technologies such as MRI and CT scans, doctors can obtain increasingly clear images of patients' internal organs. However, analyzing these images and accurately diagnosing diseases is a challenge, especially in the detection of rare diseases or early-stage cancers. Traditional computer-aided diagnosis systems rely on predefined features and rules, and these systems have limitations in dealing with complex and variable medical image data. In addition, the annotation cost of medical image data is high, and the resources of professional medical personnel are limited, which restricts the application of deep learning technologies in this field.
[0003] The goal of a deep learning model is to learn from the accumulation of a large number of labeled samples and use the learned knowledge and a small number of labeled samples to identify unknown categories. For the categories that appear in the training stage, the model can still show good performance. However, the training methods of traditional deep learning classifiers are not applicable to few-shot classification, and the variable and small number of labeled samples lead to a decline in the performance of the classifier and a decline in the model performance caused by bias towards the training data. Solving the above problems is more conducive to training a more effective few-shot classification model. How to accurately analyze these images and diagnose diseases, especially in the case of scarce data, remains a major challenge. Summary of the Invention
[0004] The purpose of the present invention is to propose a few-shot medical image classification method based on relations and graph neural networks.
[0005] The purpose of the present invention is to propose a few-shot classification method for graph neural networks based on relations.
[0006] The technical solution for achieving the purpose of the present invention is: a few-shot medical image classification method based on relations and graph neural networks, including the following steps:
[0007] Step 1, obtain medical image data and divide it into a support set and a query set;
[0008] Step 2, use a CNN as an embedding network to extract features from the support set and the query set, respectively obtain a support set feature map and a query feature map, and at the same time extract the one-hot vector of the label of the support set image, and splice them as the input of the GNN;
[0009] Step 3, construct a GNN, each node represents a vector spliced by the support set feature map, the query feature map and the one-hot vector, and the weight of the edge between two nodes represents the similarity between the medical image sample features of the two nodes.
[0010] Step 4: For any two nodes of the GNN, calculate the absolute value of the difference between the feature vectors, and learn the non-linear combination of the node differences through the MLP to determine the weight values between the nodes, and obtain the adjacency matrix;
[0011] Step 5: Perform graph convolution operations according to the adjacency matrix to update the node features. After the feature learning of multiple graph convolution layers, the output of the last graph convolution layer is used as the input of the prediction layer for classification prediction.
[0012] Further, in Step 1, obtain medical image data and divide it into a support set S and a query set R. The specific method is as follows:
[0013] Obtain medical image data and construct a set of medical image samples with partially labeled data that follows independent and identically distributed, denoted as (T i ,Y i ), where:
[0014]
[0015] Y i =(y1,…,y t )∈{1,K} t ;
[0016] T i represents the set of partially labeled medical image samples used in the i-th task, x j ,l j represents the j-th labeled medical image sample and its corresponding disease category, y j represents the j-th medical image sample to be classified and its corresponding predicted category;
[0017] s is the number of medical images with known disease categories, t is the number of medical images for which the disease category needs to be predicted, K is the number of disease types that may appear in the medical images, represents the dimension of the sample feature space, and N is the number of pixel values or other radiomics features of each sample after feature extraction;
[0018] Divide the obtained medical image data into a support set S and a query set R, denoted as:
[0019] Support set: S ={(x j ,l j )∣j = 1,2,…,s},
[0020] Query set:
[0021] Furthermore, in step 2, use a CNN as the embedding network to extract features from the support set S and the query set R, obtaining the support set feature map and the query feature map respectively. At the same time, extract the one-hot vector of the label of the support set image, and splice them as the input of the GNN. The specific method is as follows:
[0022] Let φ(x) represent the embedding network, which converts the medical image x into a feature vector Φ(x). For the support set S = {(x i , l i ) | i = 1, 2, …, s} and the query set: For any image in, respectively obtain the support set feature map φ(x i ), the query feature map
[0023] Extract the one-hot vector h(l i ) of the label l i ) of the support set image;
[0024] Connect the image sample i with the known label l and the medical image sample with unknown label and the new vector formed by connecting the one-hot vector of the label of the known image as the input of the GNN.
[0025] Furthermore, in step 4, for any two nodes of the GNN, calculate the absolute value of the difference between the feature vectors, and learn the non-linear combination of the node differences through an MLP to determine the weight value between the nodes and obtain the adjacency matrix. The specific method is as follows:
[0026] For any two nodes and of the GNN obtained in step 3, calculate the absolute value of the difference between the corresponding feature vectors Input this absolute value of the difference into a multi-layer perceptron to learn the non-linear combination of these differences through the MLP, so as to determine the weight value between the nodes and obtain the adjacency matrix The element value at position (i, j) is expressed as:
[0027]
[0028] where is a parameterized function used to calculate the similarity or relationship degree between two nodes, represents the parameter of the function, and represent the feature vectors of node i and node j in the k-th layer network respectively, Denotes the absolute value of the difference between the feature vectors of node i and node j, which is used to measure the similarity of the features of two nodes. Is a multi-layer perceptron that performs a non-linear transformation on the absolute value of the input feature difference to learn the complex relationships between nodes.
[0029] Furthermore, in step 5, graph convolution operations are performed based on the adjacency matrix to update the node features. After the feature learning of multiple graph convolution layers, the output of the last graph convolution layer is used as the input of the prediction layer for classification prediction. The specific method is as follows:
[0030] Using any two nodes of the GNN and Calculate to obtain the adjacency matrix After that, perform normalization processing on it. Use the softmax function to perform normalization operations along each row of the adjacency matrix. After normalization, incorporate it into the operator family A, and then use graph convolution operations to update the node features to
[0031]
[0032] where Gc(x (k) ) represents the graph convolution operation, using the node feature x of the k-th layer (k) as the input, ρ is the activation function, ∑ B∈A represents the sum of all operators B in the operator family , Bx (k) represents the operator B acting on the node feature x of the k-th layer (k) ; is the weight matrix of the k-th layer, which is used to map the features of the k-th layer to the l-th feature dimension of the k + 1-th layer, l = d1…d k+1 represents the feature dimension index of the k + 1-th layer;
[0033] After the previous graph convolution operations, the node feature matrix is x (L) , where L is the total number of graph convolution layers, the weight matrix of the fully connected layer is W fc , and the bias vector is b fc , calculate z = x (L) W fc + b fc , and obtain the probability value y = [y1, y2,…, y K corresponding to each category through the softmax function y = softmax(z), where K is the number of categories, and y k represents the probability that the input image belongs to the k-th category; the predicted value is the category index with the highest probability, that is The category corresponding to this index is the prediction result of the input image.
[0034] A few-shot medical image classification system based on relationships and graph neural networks, implementing the described few-shot medical image classification method based on relationships and graph neural networks, achieving few-shot medical image classification based on relationships and graph neural networks, divided into five modules, respectively performing steps 1 to 5.
[0035] A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the described few-shot medical image classification method based on relationships and graph neural networks, achieving few-shot medical image classification based on relationships and graph neural networks.
[0036] A computer-readable storage medium has a computer program stored thereon. When the computer program is executed by a processor, it implements the described few-shot medical image classification method based on relationships and graph neural networks, achieving few-shot medical image classification based on relationships and graph neural networks.
[0037] Compared with the prior art, the significant advantages of the present invention are as follows: By constructing a graph neural network, pixels or regions in medical images are regarded as nodes, and the relationships between them are regarded as edges. Through graph convolution operations, local and global features in the images are captured, thereby identifying patterns related to diseases. This method is particularly suitable for few-shot learning scenarios and can enhance the generalization ability of the model through the topological information of the graph structure in the case of scarce data. 2) By using a multi-layer perceptron (MLP) to dynamically learn the adjacency matrix, the weights between nodes can be automatically adjusted according to sample features, which helps the model to more flexibly adapt to different data distributions. 3) By learning the potential relationships between samples, it helps to improve the generalization ability of the model for new categories, enabling it to also show good classification effects when facing unknown categories. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 is a flowchart of the present invention.
[0039] Figure 2 is a flowchart of feature extraction and splicing.
[0040] Figure 3 is a structure diagram of a graph neural network (GNN).
[0041] Figure 4 is a flowchart of classification prediction. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0042] In order to make the objectives, technical solutions, and advantages of the present application clearer, the following further details the present application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0043] A few-shot medical image classification method based on relational and graph neural networks, which combines few-shot graph learning and metric learning for medical image classification. It aggregates the metric information of adjacent edges through pairwise constraint propagation of the graph structure and integrates more context information for edge learning. In addition, more reliable regional metric learning is achieved by learning the context-aware region representation of medical image data. The method includes the following steps:
[0044] Step 1: Divide the medical image data into a support set S and a query set R.
[0045] Based on the relational network DataLoader, obtain the input data, including a set of partially labeled medical image samples that follow independent and identical distributions, denoted as (T i , Y i ), where:
[0046]
[0047] Y i =(y1,…, y t )∈{1, K} t ;
[0048] T i represents the set of partially labeled medical image samples used in the i-th task, x j , l j represent the j-th labeled medical image sample and the corresponding disease category, y j represents the j-th medical image sample to be classified and the corresponding predicted category; s is the number of medical images with known disease categories, t is the number of medical images for which the disease category needs to be predicted, K is the number of disease types that may appear in medical images, represents the dimension of the sample feature space, and N is the number of pixel values or other radiomics features of each sample after feature extraction.
[0049] Divide the obtained medical image data into a support set: S ={(x j , l j )∣j = 1, 2,…, s}, and a query set: Among them, the support set contains a finite number of labeled medical image samples. These samples cover several different disease categories (usually with a small number of samples in each category), and the model classifies by learning the features and category relationships of these samples. The support set provides a basis for the model to learn. The query set contains unlabeled medical image samples, which are used to test the classification ability of the model after the model training is completed. The sample categories in the query set are the same as those in the support set, but are invisible during the training phase. The model needs to classify these samples based on the knowledge learned from the support set.
[0050] Step 2: Use CNN as the embedding network to extract features from the medical image x.
[0051] Let Φ(x) represent the embedding network, which converts the input medical image x into a feature vector Φ(x). Therefore, for the support set S = {(x i , l i ) | i = 1, 2, …, s} and the query set in Step 1, for any image, the support set feature map φ(x i ) and the query feature map
[0052] can be obtained respectively. In addition, the one-hot vector h(l i ) of the label l i of the support set image needs to be extracted. After the feature vector extracted by the embedding network is concatenated with the One-Hot label vector, it is used as the input of the graph neural network (GNN).
[0053] Step 3: Construct a graph neural network (GNN). Each node represents the features formed by concatenating the input support set and query set. The weight on each edge represents the similarity between the features of two node medical image samples. The present invention uses a densely connected graph, so there are edges connecting every two input features.
[0054] GNN is a graph model composed of many nodes and edges. The new vector formed by connecting the image samples i with known labels l , the medical image samples with unknown labels, and the one-hot vectors of the labels of the known images contains the visual features and category label information of the images. This fused vector will be used as a vertex of the graph neural network to construct a relationship network and perform classification prediction.
[0055] The support set feature map φ(x i ) obtained in Step 2, the query set feature map and the one-hot vector h(li ) are concatenated to obtain a vector as a node of the graph neural network:
[0056]
[0057] represents the initial feature representation of the i-th sample, which is a vector formed by combining image features and label information and is used as the input to the graph neural network. φ represents a convolutional neural network (CNN), a function for extracting features from images. x i represents the image data of the i-th labeled sample, represents the image data of the j-th unlabeled sample; φ(x i ) refers to the feature vector obtained by processing x through the convolutional neural network φ i ; refers to the feature vector obtained by processing through the convolutional neural network φ; h(l i ) refers to the One-Hot encoded vector of the label l of the i-th sample i . This vector is 1 at the position of the category l i and 0 at the remaining positions, indicating the category of the sample.
[0058] Step 4, for any two nodes of the graph neural network obtained in Step 3 and First, calculate the absolute value of the difference between their feature vectors Then, input this absolute difference value into a multi-layer perceptron (MLP) to learn the non-linear combination of these differences through the MLP, thereby determining the weight values between nodes and obtaining the adjacency matrix The element value at position (i, j)
[0059] Adjacency matrix
[0060]
[0061] represents the element of the adjacency matrix between nodes i and j in the k-th layer of the graph neural network. This value usually represents the weight of the connection between two nodes. is a symmetric function for calculating the similarity or relationship strength between two nodes. Here, represents the parameters of the function, and respectively represent the feature vectors of nodes i and j in the k-th layer of the graph neural network. These feature vectors are obtained through hierarchical transmission and update in the network. In the present invention, The function calculates the absolute value of the difference between the feature vectors of two nodes, and then uses a multi-layer perceptron (MLP) to learn the non-linear combination of these differences, thereby obtaining the weight value between the nodes. This method allows the model to learn complex relationships between nodes. These parameters can be learned through the training process. In the present invention, after superimposing a multi-layer perceptron (multi-layer neural network) on the absolute value of the difference between the feature vectors on two nodes, the absolute value difference between the two nodes is input, and the corresponding weight value is output:
[0062]
[0063] is a parameterized function used to calculate the similarity or relationship degree between two nodes. Here represents the parameters of the function. and respectively represent the feature vectors of node i and node j in the k-th layer network, represents the absolute value of the difference between the feature vectors of node i and node j, which is used to measure the similarity of the features of the two nodes. is a multi-layer perceptron, which is a neural network used to perform non-linear transformation on the absolute value of the input feature difference, so as to learn the complex relationship between nodes. The formula has the following functions:
[0064] (1) Metric learning, The function measures the similarity between nodes by learning the non-linear combination of the absolute value of the difference between the feature vectors of two nodes. This metric has symmetry and reflexivity
[0065] (2) Normalization, by using the softmax function for each row, the output of the function is normalized so that the sum of the weights of each row is 1. This helps to maintain the relative proportion of the weights between nodes when propagating information in the graph neural network.
[0066] (3) Constructing the adjacency matrix, adding the normalized to the operator family Here, the learned adjacency matrix is concatenated with the identity matrix. This ensures that each node is at least connected to itself, so that the information of the node itself will not be lost during information propagation.
[0067] (4) Information propagation, in the graph neural network, the adjacency matrix is used to define the connection relationship between nodes and propagate information through these connection relationships. The learned adjacency matrix can enable the network to adjust the information propagation method according to the actual relationship between nodes, which is crucial for improving the performance of the model on graph-structured data.
[0068] Step 5, calculate the adjacency matrix After that, perform normalization on it. Use the softmax function to perform normalization operation along each row of the adjacency matrix. After normalization, incorporate it into the operator family A, and then use graph convolution operation to update the node features to After feature learning through multiple graph convolution layers, the output of the last graph convolution layer is used as the input of the prediction layer for classification prediction.
[0069] A weighted graph is a graph model that assigns a numerical value to each edge in the graph, and this numerical value is called a weight. The weight can represent various metrics, such as distance, cost, or time, etc. In a weighted graph, edges connect vertices (or nodes), and the weight provides additional information about the characteristics of the edges. Such a graph can naturally represent many applications. For example, in an aviation map, the edges represent flight routes, and the weights can represent distances or costs. In the context of graph neural networks (GNNs), the concept of weighted graphs is particularly important because the weights of the edges can affect the way information propagates in the graph. Because the weights of the edges can affect the way information propagates in the graph. In this step, it describes how to use a graph convolutional neural network (GNN) to calculate the process of the next layer of the network. This process involves the update of node features and the propagation of information in the graph.
[0070] Given the input of the vertices of the weighted graph G where V is the number of vertices and d is the dimension of each vertex feature; consider a linear operator family of the graph which acts on the input feature matrix F. For example, the adjacency operator A can be defined as (AF) i := ∑ j～i w i,j F j where (i, j) ∈ E is the edge between vertices, w i,j is the weight of the edge. A GNN layer Gc(·), the input is and produces the output
[0071] represents the feature matrix of all nodes in the k-th layer of the network, x (k) represents the node feature matrix of the k-th layer, represents the set of real numbers, that is, the elements in the feature matrix are all real numbers; V represents the total number of nodes in the graph; d k represents the dimension of each node feature vector in the k-th layer. Then represents the feature matrix of all nodes in the (k + 1)-th layer of the network. For the l-th node in the (k + 1)-th layer, its feature is updated through the following formula
[0072]
[0073] Among them, Gc(x (k) ) represents a graph convolution operation, which takes the node feature x of the k-th layer (k) as input, ρ is an activation function, and the leakyReLU activation function is selected in the present invention. ∑B ∈A represents the summation of all operators B in the operator family , and Bx (k) represents the operator B acting on the node feature x of the k-th layer (k) ; is the weight matrix of the k-th layer, which is used to map the features of the k-th layer to the l-th feature dimension of the k+1-th layer. l = d1...d k+1 represents the feature dimension index of the k+1-th layer.
[0074] Suppose that after the previous graph convolution operation, the node feature matrix is x (L) (where L is the total number of graph convolution layers), the weight matrix of the fully connected layer is W fc , and the bias vector is b fc , then the output y of the prediction layer is calculated as follows: First, calculate z = x (L) W fc +b fc , and then obtain the probability value y = [y1, y2,..., y K corresponding to each category through the softmax function y = softmax(z), where K is the number of categories, and y k represents the probability that the input image belongs to the k-th category. Finally, the predicted value is the category index with the highest probability, that is The category corresponding to this index is the prediction result of the model for the input image.
[0075] Embodiment
[0076] To verify the effectiveness of the solution of the present invention, the following experiment is carried out.
[0077] Step 1, divide the data set into two parts: the support set and the query set.
[0078] Support set: S = {(x j , l j )∣j = 1, 2,..., s}, where x j represents the j-th labeled medical image sample, l j represents its corresponding disease category, and l j ∈{1, K}.
[0079] Query set: Here represents the j-th unlabeled medical image sample.
[0080] This experiment focuses on the scenario of t = 10, that is, each task T classifies 10 samples. Here refers to the distribution of a specific disease category in Y i is related to the category of medical image samples without labels .
[0081] Step 2, Feature extraction based on the CNN network
[0082] Use the embedding module Φ(x) to extract the features of 5 categories in the support set and the features of the query set category, concatenate them to obtain a feature vector, and connect the feature vector with the existing label (using a One - Hot vector) as a vertex in the graph. Then, through the prediction of the graph neural network model of the present invention, the label of the sample to be classified is obtained.
[0083] Step 3, Construction of the initial vertex based on the GNN relationship network
[0084] Connect the image samples with known labels of l i and the one - hot vector of the label of the medical image sample with unknown label and the known image to form a new vector that contains the visual features of the image and the label information of the category. This fused vector will be used as a vertex of the graph neural network to construct the relationship network and perform classification prediction.
[0085] Step 4, Obtain the adjacency matrix of the graph
[0086]
[0087] represents the element of the adjacency matrix between node i and node j in the k - th layer of the graph neural network. This value usually represents the weight of the connection between two nodes.
[0088] In the present invention, after taking the absolute value of the difference between the feature vectors on two nodes and then stacking a multi - layer perceptron (multi - layer neural network), input the absolute value difference between two nodes, and output the corresponding weight value:
[0089]
[0090] is a parameterized function used to calculate the similarity or relationship degree between two nodes. Here represents the parameter of the function. and respectively represent the feature vectors of node i and node j in the k - th layer network, Denotes the absolute value of the difference between the feature vectors of node i and node j, which is used to measure the similarity of the features of two nodes. Is a multi-layer perceptron, which is a neural network used to perform a non-linear transformation on the absolute value of the input feature difference, so as to learn the complex relationships between nodes.
[0091] After calculating the obtained adjacency matrix it is normalized. The softmax function is used for normalization along each row of the adjacency matrix. After normalization, it is incorporated into the operator family A, and then the node features are updated using graph convolution operations to be
[0092] Step 5: Use the graph convolutional neural network (GNN) to calculate the next layer of the network. After the feature learning of multiple graph convolutional layers, the output of the last graph convolutional layer is used as the input of the prediction layer. Assume that after the previous graph convolution operations, the node feature matrix is x (L) (where L is the total number of graph convolutional layers), the weight matrix of the fully connected layer is W fc and the bias vector is b fc , then the output y of the prediction layer is calculated as follows: First calculate z = x (L) W fc + b fc , and then through the softmax function y = softmax(z) to obtain the probability values corresponding to each category y = [y1, y2, …, y K , where K is the number of categories, and y k represents the probability that the input image belongs to the k-th category. Finally, the predicted value is the category index with the highest probability, that is The category corresponding to this index is the prediction result of the model for the input image.
[0093] The accuracy achieved by the present invention is compared with other small-sample classification models as shown in Table 1.
[0094] Table 1 Comparison table of classification results
[0095]
[0096] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combinations of these technical features do not conflict, they should all be considered as the scope recorded in this specification.
[0097] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.
Claims
1. A small sample medical image classification method based on relational and graph neural networks, characterized in that: The steps include: Step 1: Obtain medical imaging data and divide it into a support set and a query set; Step 2: Use CNN as the embedding network to extract features from the support set and query set, and obtain the support set feature map and query feature map respectively. At the same time, extract the one-hot vector of the label of the support set image and concatenate them as the input of GNN. Step 3: construct a GNN. Each node represents a vector formed by the concatenation of the support feature graph, the query feature graph, and the one-hot vector. The weight of the edge between two nodes represents the similarity between the features of the medical image samples of the two nodes. Step 4: For any two nodes of the GNN, the absolute value of the difference between the feature vectors is calculated, and the nonlinear combination of node differences is learned through MLP to determine the weight values between the nodes and obtain the adjacency matrix. Step 5: Perform graph convolution operation according to the adjacency matrix to update node features. After feature learning of multiple layers of graph convolution layers, the output of the last layer of graph convolution layer is used as the input of the prediction layer for classification prediction.
2. The small sample medical image classification method based on relational and graph neural network according to claim 1 is characterized in that: Step 1: Obtain medical imaging data and divide it into support set S and query set R. The specific method is: Obtain medical imaging data and construct a partially labeled medical imaging sample set that follows independent and identical distribution, expressed as (T i ,Y i ),in: Y i =(y1,…,y t )∈{1,K} t ; T i represents the set of partially labeled medical image samples used in the i-th task, x j , l j represents the jth labeled medical image sample and the corresponding disease category, y j represents the jth medical image sample to be classified and the corresponding predicted category; s is the number of medical images with known disease categories, t is the number of medical images that need to predict disease categories, and K is the type of disease that may appear in medical images. represents the dimension of the sample feature space, N is the number of pixel values or other radiomics features of each sample after feature extraction; The acquired medical imaging data is divided into a support set S and a query set R, expressed as: Support set: S = {(x j ,l j )|j=1,2,…,s}, QuerySet:
3. The small sample medical image classification method based on relational and graph neural network according to claim 1 is characterized in that: Step 2: Use CNN as the embedding network to extract features from the support set S and query set R, and obtain the support set feature map and query feature map respectively. At the same time, extract the one-hot vector of the label of the support set image and concatenate them as the input of GNN. The specific method is: Let Φ(x) represent the embedding network, convert the medical image x into a feature vector Φ(x), and for the support set divided in step 1: S = {(x i ,l i )|i=1,2,…,s} and the query set: For any image in the support set, we can obtain the support set feature map φ(x i ), query feature graph Extract the labels l of the support set images i The one-hot vector h(l i ); The l with known label i Image samples The new vector formed by connecting the one-hot vector of the unknown label medical image sample and the label of the known image As the input of GNN.
4. The small sample medical image classification method based on relational and graph neural network according to claim 1 is characterized in that: Step 4: For any two nodes of the GNN, calculate the absolute value of the difference between the feature vectors, and use MLP to learn the nonlinear combination of node differences to determine the weight values between the nodes and obtain the adjacency matrix. The specific method is: For any two nodes of the GNN obtained in step 3 and Compute the absolute value of the difference between corresponding eigenvectors The absolute value of this difference is input into a multi-layer perceptron In the example, the nonlinear combination of these differences is learned through MLP to determine the weight values between nodes and obtain the adjacency matrix The value of the element at position (i,j) It is expressed as: in is a parameterized function used to calculate the similarity or relationship between two nodes. Represents the function's parameters, and Respectively represent the feature vectors of node i and node j in the k-th layer network, Represents the absolute value of the difference between the feature vectors of node i and node j, which is used to measure the similarity of the features of two nodes. It is a multi-layer perceptron that is used to perform nonlinear transformation on the absolute value of the input feature difference, thereby learning the complex relationship between nodes.
5. The small sample medical image classification method based on relational and graph neural network according to claim 1 is characterized in that: Step 5: Perform graph convolution operation according to the adjacency matrix to update node features. After feature learning of multiple layers of graph convolution layers, the output of the last layer of graph convolution layers is used as the input of the prediction layer for classification prediction. The specific method is as follows: Using any two nodes of GNN and Calculate the adjacency matrix After that, it is normalized, and the softmax function is used to normalize each row of the adjacency matrix. After normalization, it is included in the operator family A, and then the graph convolution operation is used to update the node features: Where Gc(x (k) ) represents the graph convolution operation, with the k-th layer node feature x (k) As input, ρ is the activation function, ∑ B∈A Represents the operator family Sum all operators B in Bx (k) Indicates that operator B acts on the node feature x of the kth layer (k) ; is the weight matrix of the kth layer, which is used to map the features of the kth layer to the lth feature dimension of the k+1th layer, l = d1...d k+1 Represents the feature dimension index of the k+1th layer; After the previous graph convolution operation, the node feature matrix is x (L) , where L is the total number of graph convolutional layers and the weight matrix of the fully connected layer is W fc , the bias vector is b fc , calculate z = x (L) W fc +b fc , through the softmax function y = softmax (z) to get the probability value y = [y1, y2, ..., y K ], where K is the number of categories, y k represents the probability that the input image belongs to the kth category; the predicted value is the category index with the highest probability, that is, The category corresponding to this index is the predicted result of the input image.
6. A small sample medical image classification system based on relational and graph neural networks, characterized in that: Implement the small sample medical image classification method based on relational and graph neural networks as described in any one of claims 1-5 to realize small sample medical image classification based on relational and graph neural networks, which is divided into five modules and executes steps 1 to 5 respectively.
7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the small sample medical image classification method based on relational and graph neural networks described in any one of claims 1 to 5 is implemented to achieve small sample medical image classification based on relational and graph neural networks.
8. A computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the small sample medical image classification method based on relational and graph neural networks described in any one of claims 1 to 5 is implemented to achieve small sample medical image classification based on relational and graph neural networks.