A rare disease classification method based on prototype graph neural network and small sample learning
By using prototype graph neural network and small sample learning methods in the classification of rare diseases, and using "prototype points" and graph neural network propagation technology, the problem of low classification accuracy under a small number of samples is solved, and stronger robustness and adaptability are achieved.
Patent Information
- Application Number
- CN202210565650.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-23
- Publication Date
- 2025-04-04
- Estimated Expiration
- 2042-05-23
AI Technical Summary
The existing rare disease classification method based on deep learning is difficult to understand the intrinsic characteristics of the training sample when facing a small number of samples, the model is not robust and the relationship between the sample is lacking, resulting in low classification accuracy.
The prototype graph neural network and small sample learning method are used to calculate the "prototype points" of each class by extracting the features of the sample, and the low-dimensional feature vectors and "prototype points" of the sample are brought into the graph neural network for propagation, and the loss function is optimized to obtain the optimal prototype graph neural network.
The protograph neural network better utilizes the relationship between samples, improves the classification effect in the classification of rare diseases, and enhances the robustness and adaptability of the model.
Smart Images
Figure CN114898436B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of small sample classification, and in particular to a rare disease classification method based on a prototype graph neural network. Background Art
[0002] The ability to quickly learn and generalize from a small number of samples is a key feature of human intelligence, as humans can leverage prior knowledge gained from previous learning experiences. Although deep learning methods have achieved remarkable success in many tasks, large amounts of labeled data are difficult to obtain, especially in the field of rare diseases, where samples are not easy to collect.
[0003] In addition, training also requires a lot of time because each task is trained independently and the model parameters are learned from scratch without incorporating task-specific prior knowledge. How to extract prior knowledge with limited data and transfer it to unseen tasks has become a focus of recent research.
[0004] Existing rare disease classification methods based on deep learning usually require a large amount of data to optimize the parameters of the model. Once the amount of data is insufficient, it is difficult to understand the intrinsic characteristics of the training samples, the model is not very robust, and it lacks consideration of the relationship between samples. The adaptability to new samples is very weak, and the classification accuracy is not high. Therefore, when facing the problem of a small number of samples, the network must not only pay attention to the data samples themselves, but also pay attention to the relationship between training samples. Summary of the invention
[0005] The purpose of the present invention is to provide a method that can use a small amount of rare disease sample data for accurate classification. The method uses a prototype graph neural network and a small sample learning method. First, the sample features are extracted to calculate the "prototype point" of each class, and then the low-dimensional feature vector and "prototype point" of the sample are brought into the graph neural network for initialization, and then propagated through the node feature propagation network, the edge feature propagation network and the label propagation network in turn, and finally optimized with a loss function to obtain the optimal prototype graph neural network. The prototype graph neural network can better utilize the relationship between samples, so that it has a better classification effect in the classification of rare diseases.
[0006] The specific technical solution for achieving the purpose of the present invention is:
[0007] A rare disease classification method based on prototype graph neural network and small sample learning, characterized in that the method includes the following specific steps:
[0008] Step 1: Collection and division of rare patient face datasets
[0009] 1.1) Collect rare patient face images to form a dataset. The entire dataset is divided into a training set and a test set in a ratio of 8:2. The category labels of the training set and the test set do not overlap.
[0010] 1.2) Both the training set and the test set are divided h times using the n-way k-shot small sample learning method, each time as an iteration set. Each iteration set is divided into a support set and a query set. The support set has n classes, each class has k samples, k is greater than 0 and less than or equal to 15;
[0011] Step 2: Sample feature extraction
[0012] 2.1) Perform 2D face registration on the rare patient face samples in each iteration set, and use the Adaboost cascade face detection classifier based on Haar features to determine whether there is a face. If there is, the face area is cut out and scaled to 84×84 to obtain the preprocessed sample to avoid interference from factors other than the face in the sample. Otherwise, reselect the image.
[0013] 2.2) Take the support set samples and query set samples from an iteration set that have been preprocessed in the previous step and send them to the embedding subnetwork: they pass through a 3×3 convolution with 64 filters, a batch normalization layer, a ReLU nonlinear layer, and a 2×2 maximum pooling layer in sequence to obtain the feature map of each sample;
[0014] 2.3) All feature maps are input into two fully connected layers of 8×64 and 1×128 dimensions respectively to obtain the corresponding feature vectors, which are then batch normalized and ReLU nonlinearly activated to obtain the low-dimensional feature vector of the sample;
[0015] 2.4) Calculate the average value of the low-dimensional feature vectors of all samples in the support set of each class in an iteration set, that is, sum the low-dimensional feature vectors of all samples in the support set of each class, and divide it by the number of samples in the support set of each class to calculate a virtual point as the "prototype point" of this class in the feature space;
[0016] Step 3: Prototype graph neural network construction
[0017] 3.1) First, the graph neural network is initialized. The prototype graph neural network is constructed by the low-dimensional feature vectors of all samples calculated in step 2 and the "prototype points". The nodes are the low-dimensional feature vectors and "prototype points" of the samples. The edge features are between 0 and 1, indicating the possibility that two nodes belong to the same class. If the labels of two nodes are equal, the edge feature is set to 1. If they are not equal, the edge feature is set to 0. Otherwise, the edge feature is set to 0.5. 3.2) Node feature update: First, a node feature update network is set up, and the node features and edge features are input into the node feature update network to obtain updated node features;
[0018] 3.3) Edge feature update, after the node features are updated, the edge features are updated next. An edge feature update network is set up, and all updated node features are input into the edge feature update network to obtain updated edge features;
[0019] 3.4) Node label propagation: set up a node label propagation network, input the updated edge features and node labels into the node label propagation network, and obtain the updated node labels;
[0020] 3.5) After L times of node feature updates, edge feature updates, and node label propagation, the updated prototype graph neural network is obtained;
[0021] Step 4: Label prediction
[0022] 4.1) After obtaining the updated prototype graph neural network, the predicted label of the query image is selected according to the maximum independent variable point set, and the rare disease class with the largest probability value is selected as the classification prediction result of the query image;
[0023] 4.2) Design two loss functions to optimize network parameters. One is to calculate the Euclidean distance between each query point and the corresponding “prototype point” in the feature space to obtain the geometric regularization loss. The other is to calculate the binary cross entropy loss between the predicted label and the true label of each query point in the query set. The final loss of the entire network is constructed from these two loss functions, and back propagation is performed to update the prototype graph neural network model parameters.
[0024] 4.3) Calculate the classification accuracy of the prototype graph neural network model through the test set, and select the model parameters with the highest accuracy as the optimal model parameters;
[0025] 4.4) During classification, the optimal model parameters are used to assign values to the prototype graph neural network. The rare patient face samples to be classified are used as the query set, and all samples in the rare patient face dataset are used as the support set to be sent to the prototype graph neural network. The network output is the classification result.
[0026] The training set and the test set are both divided using the n-way k-shot small sample learning method, which is to randomly select n classes from the data set, randomly select k samples from each of the n classes as the support set, and then randomly select q samples from the remaining samples of the n classes as the query set; these n×k+q samples constitute an iteration set.
[0027] A node feature update network is set up, which consists of an aggregation module and a convolution module. The aggregation module means that for each current node, the features of all its adjacent nodes and the corresponding labels are used to aggregate the current node features, and the adjacent node features are divided by the Euclidean distance between the current node label and the adjacent node label to obtain the aggregation information; the convolution module is composed of two convolution layers, a batch normalization layer and a ReLU nonlinear activation layer; the aggregation information and the current node features are sent to the convolution module together, and the output is the updated node feature of the current node.
[0028] The described method sets up an edge feature update network, which consists of four convolutional layers, a batch normalization layer, and a ReLU nonlinear activation layer, and the output of the network is the edge feature between two nodes.
[0029] A node label propagation network is set up, which consists of a weight update module and a convolution module. The weight update module means that for each current node label, all its adjacent node labels are first re-weighted, that is, each edge feature of the current node is divided by the sum of all its edge features, and then multiplied with all the adjacent node labels of the current node to obtain the weighted updated node label of all adjacent node labels; the convolution module is composed of two convolution layers, a batch normalization layer and a ReLU nonlinear activation layer. The current node label and all weighted updated node labels are sent to the convolution module, and the output is the updated label of the current node to realize label propagation.
[0030] The present invention constructs a graph neural network in a "prototype point" manner, ignoring the influence of the number of samples. Although each class has only one point, it is very meaningful for small sample learning. Due to the existence of "prototype points", geometric constraints are introduced to supplement the cross entropy loss, and the loss function is more reliable. At the same time, the model is encouraged to pay more attention to common features within the class, reduce the risk of overfitting, and make the model more robust. On the other hand, the prototype graph neural network not only pays attention to the data samples themselves, but also pays attention to the relationship between samples. It shortens the distance in the feature space to pull points of the same category together, and propagates their labels in the fully connected graph neural network, thereby improving the accuracy of small sample learning classification. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 This is a flow chart of collecting and dividing a rare patient face dataset according to an embodiment of the present invention;
[0032] Figure 2 This is a flow chart of sample feature extraction according to an embodiment of the present invention;
[0033] Figure 3 A flowchart for constructing a neural network prototype of an embodiment of the present invention;
[0034] Figure 4The figure is a flow chart of an embodiment of the present invention. DETAILED DESCRIPTION
[0035] In order to facilitate the understanding of the present invention, the present invention is described in detail below with reference to the accompanying drawings and embodiments. Example
[0036] See also Figure 1 In step 1 of this embodiment, a data set is first collected and divided into a training set and a test set. The sample division method of the training set and the test set using meta-learning adopts the form of 20-way 5-shot, which specifically includes:
[0037] S100: Cooperate with hospitals to collect rare patient face samples to form a rare patient face dataset, classify the face samples according to the disease name and make statistics;
[0038] S110: Divide the collected data set into a training set and a test set in a ratio of 8:2, and the categories of the training set and the test set do not overlap;
[0039] S120: Both the training set and the test set are divided 100 times using the 20-way5-shot sample partitioning method. Each time, 20 classes are randomly selected from the data set. Five samples are randomly selected from each of these 20 classes as the support set. 20 samples are randomly selected from the remaining samples in these 20 classes as the query set. These 120 (20×5+20) samples constitute an iteration set.
[0040] See also Figure 2 In step 2 of this embodiment, for each sample in the iteration set, the rare patient face sample preprocessing is performed. The processed image is input into the embedding subnetwork, its features are extracted, and the feature vector is obtained through two fully connected layers. After that, the feature vectors of each class sample in the support set are summed and averaged as the "prototype point" in the feature space, specifically including:
[0041] S210: Perform two-dimensional face registration on the rare patient face samples in each iteration set, and use the Adaboost cascade face detection classifier based on Haar features to determine whether there is a face; if there is, the face area of the person is cut out and scaled to 84×84 size to obtain a preprocessed sample to avoid interference from factors other than the face in the sample; otherwise, reselect the image;
[0042] S220: The support set and query set samples of the current iteration set after preprocessing are sequentially subjected to a 3×3 convolution with 64 filters, a batch normalization layer, a ReLU nonlinear layer, a 2×2 maximum pooling, and a fully connected layer with dimensions of 8×64 and 1×128 respectively, and then batch normalization and ReLU nonlinear activation are performed to obtain a feature vector of each sample;
[0043] S230, S240: Sum the feature vectors of all samples of each class in the support set, divide by the number of samples in the support set of each class, and obtain the average feature vector of the samples of each class, i.e., the "prototype point".
[0044] See also Figure 3 In step 3 of this embodiment, for each sample in the data set and the calculated "prototype point", a prototype graph neural network is established, and node feature updates, edge feature updates, and node label propagation are performed three times (three layers) to obtain a prototype graph neural network, specifically including:
[0045] S300: Send the support set feature vector, "prototype point" and query set feature vector into the graph neural network as node features for initialization, and initialize the edge features at the same time. The edge features between nodes with the same label are initialized to 1, and the edge features between nodes with different labels are initialized to 0, otherwise the initial value is 0.5;
[0046] S310: Node feature update: For each current node, all its adjacent node features and corresponding labels are used to aggregate the current node, and the adjacent node features are divided by the Euclidean distance between the current node label and the adjacent node label to obtain the aggregate information. The aggregate information and the current node features are passed through two convolutional layers, a batch normalization layer, and a ReLU nonlinear activation layer to obtain the updated node features of the current node.
[0047] S320: Edge feature update is to send the two node features into four convolutional layers, batch normalization layers, and ReLU nonlinear activation layers to obtain the edge features between the two nodes;
[0048] S330: Node label propagation For each current node label, first re-weight all its adjacent node labels, that is, each edge feature of the current node is divided by the sum of all its edge features, and then multiplied by all the adjacent node labels of the current node to obtain the weighted updated node label of all adjacent node labels. The current node label and all weighted updated node labels are sent to two convolutional layers, a batch normalization layer, and a ReLU nonlinear activation layer to obtain the updated label of the current node, thereby realizing label propagation;
[0049] S340: Perform node feature updates, edge feature updates, and node label propagations three times (i.e., the graph neural network has three layers) to obtain an updated prototype graph neural network.
[0050] See also Figure 4 In step 4 of this embodiment, the samples of the query set are classified by the updated prototype graph, and the loss function is constructed with the real labels of the query set samples, and the loss function is back-propagated to update the model parameters. The model parameters with the highest accuracy are recorded and used to classify the samples to be classified, specifically including:
[0051] S400: The predicted label of the query image is selected based on the maximum value independent variable point set of the node, and the rare disease class with the largest probability value is selected as its predicted classification label;
[0052] S410: Calculate the Euclidean distance between each query point and the corresponding "prototype point" in the feature space to obtain the geometric regularization loss, and calculate the binary cross entropy loss between the predicted classification label and the true label of each query point in the query set. The final loss of the entire network is constructed from these two loss functions, and back propagation is performed to update the prototype graph neural network model parameters.
[0053] S420: During the training of 100 iterations, the parameters of the optimal model are recorded. The optimal model parameters are selected to assign values to the prototype graph neural network, and the rare patient face samples to be classified are used as query sets, and all samples in the rare patient face data set are sent to the prototype graph neural network as support sets, and the network output is the classification result.
Claims
1. A rare disease classification method based on prototype graph neural network and small sample learning, characterized in that the method comprises the following specific steps: Step 1: Collection and division of rare patient face datasets 1.1) Collect rare patient face images to form a dataset. The entire dataset is divided into a training set and a test set in a ratio of 8:
2. The category labels of the training set and the test set do not overlap. 1.2) Both the training set and the test set are divided h times using the n-way k-shot small sample learning method, each time as an iteration set. Each iteration set is divided into a support set and a query set. The support set has n classes, each class has k samples, k is greater than 0 and less than or equal to 15; Step 2: Sample feature extraction 2.1) Perform 2D face registration on the rare patient face samples in each iteration set, and use the Adaboost cascade face detection classifier based on Haar features to determine whether there is a face. If there is, the face area is cut out and scaled to 84×84 to obtain the preprocessed sample to avoid interference from factors other than the face in the sample; Otherwise, reselect the image; 2.2) Take the support set samples and query set samples from an iteration set that have been preprocessed in the previous step and send them to the embedding subnetwork: they pass through a 3×3 convolution with 64 filters, a batch normalization layer, a ReLU nonlinear layer, and a 2×2 maximum pooling layer in sequence to obtain the feature map of each sample; 2.3) All feature maps are input into two fully connected layers of 8×64 and 1×128 dimensions respectively to obtain the corresponding feature vectors, which are then batch normalized and ReLU nonlinearly activated to obtain the low-dimensional feature vector of the sample; 2.4) Calculate the average value of the low-dimensional feature vectors of all samples in the support set of each class in an iteration set, that is, sum the low-dimensional feature vectors of all samples in the support set of each class, and divide it by the number of samples in the support set of each class to calculate a virtual point as the prototype point of this class in the feature space; Step 3: Prototype graph neural network construction 3.1) First, the graph neural network is initialized. The prototype graph neural network is constructed by the low-dimensional feature vectors and prototype points of all samples calculated in step 2. The nodes are the low-dimensional feature vectors and prototype points of the samples. The edge features are between 0 and 1, indicating the possibility that two nodes belong to the same class. If the labels of two nodes are equal, the edge feature is set to 1. If they are not equal, the edge feature is set to 0. Otherwise, the edge feature is set to 0.
5. 3.2) Node feature update: First, a node feature update network is set up, and the node features and edge features are input into the node feature update network to obtain updated node features; 3.3) Edge feature update: set up an edge feature update network, input all updated node features into the edge feature update network, and obtain updated edge features; 3.4) Node label propagation: set up a node label propagation network, input the updated edge features and node labels into the node label propagation network, and obtain the updated node labels; 3.5) After L times of node feature updates, edge feature updates, and node label propagation, the updated prototype graph neural network is obtained; Step 4: Label prediction 4.1) After obtaining the updated prototype graph neural network, the predicted label of the query image is selected according to the maximum independent variable point set, and the rare disease class with the largest probability value is selected as the classification prediction result of the query image; 4.2) Design two loss functions to optimize network parameters. One is to calculate the Euclidean distance between each query point and the corresponding prototype point in the feature space to obtain the geometric regularization loss. The other is to calculate the binary cross entropy loss between the predicted label and the true label of each query point in the query set. The final loss of the entire network is constructed from these two loss functions, and back propagation is performed to update the prototype graph neural network model parameters. 4.3) Calculate the classification accuracy of the prototype graph neural network model through the test set, and select the model parameters with the highest accuracy as the optimal model parameters; 4.4) During classification, the optimal model parameters are used to assign values to the prototype graph neural network. The rare patient face samples to be classified are used as the query set, and all samples in the rare patient face dataset are used as the support set to be sent to the prototype graph neural network. The network output is the classification result.
2. The rare disease classification method based on prototype graph neural network and small sample learning according to claim 1 is characterized in that: The training set and the test set are both divided using the n-way k-shot small sample learning method, which is to randomly select n classes from the data set, randomly select k samples from each of the n classes as the support set, and then randomly select q samples from the remaining samples of the n classes as the query set; these n×k+q samples constitute an iteration set.
3. The rare disease classification method based on prototype graph neural network and small sample learning according to claim 1 is characterized in that: A node feature update network is set up, which consists of an aggregation module and a convolution module. The aggregation module means that for each current node, the features of all its adjacent nodes and the corresponding labels are used to aggregate the current node features, and the adjacent node features are divided by the Euclidean distance between the current node label and the adjacent node label to obtain the aggregation information; the convolution module is composed of two convolution layers, a batch normalization layer and a ReLU nonlinear activation layer; the aggregation information and the current node features are sent to the convolution module together, and the output is the updated node feature of the current node.
4. The rare disease classification method based on prototype graph neural network and small sample learning according to claim 1, characterized in that: The described method sets up an edge feature update network, which consists of four convolutional layers, a batch normalization layer, and a ReLU nonlinear activation layer, and the output of the network is the edge feature between two nodes.
5. The rare disease classification method based on prototype graph neural network and small sample learning according to claim 1, characterized in that: A node label propagation network is set up, which consists of a weight update module and a convolution module. The weight update module means that for each current node label, all its adjacent node labels are first re-weighted, that is, each edge feature of the current node is divided by the sum of all its edge features, and then multiplied with all the adjacent node labels of the current node to obtain the weighted updated node label of all adjacent node labels; the convolution module is composed of two convolution layers, a batch normalization layer and a ReLU nonlinear activation layer. The current node label and all weighted updated node labels are sent to the convolution module, and the output is the updated label of the current node to realize label propagation.