Rare disease detection method based on generative adversarial network (GAN)

Abstract

The invention discloses a rare disease detection method based on a generative adversarial network. The method is characterized by comprising the following steps: 1) acquiring multiple treatment records; 2) adopting a multi-head self-attention mechanism and a bidirectional long-short term memory network to generate an overall embedding vector of the patient; 3) generating a supplementary sample by using GAN; and 4) predicting whether a patient suffers from a rare disease according to the embedded representation and the supplementary sample of the patient, and the like. Compared with the prior art, the method has the advantages that the detection precision of rare diseases is improved, the problem of rare samples in rare disease detection is effectively solved, and the method can be widely applied to the fields of clinical medicine, medical data analysis, early prediction and diagnosis of rare diseases and the like, is particularly suitable for the conditions of data imbalance and rare positive samples, and has a wide application prospect. The method improves the detection precision, and has a good application prospect and commercial development value.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning models, and in particular to a rare disease detection method based on a generative adversarial network (GAN). Background Art

[0002] The detection and diagnosis of rare diseases is a major challenge in the global medical system. Rare diseases are defined as diseases with extremely low prevalence (less than 1 in every 10,000 people). Such diseases often present with complex symptoms and overlap with common diseases, which can easily lead to misdiagnosis and missed diagnosis. According to statistics, the diagnosis process for patients with rare diseases takes an average of 7 to 10 years, and patients usually need to see multiple doctors before a diagnosis is confirmed.

[0003] Due to the small number of patients, there are very limited positive samples in the existing electronic health record (EHR) data, and the data imbalance problem is serious, which poses a huge challenge to the early detection and accurate diagnosis of rare diseases. When dealing with imbalanced data, traditional machine learning and deep learning models tend to identify negative samples and ignore positive samples, resulting in a low detection rate for rare diseases. In addition, although existing sampling techniques (such as SMOTE, etc.) can enhance positive data by generating samples, the samples generated by these methods are often difficult to truly reflect the complex characteristics of rare diseases, and therefore have limited effects in practical applications.

[0004] In order to deal with the above problems, generative adversarial networks (GANs) have been widely used in the field of small sample generation in recent years. Positive samples generated by GAN can improve the generalization ability of the model without increasing the collection of real data. However, the GAN model of the existing technology is mainly used to generate raw patient data, and fails to effectively combine the complexity and hierarchical characteristics of medical data, resulting in the quality of generated samples and the accuracy of test results. Summary of the invention

[0005] The present invention proposes a rare disease detection method based on a generative adversarial network (GAN), which adopts a supplementary sample generation method based on a generative adversarial network (GAN) to supplement the lack of positive samples, and introduces a self-attention mechanism and a hierarchical patient embedding model to fully mine the useful information in the patient's electronic health record (EHR), and predict whether the patient has a rare disease based on the patient's embedded representation and supplementary samples. The discriminator distinguishes between generated samples and real samples to improve the detection accuracy of rare diseases. The method is simple and effective, and effectively solves the problem of scarce samples in rare disease detection. It can be widely used in clinical medicine, medical data analysis, and early prediction and diagnosis of rare diseases. It is particularly suitable for situations where data is unbalanced and positive samples are scarce, and the detection accuracy is improved. It has good application prospects and commercial development value.

[0006] The object of the present invention is achieved as follows: a rare disease detection method based on a generative adversarial network (GAN), which is characterized by using a supplementary sample generation method based on a generative adversarial network and a self-attention mechanism and a hierarchical patient embedding model, and predicting rare diseases according to the patient's embedded representation and supplementary samples, specifically comprising the following steps:

[0007] Step 1: Obtain multiple medical visit records of several medical codes from the patient's electronic health record, wherein the medical codes include: diagnosis code, symptom code, treatment procedure code and drug code.

[0008] Step 2: Map each medical code in each visit record into a vector representation through the embedding layer The visit vector V is obtained as shown in the following formula (a): (t) :

[0009]

[0010] in, represents the i-th medical code of the t-th visit.

[0011] The output vector of the embedding layer is 128-dimensional.

[0012] Step 3: Use the multi-head self-attention mechanism to process the medical code embedding of each visit record and generate the visit embedding vector v (t) , the multi-head self-attention mechanism is expressed by the following formula (b):

[0013]

[0014] Where Q, K and V represent the query matrix, key matrix and value matrix respectively; d k is the dimension of the key vector.

[0015] Step 4: Process the patient’s multiple visits through a bidirectional long short-term memory network to generate the forward hidden state of each visit and the backward hidden state The forward hidden state It is expressed by the following formula (c):

[0016]

[0017] in, represents the hidden state of the forward propagation in the bidirectional LSTM at step t; represents the hidden state of the forward propagation in the bidirectional LSTM at step t-1, which is the forward hidden state of the previous step and serves as one of the inputs of the current step; v (t)The code embedding vector representing the patient’s t-th visit is obtained through the multi-layer Transformer and other operations mentioned above and is input into the LSTM for processing.

[0018] The backward hidden state It is expressed by the following formula (d):

[0019]

[0020] in, represents the hidden state of the backpropagation in the bidirectional LSTM at step t; represents the hidden state of the backward propagation in the bidirectional LSTM at the t+1th step. It is the backward hidden state of the next step and serves as one of the inputs of the current step. (t) The code embedding vector representing the patient’s t-th visit is obtained through the multi-layer Transformer and other operations mentioned above and is input into the LSTM for processing.

[0021] Step 5: Calculate the importance weight α of each visit through the self-attention mechanism (t) , used to generate the final embedding h of the patient, with attention weight α (t) It is expressed by the following formula (e):

[0022]

[0023] Among them, u (t) is an intermediate variable used to calculate the attention weight, which helps determine which are the most important among the different medical visits of the patient and is expressed by the following formula (f):

[0024]

[0025] Among them, W s and b s are learnable model parameters, is the vector used to calculate the attention score.

[0026] Step 6: Input the patient's embedding vector \(h\) into a generative adversarial network, which includes: a generator and a discriminator; the generator receives the embedding vector h of the negative patient - , generate complementary positive sample embedding G(h - ); the discriminator receives the generated samples and the real samples, and distinguishes the generated samples from the real positive samples.

[0027] Step 7: The patient’s embedding vector h is used to generate complementary positive sample embedding G(h - ) inputs a disease detector, whose output is a rare disease prediction result for the patient.

[0028] The number of attention heads of the multi-head self-attention mechanism is 8, and the dimensions of the query matrix, key matrix and value matrix of the multi-head self-attention mechanism are 128.

[0029] The hidden state dimension of the bidirectional long short-term memory network is 256 dimensions, of which the forward and backward hidden states are 128 dimensions respectively. The learning rate of the bidirectional long short-term memory network is set to 0.001, and training is performed using the Adam optimization algorithm.

[0030] The generator includes two layers of fully connected neural networks, the output dimension of each layer is 128, and a ReLU activation function is used. The initial weight of the generator is set by the Xavier initialization method.

[0031] The discriminator includes two layers of fully connected neural networks, and finally outputs positive or negative classification probability, and the output layer adopts sigmoid activation function.

[0032] The loss function of the discriminator is the focal loss L shown in the following formula (g): c (θ e ,θ c ):

[0033]

[0034] Among them, γ is the focusing parameter; α is the weight of positive samples; θ e are the network parameters used for embedding generation; θ c are the parameters used for the disease detector; is the distribution of positive samples; is the negative sample distribution; h + Patients with positive samples are embedded; h - Patients with negative samples were embedded.

[0035] The loss function L of the generator g (θ g ) is defined by the following formula (h):

[0036]

[0037] in, To generate the similarity between the sample and the positive distribution; |||G(h - θ g )-h - ∣∣ 2 is the distance between the generated sample and the negative sample; θ g is the model parameter of the generator; λ is the regularization parameter.

[0038] The patient's embedding vector h represents the hidden state of multiple medical records s (t) The final embedding h of the patient is obtained by weighted summation of the self-attention mechanism defined by formula (i) below:

[0039] h=∑ t α (t) s (t) (i).

[0040] During the training process of the generative adversarial network, the generator and the discriminator are trained alternately until the supplementary samples generated by the generator can pass the discriminator.

[0041] The probability output by the disease detector is normalized by a sigmoid activation function to represent the probability that the patient is a positive rare disease patient.

[0042] Compared with the prior art, the present invention has the following beneficial technical effects and significant technical progress:

[0043] 1) This paper applies generative adversarial networks (GANs) to rare disease detection tasks, and solves the problem of data imbalance in traditional rare disease detection methods by generating realistic supplementary positive samples. Unlike existing oversampling methods (such as SMOTE), GANs can generate positive samples that are closer to the distribution of real samples, thereby effectively avoiding overfitting and improving the generalization ability of the model in the case of scarce data.

[0044] 2) The present invention introduces a multi-head self-attention mechanism in the processing of multiple medical records of patients, which can automatically identify the medical records that are most critical for rare disease detection. This mechanism can not only capture the interaction between medical codes, but also assign different weights to different medical records, highlighting the important data in the disease diagnosis process.

[0045] 3) This paper uses a bidirectional long short-term memory network (Bi-LSTM) to serialize the patient's multiple medical records, which can capture the context information at the same time and generate a richer embedded representation of the medical records. Compared with traditional RNN or LSTM networks, Bi-LSTM can better handle complex time series data in rare disease diagnosis and improve the model's understanding of the patient's global health status.

[0046] 4) The present invention introduces a focal loss function in the discriminator training process, which can better handle difficult-to-classify samples, especially focusing on minority samples (i.e., positive rare disease patients), effectively reducing the misclassification rate.

[0047] 5) The experimental results of the present invention on actual rare disease data sets show that the detection accuracy has been significantly improved. Taking inflammatory bowel disease (IBD) detection as an example, the PR-AUC (area under the precision-recall curve) indicator of the present invention reached 0.96, which is 50.1% higher than the best baseline method in the prior art, and the F1 score was also improved to 0.9601. In the idiopathic pulmonary fibrosis (IPF) detection task, the PR-AUC reached 0.2229, which is 41.3% higher than the prior art.

[0048] 6) The present invention significantly alleviates the negative impact of data imbalance on model performance by generating supplementary positive samples through generative adversarial networks (GAN). When rare disease samples are extremely scarce, the generator can generate high-quality positive samples to help the model better identify patients with rare diseases. Experimental results show that the present invention can still maintain efficient detection performance when the proportion of positive samples is extremely low (about 1%), and has good robustness to unbalanced data:

[0049] 7) The present invention can improve the ability to detect rare diseases in the early stages. By introducing the self-attention mechanism and the Bi-LSTM model, it can better capture the course of disease information of patients with rare diseases, especially in the early stages of the disease, and can promptly detect potential diseases. Experiments show that when only the first 20% of patient records are used, the model of the present invention can still maintain a high PR-AUC (0.81) and F1 score (0.79), which indicates that this method has the ability of early detection.

[0050] 8) The present invention can be widely used in clinical medicine, medical data analysis, early prediction and diagnosis of rare diseases, and is particularly suitable for situations where data is unbalanced and positive samples are scarce. It can improve the accuracy of detection and has good application prospects and commercial development value. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flow chart of the present invention;

[0052] Figure 2 It is an algorithm block diagram of the architecture of the present invention;

[0053] Figure 3 A system schematic diagram of the architecture of the present invention. DETAILED DESCRIPTION

[0054] See also Figure 1 The present invention includes: from the collection of patient medical records, data preprocessing, sequence modeling to generative adversarial network training and detection, the specific process is as follows:

[0055] S110: Obtain multiple medical visit records of several medical codes from the patient's electronic health record, including: positive (rare disease patients) and negative (non-rare disease patients) samples.

[0056] S120: Embed each medical code into a 128-dimensional vector and arrange all medical records in chronological order to form a time series.

[0057] S130: Process each visit record through a multi-head self-attention mechanism to generate a visit embedding, and generate an embedding representation of the patient through Bi-LSTM.

[0058] S140: The generator generates supplementary positive samples, the discriminator distinguishes between real samples and generated samples, and adversarial training improves the quality of sample generation.

[0059] S150: Detect whether the patient has a rare disease using the patient embedding and the generated supplementary samples as input.

[0060] S160: Use PR-AUC and F1 scores to evaluate the rare disease detection results and verify the accuracy of the model.

[0061] See also Figure 2 , the algorithm block diagram of the architecture of the present invention mainly contains three parts: self-attention and self-attention and hierarchical patient embedding modules, disease detector module and complementary generator module. First, the patient embedding is calculated through the hierarchical attention mechanism; then, the disease detector is trained using focal loss; finally, a complementary generator is established to convert the negative embedding into a positive embedding with slight modification, and the disease detector is fine-tuned with the real and generated embeddings. The implementation of each module and its algorithmic details are described in detail below:

[0062] 1) Self-Attention and Hierarchical Patient Embedding Module

[0063] A key challenge in rare disease detection is how to extract useful features from complex and long-term patient electronic health record (EHR) data. EHR contains detailed records of multiple visits to the patient, each of which contains several medical codes (such as diagnosis codes, treatment codes, and drug codes). These data are often multidimensional, time-irregular, and the importance of different visits varies greatly. Therefore, traditional neural network architectures have difficulty in effectively processing such complex data. To address these problems, the present invention adopts a hierarchical structure, first embedding each visit record, and then processing all visit embeddings to generate an overall embedded representation of the patient.

[0064] 2) Visit Embedding

[0065] Each medical record consists of several medical codes, such as diagnosis code, symptom code and treatment procedure code. In order to process this information, the present invention adopts a self-attention mechanism based on Transformer, and its specific steps are as follows:

[0066] 2-1: Vector representation of medical codes

[0067] Each medical code is mapped to a vector representation These vectors can be obtained through pre-trained embedding models (such as Word2Vec or ICD-10 medical code embedding models). Mapped to vector representation through the embedding layer The visit vector V is obtained as shown in the following formula (a): (t) :

[0068]

[0069] in, represents the i-th medical code of the t-th visit.

[0070] 2-2: Transformer's multi-head self-attention mechanism

[0071] In order to capture the relationship between different medical codes, the multi-head self-attention mechanism in the Transformer model is used. This mechanism allows the model to calculate attention weights on different "heads" at the same time, thereby learning the interactive information between medical codes. The multi-head self-attention mechanism is expressed by the following formula (b):

[0072]

[0073] Where Q, K and V represent the query matrix, key matrix and value matrix respectively, d k is the dimension of the key vector.

[0074] By combining the interactive information of different dimensions through multiple self-attention heads, the final visit embedding vector v is obtained. (t) .

[0075] 2-3: Remove position embedding

[0076] Since there is no obvious sequential relationship between medical codes, the present invention omits the position embedding in Transformer, which can better handle the disorder of the medical code set.

[0077] 3) Patient Embedding

[0078] The patient's visit embedding needs to integrate all of his or her visit records. In order to make full use of the patient's historical and future visit information, a bidirectional LSTM (Bi-LSTM) model is used. Bi-LSTM can process past and future visit sequences at the same time, thereby generating richer patient feature embeddings.

[0079] 3-1: Forward and backward calculation of Bi-LSTM

[0080] For each patient, the embedding vector v of the tth visit (t) The forward hidden state is calculated by Bi-LSTM and the backward hidden state The forward hidden state It is expressed by the following formula (c):

[0081]

[0082] in, represents the hidden state of the forward propagation in the bidirectional LSTM at step t; represents the hidden state of the forward propagation in the bidirectional LSTM at step t-1, which is the forward hidden state of the previous step and serves as one of the inputs of the current step; v (t) The code embedding vector representing the patient’s t-th visit is obtained through the multi-layer Transformer and other operations mentioned above and is input into the LSTM for processing.

[0083] The backward hidden state It is expressed by the following formula (d):

[0084]

[0085] in, represents the hidden state of the backpropagation in the bidirectional LSTM at step t; represents the hidden state of the backward propagation in the bidirectional LSTM at the t+1th step. It is the backward hidden state of the next step and serves as one of the inputs of the current step. (t) The code embedding vector representing the patient’s t-th visit is obtained through the multi-layer Transformer and other operations mentioned above and is input into the LSTM for processing.

[0086] Finally, the expression of the tth visit is obtained This bidirectional processing can capture the contextual information of the patient's condition throughout the entire visit sequence.

[0087] 3-2: Application of self-attention mechanism

[0088] In order to weight the patient's multiple medical records, the present invention further uses a self-attention mechanism to determine the importance of each medical visit to the patient's overall embedding. By calculating the attention weight u of each medical visit (t) , and normalized to obtain the attention weight α defined by the following formula (e) (t) :

[0089]

[0090] Among them, u (t) is an intermediate variable used to calculate the attention weight, which helps determine which are the most important among the different medical visits of the patient and is expressed by the following formula (f):

[0091]

[0092] Among them, W s and b s are learnable model parameters, is the vector used to calculate the attention score.

[0093] The role of the attention mechanism is to automatically assign a weight to each patient visit, thereby identifying those visits that are most important for diagnosing rare diseases.

[0094] 3-3: Final embedding of the patient

[0095] By dividing the embedding vector of each visit by weight α (t) The weighted summation yields the final embedding h of the patient represented by (i) below:

[0096] h=∑ t α (t) s (t) (i).

[0097] Among them, s (t) is the information of the bidirectional LSTM at time step t.

[0098] This embedded representation contains all the visit information and highlights the key visit records related to the disease.

[0099] 4) Generative Adversarial Networks (GANs)

[0100] In order to improve the detection performance of rare diseases, this paper proposes a supplementary sample generator based on generative adversarial network (GAN). By using the data of patients with uncertain diagnosis to generate positive and negative samples, the rare disease detection model can be more effective in the case of extremely unbalanced data.

[0101] 5) GAN Architecture

[0102] The generative adversarial network consists of a generator G and a discriminator D, where the input of the generator G is the embedding h of the negative patient - , these embeddings are slightly modified by the generator to generate samples similar to positive patients, that is, candidate positive samples are generated; the network structure of the generator G consists of two fully connected layers, the output dimension of each layer is 128, and the ReLU activation function is adopted; the discriminator D is used as a detector of rare diseases to distinguish the generated positive samples from the real positive samples, and outputs the probability that the sample is positive by performing a binary classification task on the input sample; the network structure of the discriminator D also consists of two fully connected layers, and finally outputs the probability value through the sigmoid activation function.

[0103] 6) Loss Function

[0104] In order to optimize the adversarial training of the generator and the discriminator, the following loss function is defined:

[0105] 6-1: The goal of the generator is to generate positive samples that can deceive the discriminator. The loss function of the generator is defined by the following formula (h):

[0106]

[0107] in, To generate the similarity between the sample and the positive distribution, this loss ensures that the generated sample is as close to the positive sample distribution as possible; |||G(h - θ g )-h - ∣∣ 2 To increase the distance between the generated samples and the negative samples, this loss controls the modification of the generated samples, ensuring that the generator does not deviate from its original distribution when modifying the negative samples.

[0108] 6-2: The discriminator uses a focal loss function to deal with the situation where positive and negative samples are extremely unbalanced. This loss function focuses on samples that are difficult to classify, thereby enhancing the model's ability to detect rare diseases. The focal loss L c (θ e ,θ c ) is defined by the following formula (g):

[0109]

[0110] Among them, γ is the focusing parameter used to weight the difficult-to-distinguish samples, and α is the weight of the positive samples.

[0111] 7) Supplementary sample generation

[0112] Through training, the generator can transform negative samples into complementary samples that are close to positive samples. The discriminator, as a detector for rare diseases, continuously optimizes the classification boundary by distinguishing between generated samples and real samples. The generated complementary samples are input into the detector through the self-attention embedding module, ultimately improving the detection accuracy of rare diseases.

[0113] 8) Experimental design and results

[0114] The invention is validated in two real rare disease detection tasks: inflammatory bowel disease (IBD) and idiopathic pulmonary fibrosis (IPF). The two datasets contain millions of patient records, and the experimental results show that the CONAN framework of the invention performs significantly better than the existing technology in rare disease detection tasks.

[0115] 9) The dataset contains 1405 positive samples and 108,047 negative samples, with an average of 798 visits per patient. IPF dataset: contains 9996 positive samples and 24,757,572 negative samples, with an average of 597 visits per patient.

[0116] 10) Experimental results

[0117] The experiment used evaluation indicators such as PR-AUC (area under the precision-recall curve) and F1 score. In the IBD detection task, CONAN's PR-AUC reached 0.96, which is 50.1% higher than the existing best baseline method. In the IPF detection task, CONAN's PR-AUC reached 0.2229, which is 41.3% higher than the baseline method. Through a large number of experimental evaluations, the present invention has demonstrated significant technical effects in rare disease detection tasks.

[0118] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0119] Example 1

[0120] See also Figure 3, a rare disease detection system based on a generative adversarial network using the architecture of the present invention, the system comprising: an electronic health record (EHR) processing module, a multi-head self-attention processing module, a bidirectional long short-term memory network (Bi-LSTM) processing module, a self-attention weighting module, a generative adversarial network (GAN) module and a disease detector module, the EHR processing module is used to obtain and embed multiple medical records of a patient; the multi-head self-attention processing module is used to process the medical code of each medical visit and generate an embedded representation of each medical visit; the Bi-LSTM processing module is used to generate a hidden state representation of the patient's medical record;

[0121] The self-attention weighted module is used to generate an overall embedding representation of the patient according to the importance weight of each medical record; the GAN module is used to generate supplementary samples and distinguish between positive and negative samples; and the disease detector module is used to generate prediction results for rare diseases according to the patient's embedding representation and supplementary samples.

[0122] Example 1

[0123] This embodiment uses the present invention to detect inflammatory bowel disease (IBD), and the specific steps are as follows:

[0124] Step 1: Data preparation

[0125] 1-1: A dataset containing electronic health records (EHR) was extracted from a medical database. The EHR dataset included positive samples: 1,405 patients diagnosed with IBD; negative samples: 108,047 non-IBD patients.

[0126] 1-2: Each patient's record contains multiple visits, and each visit record includes several medical codes, such as diagnosis code, symptom code, treatment code, etc. On average, each patient has 798 visit records.

[0127] Step 2: Data preprocessing

[0128] 2-1: The medical code in each visit is converted into a 128-dimensional vector through a pre-trained embedding model (such as Word2Vec or ICD-10 embedding model) to form a visit embedding matrix.

[0129] 2-2: Arrange all medical records of each patient in chronological order to form time series data, which will be input into the subsequent model.

[0130] Step 3: Self-Attention and Bi-LSTM Processing

[0131] 3-1: All the patient's medical records are input into the multi-head self-attention mechanism. For each visit, the interaction between medical codes is processed by the self-attention mechanism to generate the visit embedding vector v (t) .

[0132] 3-2: Process the patient’s multiple visit records through a bidirectional long short-term memory network (Bi-LSTM) to generate the forward hidden state of each visit and the backward hidden state These hidden states are aggregated to form the final embedding representation s for each visit (t) .

[0133] 3-3: Use the self-attention mechanism to calculate the importance weight of each visit \(\alpha^{(t)}\), and perform weighted summation of each visit according to the weight to generate the final embedding representation of the patient \(h\) for disease detection.

[0134] Step 4: Generative Adversarial Network (GAN) Training

[0135] 4-1: Generator training: Generator G accepts negative patient embedding h - As input, to generate complementary positive samples, the generator uses two fully connected layers, each layer outputs a 128-dimensional vector, and the activation function is ReLU.

[0136] 4-2: Discriminator training: The discriminator D distinguishes true positive samples h + And the generated supplementary sample G(h - ) is trained, and the discriminator passes through two fully connected layers and outputs the classification probability.

[0137] 4-3: Adversarial training: The training of the generator and the discriminator is carried out alternately. The generator tries to generate samples that can deceive the discriminator, while the discriminator continuously improves its ability to distinguish between generated samples and real samples.

[0138] Step 5: Training of Disease Detector

[0139] 5-1: The supplementary samples generated by the generative adversarial network (GAN) and the patient’s true embedding representation h are input into the disease detector together.

[0140] 5-2: The disease detector is classified through two fully connected layers and finally outputs the probability of whether the patient has IBD. The output of the disease detector is normalized by the sigmoid activation function.

[0141] Step 6: Experimental results

[0142] 6-1: The above method was experimentally evaluated in the IBD detection task. The results showed that the present invention can significantly improve the classification performance. The PR-AUC (area under the precision-recall curve) reached 0.96, which is 50.1% higher than the prior art. The F1 score was 0.9601, indicating that the method of the present invention performs well in the case of rare positive samples.

[0143] Example 2

[0144] The present invention detects idiopathic pulmonary fibrosis (IPF), and the specific steps are as follows:

[0145] Step 1: Data preparation

[0146] 1-1: A dataset related to idiopathic pulmonary fibrosis (IPF) was obtained from the EHR database. The dataset included positive samples: 9996 patients diagnosed with IPF; negative samples: 24,757,572 non-IPF patients.

[0147] 1-2: Each patient's record contains multiple visits. On average, each patient has 597 visit records, and each visit record contains multiple medical codes.

[0148] Step 2: Data preprocessing

[0149] 2-1: Use a pre-trained Word2Vec or ICD-10 embedding model to convert each medical code into a 128-dimensional embedding vector.

[0150] 2-2: Arrange all the patient's medical records in chronological order to form time series data as input for subsequent models.

[0151] Step 3: Self-Attention and Bi-LSTM Processing

[0152] 3-1: Input all the patient's medical records, and the medical code is processed through a multi-head self-attention mechanism to generate an embedding representation v for each visit (t) .

[0153] 3-2: Bi-LSTM processes all medical records, generates the forward hidden state and backward hidden state of each medical visit, and combines them to generate the medical visit embedding vector s (t) .

[0154] 3-3: Use the self-attention mechanism to calculate the importance of each visit and generate the overall embedding representation h of the patient.

[0155] Step 4: Generative Adversarial Network (GAN) Training

[0156] 4-1: Generator training: The generator is trained by accepting negative patient embedding h- As input, complementary positive samples are generated. The generator consists of two fully connected layers, each of which outputs a 128-dimensional vector.

[0157] 4-2: Discriminator training: The discriminator is trained for classification by receiving the embedded representations of generated samples and true positive samples.

[0158] 4-3: Adversarial training: The training of the generator and the discriminator is performed alternately until the samples generated by the generator can pass the discriminator.

[0159] Step 5: Training of Disease Detector

[0160] 5-1: The generated supplementary samples are input together with the patient embedding representation h through the disease detector for rare disease classification.

[0161] 5-2: The classification result output of the disease detector is the probability of whether the patient has IPF and is normalized by the sigmoid activation function.

[0162] Step 6: Experimental results

[0163] 6-1: The experimental results in the IPF detection task show that the PR-AUC reaches 0.2229, which is 41.3% higher than the existing baseline method. This experiment shows that the present invention can still effectively improve the detection performance when there are very few positive samples, and is suitable for the early detection of rare diseases such as idiopathic pulmonary fibrosis.

[0164] The above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A rare disease detection method based on generative adversarial networks, characterized in that: A supplementary sample generation method based on a generative adversarial network, a self-attention mechanism, and a hierarchical patient embedding model are used to predict rare diseases based on the patient's embedded representation and supplementary samples, specifically including the following steps: Step 1: Obtain multiple medical visit records of several medical codes from the patient's electronic health record, wherein the medical codes include: diagnosis code, symptom code, treatment procedure code and drug code; Step 2: Map each medical code in each visit record into a vector representation through the embedding layer The visit vector V is obtained as shown in the following formula (a): (t) : in, represents the i-th medical code of the t-th visit; The output vector of the embedding layer is 128-dimensional; Step 3: Use the multi-head self-attention mechanism to process the medical code embedding of each visit record and generate the visit embedding vector v (t) , the multi-head self-attention mechanism is expressed by the following formula (b): Where Q, K and V represent the query matrix, key matrix and value matrix respectively; d k is the dimension of the key vector; Step 4: Process the patient’s multiple visits through a bidirectional long short-term memory network to generate the forward hidden state of each visit and the backward hidden state The forward hidden state It is expressed by the following formula (c): in, represents the hidden state of the forward propagation in the bidirectional LSTM at step t; represents the hidden state of the forward propagation in the bidirectional LSTM at step t-1, which is the forward hidden state of the previous step and serves as one of the inputs of the current step; v (t) represents the code embedding vector of the patient at the tth visit; The backward hidden state It is expressed by the following formula (d): in, represents the hidden state of the backpropagation in the bidirectional LSTM at step t; represents the hidden state of the backward propagation in the bidirectional LSTM at the t+1th step, which is the backward hidden state of the next step and serves as one of the inputs of the current step; v (t) represents the code embedding vector of the patient at the tth visit; Step 5: Calculate the importance weight α of each visit through the self-attention mechanism (t) , used to generate the final embedding h of the patient, with attention weight α (t) It is expressed by the following formula (e): Among them, u (t) is an intermediate variable for calculating the attention weight and is expressed by the following formula (f): Among them, W s and b s are learnable model parameters; is the vector for calculating the attention score; Step 6: Input the patient's embedding vector h into a generative adversarial network, which includes a generator and a discriminator; the generator receives the embedding vector h of the negative patient - , generate complementary positive sample embedding G(h - ); the discriminator receives the generated samples and the real samples, and distinguishes the generated samples from the real positive samples; Step 7: The patient’s embedding vector h is used to generate complementary positive sample embedding G(h - ) inputs a disease detector, whose output is a rare disease prediction result for the patient.

2. The rare disease detection method based on generative adversarial network according to claim 1, characterized in that: The number of attention heads of the multi-head self-attention mechanism is 8, and the dimensions of the query matrix, key matrix and value matrix of the multi-head self-attention mechanism are 128.

3. The rare disease detection method based on generative adversarial network according to claim 1, characterized in that: The hidden state dimension of the bidirectional long short-term memory network is 256 dimensions, of which the forward and backward hidden states are 128 dimensions respectively. The learning rate of the bidirectional long short-term memory network is 0.001, and it is trained by the Adam optimization algorithm.

4. The rare disease detection method based on generative adversarial network according to claim 1, characterized in that: The generator includes two layers of fully connected neural networks, the output dimension of each layer is 128, and a ReLU activation function is used. The initial weight of the generator is set by the Xavier initialization method.

5. The rare disease detection method based on generative adversarial network according to claim 1, characterized in that: The discriminator includes two layers of fully connected neural networks, and finally outputs positive or negative classification probability, and the output layer adopts sigmoid activation function.

6. The rare disease detection method based on generative adversarial network according to claim 1, characterized in that: The loss function of the discriminator is the focal loss L defined by the following formula (g): c (θ e ,θ c ): Among them, γ is the focusing parameter; α is the weight of positive samples; θ e is the network parameter generated by embedding; θ c are the parameters of the disease detector; is the distribution of positive samples; is the negative sample distribution; h + Patients with positive samples are embedded; h - Patients with negative samples were embedded.

7. The rare disease detection method based on generative adversarial network according to claim 1, characterized in that: The loss function L of the generator g (θ g ) is defined by the following formula (h): in, is the similarity between the generated sample and the positive distribution; |||G(h - θ g )-h - ∣∣ 2 is the distance between the generated sample and the negative sample; θ g is the model parameter of the generator; λ is the regularization parameter.

8. The rare disease detection method based on generative adversarial network according to claim 1, characterized in that: The patient's embedding vector \(h\) represents the hidden state of multiple medical records s (t) The final embedding h of the patient is obtained by weighted summation of the self-attention mechanism defined in formula (i): h=∑ t a (t) s (t) (i)。 9. The rare disease detection method based on generative adversarial network according to claim 1, characterized in that: During the training process of the generative adversarial network, the generator and the discriminator are trained alternately until the supplementary samples generated by the generator can pass the discriminator.

10. The rare disease detection method based on generative adversarial network according to claim 1, characterized in that: The probability output by the disease detector is normalized by a sigmoid activation function to represent the probability that the patient is a positive rare disease patient.

Images