AI-based HAE rare disease risk prediction method and device

Abstract

The present invention relates to an AI-based HAE rare disease risk prediction method and device, including: training and fine-tuning an open source Llama3 base model through preprocessed medical data, extracting HAE disease medical knowledge, and inputting the fine-tuned open source Llama3 base model as training data to output a first risk probability distribution. The training data is used as the input of the TinyBERT model to output a second risk probability distribution corresponding to the HAE disease medical knowledge, and the cross entropy loss function is called to calculate the difference between the first risk probability distribution and the second risk probability distribution. The gradient of the difference is calculated by the back propagation algorithm, the model parameters are updated using an optimizer, and the TinyBERT model is iteratively trained and fine-tuned to obtain a HAE rare disease risk prediction model. The preprocessed medical record data of the current patient is used as the input of the HAE rare disease risk prediction model, and the risk prediction score corresponding to the medical record data is output.

Description

Technical Field

[0001] The present invention relates to the field of medical assistance and intelligent medical technology, and in particular to an AI-based HAE rare disease risk prediction method and device. Background Art

[0002] The low incidence of rare diseases makes it challenging to screen for these diseases in large populations, and the effectiveness of screening depends on its cost-effectiveness ratio, and screening for rare diseases usually requires higher costs and more complex technical support. Rare diseases often show great clinical heterogeneity, and even the same disease may have different manifestations in different patients. This makes it more difficult to develop unified screening standards and detection methods. The traditional manual screening model is inefficient due to individual differences in understanding and the need to check medical records one by one. Therefore, there is an urgent need to improve screening efficiency through certain technical means.

[0003] At present, in the traditional screening method for rare diseases, for example, the patient's symptoms are collected through intelligent medical devices, relevant disease data is accessed, and various input data are standardized, the standardized diseases are analyzed, a data decision tree is constructed, the disease data is classified and screened, and a relational database corresponding to the phenotypic disease information and the disease type is established, and the classified screening data is transmitted to the relational database. According to the results of the classification screening, the classified screening data information is compared with the sample parameters of the relational database, and the data difference information and disease characteristic information after the comparison are summarized. The summarized data is generated into a report and stored in the database and displayed on the terminal device. This solution collects data through intelligent devices, standardizes the data, classifies and screens the disease data through a data decision tree, and finally compares the sample parameters of the relational database by building a relational database, stores the compared data, and summarizes it on the terminal.

[0004] However, this method cannot provide pop-up reminders to patients at risk of HAE rare diseases when the patient is visiting the doctor or the doctor is editing the medical record. It is difficult to provide timely reminders and send the patient for examination in time, and the cost of contacting the patient after he leaves is high. In addition, this method makes it difficult to automatically classify the risk of HAE rare diseases, making it difficult for clinicians to perform graded management of patients at risk of rare diseases. It also does not provide an AI-based intelligent screening method for batches of historical patients, so the screening efficiency and accuracy are relatively low.

[0005] In summary, the intelligence level of existing rare disease screening methods needs to be further improved, and the accuracy and efficiency of rare disease screening also need to be further improved. Summary of the invention

[0006] Based on this, it is necessary to provide an AI-based HAE rare disease risk prediction method and device with a high degree of intelligence and high rare disease screening accuracy and efficiency to address the above-mentioned technical problems.

[0007] The present invention provides an AI-based method for predicting the risk of HAE rare diseases, the method comprising:

[0008] Obtaining public medical data on the Internet and preprocessing the medical data to train and fine-tune the open source Llama3 base model through the preprocessed medical data, wherein the preprocessing includes data cleaning and data partitioning;

[0009] Extracting HAE disease medical knowledge from the open source Llama3 base model, and inputting the HAE disease medical knowledge as training data into the fine-tuned open source Llama3 base model to output a first risk probability distribution corresponding to the HAE disease medical knowledge;

[0010] Using the training data as input of the TinyBERT model to output a second risk probability distribution corresponding to the HAE disease medical knowledge, and calling a cross entropy loss function to calculate the difference between the first risk probability distribution and the second risk probability distribution;

[0011] The gradient of the difference is calculated by a back-propagation algorithm, and the parameters of the TinyBERT model are updated using an optimizer to iteratively train and fine-tune the TinyBERT model to obtain a HAE rare disease risk prediction model;

[0012] The preprocessed medical record data of the current patient is used as the input of the HAE rare disease risk prediction model to output the HAE rare disease risk prediction score corresponding to the medical record data, and the risk prediction score is used to characterize the probability of the current patient having the HAE rare disease.

[0013] In one embodiment, obtaining medical data publicly available on the Internet and preprocessing the medical data to train and fine-tune the open source Llama3 base model through the preprocessed medical data includes:

[0014] Acquire a preset target task type of the open source Llama3 base model to select target medical data related to the target task type from the medical data;

[0015] Removing noise, duplicate data, and useless data from the target medical data to clean the target medical data, and dividing the cleaned target medical data into a training set, a test set, and a validation set;

[0016] Among them, the target task type is to score the disease risk of the corresponding patient based on the medical record data, and to grade the risk score results. The training set is used to train and fine-tune the open source Llama3 base model by fine-tuning the hyperparameters. The validation set is used to monitor and evaluate the indicator performance of the open source Llama3 base model. The test set is used to evaluate the performance of the fine-tuned open source Llama3 base model.

[0017] In one embodiment, extracting HAE disease medical knowledge from the open source Llama3 base model and inputting the HAE disease medical knowledge as training data into the fine-tuned open source Llama3 base model to output a first risk probability distribution corresponding to the HAE disease medical knowledge includes:

[0018] Taking the HAE disease risk prediction task as the target task of the TinyBERT model, knowledge distillation is performed on the medical data in the fine-tuned open source Llama3 base model to extract the corresponding HAE disease medical knowledge;

[0019] Calling the fine-tuned open source Llama3 base model to predict the HAE disease medical knowledge to output hard labels and soft labels corresponding to each input data;

[0020] The hard label is the risk score result corresponding to each input data, and the soft label is the probability distribution corresponding to the risk score result. The probability distribution is used to characterize the probability that each input data belongs to a preset risk score interval.

[0021] In one embodiment, the gradient of the difference is calculated by a back-propagation algorithm, and the parameters of the TinyBERT model are updated using an optimizer to iteratively train and fine-tune the TinyBERT model to obtain a HAE rare disease risk prediction model, including:

[0022] Training the last layer of the TinyBERT model and evaluating the performance of the TinyBERT model using a validation set of HAE disease medical knowledge every two epochs;

[0023] When the difference between the performance evaluation results is lower than a first threshold, the previous layer of the TinyBERT model is gradually unfrozen to avoid initial overfitting;

[0024] Each period is a process in which the HAE disease medical knowledge undergoes one forward propagation and one backward propagation in the TinyBERT model.

[0025] In one embodiment, the method further comprises:

[0026] The activation frequency of each neuron in the TinyBERT model in the training set is counted, and neurons whose activation frequency is lower than a second threshold are removed, where the second threshold is the average activation frequency of neurons in the training set.

[0027] In one embodiment, the pre-processed medical record data of the current patient is used as the input of the HAE rare disease risk prediction model to output the HAE rare disease risk prediction score corresponding to the medical record data, including:

[0028] Acquire the current medical record data and historical medical record data corresponding to the current patient, and pre-process the current medical record data and historical medical record data to obtain a data set result;

[0029] The HAE rare disease risk prediction model is called to predict the data set results to output a probability value corresponding to the HAE rare disease risk corresponding to the current patient.

[0030] In one embodiment, the method further comprises:

[0031] The probability values ​​are sorted according to their sizes to obtain a probability value sequence, and the probability value sequence is fitted to a corresponding monotone non-decreasing function through Isotonic regression to calibrate the probability value.

[0032] In one embodiment, the pre-processed medical record data of the current patient is used as the input of the HAE rare disease risk prediction model to output the HAE rare disease risk prediction score corresponding to the medical record data, and then includes:

[0033] Based on the HAE rare disease risk prediction score, the current patient is divided into different risk levels, and the risk levels include at least a first risk level, a second risk level and a third risk level.

[0034] In one embodiment, the first risk level is the risk level corresponding to when the HAE rare disease risk prediction score exceeds the third threshold, the second risk level is the risk level corresponding to when the HAE rare disease risk prediction score is between the third threshold and the fourth threshold, the third risk level is the risk level corresponding to when the HAE rare disease risk prediction score is lower than the fourth threshold, and the third threshold and the fourth threshold are both preset according to clinical needs and statistics.

[0035] The present invention also provides an AI-based HAE rare disease risk prediction device, the device comprising:

[0036] A large model fine-tuning module, which is used to obtain public medical data on the Internet and pre-process the medical data, so as to train and fine-tune the open source Llama3 base model through the pre-processed medical data, wherein the pre-processing includes data cleaning and data partitioning;

[0037] A knowledge distillation module, for extracting HAE disease medical knowledge from the open source Llama3 base model, and inputting the HAE disease medical knowledge as training data into the fine-tuned open source Llama3 base model to output a first risk probability distribution corresponding to the HAE disease medical knowledge;

[0038] An output difference calculation module is used to use the training data as an input of the TinyBERT model to output a second risk probability distribution corresponding to the HAE disease medical knowledge, and call a cross entropy loss function to calculate the difference between the first risk probability distribution and the second risk probability distribution;

[0039] A small model fine-tuning module, used to calculate the gradient of the difference through a back-propagation algorithm, and use an optimizer to update the parameters of the TinyBERT model, so as to iteratively train and fine-tune the TinyBERT model to obtain a HAE rare disease risk prediction model;

[0040] The HAE rare disease risk prediction module is used to use the pre-processed medical record data of the current patient as the input of the HAE rare disease risk prediction model to output the risk prediction score corresponding to the medical record data, and the risk prediction score is used to characterize the probability of the current patient having the HAE rare disease.

[0041] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements any of the AI-based HAE rare disease risk prediction methods described above.

[0042] The present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements any of the AI-based HAE rare disease risk prediction methods described above.

[0043] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the AI-based HAE rare disease risk prediction methods described above.

[0044] The above-mentioned AI-based HAE rare disease risk prediction method and device obtains public medical data on the Internet, and pre-processes the medical data by data cleaning and data partitioning, so as to train and fine-tune the open source Llama3 base model through the pre-processed medical data. Then extract HAE disease medical knowledge from the open source Llama3 base model, and input the HAE disease medical knowledge as training data into the fine-tuned open source Llama3 base model to output a risk probability distribution corresponding to the HAE disease medical knowledge. The training data is used as the input of the TinyBERT model to output another risk probability distribution corresponding to the HAE disease medical knowledge, and the cross entropy loss function is called to calculate the difference between the two risk probability distributions. Subsequently, the gradient of the difference is calculated by the back propagation algorithm, and the parameters of the TinyBERT model are updated using the optimizer to iteratively train and fine-tune the TinyBERT model to obtain the HAE rare disease risk prediction model. Finally, the pre-processed medical record data of the current patient is used as the input of the HAE rare disease risk prediction model to output the HAE rare disease risk prediction score corresponding to the medical record data to characterize the probability of the current patient's HAE rare disease. This method is based on AI and identifies all the patient's in-hospital medical records (from admission to discharge) to predict HAE risks. While ensuring the comprehensiveness of the screening, it is not easy to miss information due to human errors, thereby improving the screening efficiency and accuracy of rare disease risks, and has a high degree of intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 One of the flow charts of the AI-based HAE rare disease risk prediction method provided by the present invention;

[0047] Figure 2 A schematic diagram of the overall process of HAE rare disease risk prediction of the HAE rare disease risk prediction method based on AI in a specific embodiment provided by the present invention;

[0048] Figure 3 A schematic diagram of a medical record information preprocessing process of a method for predicting the risk of HAE rare diseases based on AI in a specific embodiment provided by the present invention;

[0049] Figure 4 The second flowchart of the AI-based HAE rare disease risk prediction method provided by the present invention;

[0050] Figure 5 The third flowchart of the AI-based HAE rare disease risk prediction method provided by the present invention;

[0051] Figure 6 The fourth flowchart of the method for predicting the risk of HAE rare diseases based on AI provided by the present invention;

[0052] Figure 7 The fifth flowchart of the method for predicting the risk of HAE rare diseases based on AI provided by the present invention;

[0053] Figure 8 The sixth flowchart of the method for predicting the risk of HAE rare diseases based on AI provided by the present invention;

[0054] Fig. 9 A schematic diagram of the structure of the HAE rare disease risk prediction device based on AI provided by the present invention;

[0055] Fig.10 This is a diagram of the internal structure of the computer device provided by the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] Combine the following Figures 1 to 10 The present invention describes the AI-based HAE rare disease risk prediction method and device.

[0058] like Figure 1 As shown, in one embodiment, a method for predicting the risk of HAE rare diseases based on AI comprises the following steps:

[0059] Step S110, obtaining medical data publicly available on the Internet, and preprocessing the medical data, so as to train and fine-tune the open source Llama3 base model through the preprocessed medical data, and the preprocessing includes data cleaning and data partitioning.

[0060] Specifically, the server obtains medical data publicly available on the Internet, and preprocesses the medical data by data cleaning and data partitioning, so as to train and fine-tune the open source Llama3 base model through the preprocessed medical data.

[0061] Step S120, extracting HAE disease medical knowledge from the open source Llama3 base model, and inputting the HAE disease medical knowledge as training data into the fine-tuned open source Llama3 base model to output a first risk probability distribution corresponding to the HAE disease medical knowledge.

[0062] Specifically, the server extracts the corresponding HAE disease medical knowledge from the open source Llama3 base model fine-tuned in step S110, and inputs the HAE disease medical knowledge as training data into the fine-tuned open source Llama3 base model to output the risk probability distribution corresponding to the HAE disease medical knowledge, that is, the first risk probability distribution.

[0063] Step S130, using the training data as input of the TinyBERT model to output a second risk probability distribution corresponding to the HAE disease medical knowledge, and calling the cross entropy loss function to calculate the difference between the first risk probability distribution and the second risk probability distribution.

[0064] Specifically, the server inputs the HAE disease medical knowledge extracted in step S120 as training data into the TinyBERT model to output the risk probability distribution corresponding to the HAE disease medical knowledge, that is, the second risk probability distribution, and calls the cross entropy loss function to calculate the difference between the first risk probability distribution and the second risk probability distribution.

[0065] Step S140, calculate the gradient of the difference through the back propagation algorithm, and use the optimizer to update the parameters of the TinyBERT model to iteratively train and fine-tune the TinyBERT model to obtain the HAE rare disease risk prediction model.

[0066] Specifically, the server calculates the gradient of the difference through the back-propagation algorithm, and uses the optimizer to update the parameters of the TinyBERT model. This step is repeated to iteratively train and fine-tune the TinyBERT model to obtain the HAE rare disease risk prediction model.

[0067] Step S150, using the pre-processed medical record data of the current patient as the input of the HAE rare disease risk prediction model to output a risk prediction score corresponding to the medical record data, and the risk prediction score is used to characterize the probability of the current patient having the HAE rare disease.

[0068] Specifically, the server uses the pre-processed medical record data of the current patient as the input of the HAE rare disease risk prediction model to predict and output the risk prediction score corresponding to the current patient's medical record data to characterize the probability of the current patient having the HAE rare disease.

[0069] The above-mentioned AI-based HAE rare disease risk prediction method obtains public medical data on the Internet, and pre-processes the medical data by data cleaning and data partitioning, so as to train and fine-tune the open source Llama3 base model through the pre-processed medical data. Then extract HAE disease medical knowledge from the open source Llama3 base model, and input the HAE disease medical knowledge as training data into the fine-tuned open source Llama3 base model to output a risk probability distribution corresponding to the HAE disease medical knowledge. The training data is used as the input of the TinyBERT model to output another risk probability distribution corresponding to the HAE disease medical knowledge, and the cross entropy loss function is called to calculate the difference between the two risk probability distributions. Subsequently, the gradient of the difference is calculated by the back propagation algorithm, and the parameters of the TinyBERT model are updated using the optimizer to iteratively train and fine-tune the TinyBERT model to obtain the HAE rare disease risk prediction model. Finally, the pre-processed medical record data of the current patient is used as the input of the HAE rare disease risk prediction model to output the HAE rare disease risk prediction score corresponding to the medical record data to characterize the probability of the current patient's HAE rare disease. This method is based on AI and identifies all the patient's in-hospital medical records (from admission to discharge) to predict HAE risks. While ensuring the comprehensiveness of the screening, it is not easy to miss information due to human errors, thereby improving the screening efficiency and accuracy of rare disease risks, and has a high degree of intelligence.

[0070] See also Figure 2 and Figure 3 As shown, in a specific embodiment, the AI-based HAE rare disease risk prediction method provided by the present invention includes model training, data preprocessing, HAE rare disease model risk prediction, automatic HAE rare disease risk classification, CDSS system automatic pop-up reminder, clinician confirmation, and HAE warning patients sent for examination and diagnosis.

[0071] During the model training process, you first need to select a pre-trained large model suitable for the target task and select the large model type related to the target task. Since medical record data recognition is a natural language processing task, the open source Llama3 base model is selected after evaluation.

[0072] Subsequently, fine-tuning training of the model is performed based on the public medical data on the Internet. First, the task type needs to be determined, that is, the target task type of fine-tuning is determined to be "risk scoring for patients based on medical record information and automatic classification of risk results". Then, high-quality labeled data related to the target task is collected from the public data on the Internet according to the task requirements. Subsequently, data preprocessing is performed, including data cleaning and data partitioning. In data cleaning, noise, duplicates and useless information need to be removed to ensure that the data is clean. In data partitioning, the data set is divided into training set, validation set and test set, usually in a ratio of 80:10:10 or 70:15:15. In addition, training parameters need to be set, including optimizers. AdamW optimizer is usually used because it performs well when processing large language models. It also includes learning rate. During fine-tuning, the initial learning rate is set to 3e-5, and the learning rate scheduler (linear decay) is used to dynamically adjust the learning rate. It also includes loss function. The text generation task uses the cross entropy loss function to calculate the difference between the generated text and the target text. The classification task uses the cross entropy loss to calculate the difference between the classification result and the true label.

[0073] In this embodiment, during the fine-tuning process of the large model, it is necessary to fine-tune the model on the training set and optimize the model by adjusting hyperparameters (such as learning rate and batch size). The model performance is evaluated on the validation set, and overfitting is prevented by monitoring indicators such as loss, accuracy, and F1 score on the validation set. It is also necessary to use an early stopping strategy to stop training in advance when the performance of the validation set no longer improves to avoid overfitting of the model. When testing and evaluating the model, the performance of the fine-tuned model is evaluated on the test set to ensure the performance of the model on unseen data, and to select appropriate evaluation indicators according to the task type, such as BLEU score (for text generation), accuracy, F1 score, etc.

[0074] In this embodiment, after determining the pre-trained large model, the next step is to define the specific target task (HAE disease risk prediction) and the architecture of the small model. First, it is necessary to clarify that the target task of the small model is HAE risk prediction, and prepare the training data set and data. Secondly, it is necessary to select a simplified small model architecture. Considering that the model must be able to be deployed to the CPU server for efficient operation, the more lightweight TinyBERT model is selected.

[0075] In addition, the model training and fine-tuning process also includes knowledge distillation, which is the process of extracting knowledge from the pre-trained large model and transferring it to the small model. The core idea is to let the small model learn the output of the large model so that the small model can retain the performance of the large model at a small scale. The "soft labels" generated by the large model contain more probabilistic information than hard labels, and the small model learns by imitating these outputs.

[0076] It should be noted that the training data of the small model is the patient data set obtained based on the step data processing and its result labels (HAE disease risk score (0,1) and risk level [low risk, medium risk, high risk]).

[0077] In this embodiment, the training process of the small model learning the large model first requires the large model to generate soft labels, input the training data into the large model, and the large model predicts each input data, but not only outputs a hard label (such as a specific risk score), but also outputs an additional risk score probability distribution. This probability distribution indicates the possibility that the model believes that the input data belongs to each risk score interval. For example, for the three risk score intervals of the present invention (<0.5, [0.5, 0.8], >0.8), the large model may output [0.2, 0.6, 0.2], indicating that the model believes that the probability of the HAE risk score interval obtained based on the input data being [0.5, 0.8] is the highest (0.6), and the probability of belonging to <0.5 and >0.8 is lower. These predicted probability distributions are regarded as "soft labels". They contain more information than hard labels, reflecting the uncertainty of the model for each prediction result.

[0078] When the small model imitates learning, the soft label is used as the target. When training the small model, the original hard label is no longer used, but the soft label generated by the large model is used as the target. The cross entropy loss function is usually used in the loss function to calculate the difference between the probability distribution predicted by the small model and the soft label generated by the large model. At the same time, the gradient is calculated by the back propagation algorithm, and the optimizer (such as the Adam optimizer) is used to update the parameters of the small model so that the predicted probability distribution of the small model is as close as possible to the soft label generated by the large model. Repeat this process until the performance of the small model reaches the expected level. After knowledge distillation, the small model can be further fine-tuned to improve its performance on specific tasks, using more training data or data enhancement to improve the generalization ability of the small model.

[0079] In this embodiment, when training the small model, only the last layer is trained. Every 2 epochs, the performance is evaluated based on the validation set. If the performance is no longer significantly improved (≥0.1) after the evaluation, the previous layer is unfrozen to avoid initial overfitting. In addition, a lower learning rate is required for fine-tuning (initial learning rate: 3e-5) to avoid destroying the knowledge that the small model has learned.

[0080] To further reduce the model size and computational complexity, model compression and optimization techniques can be applied to reduce the model size and computational complexity by removing unimportant neuron connections. Criteria for judging unimportant neurons: Count the frequency of activation of each neuron on the entire training set. If the activation frequency of a neuron is low (lower than the threshold), it is considered unimportant. The threshold is calculated as the average activation frequency of each neuron. Reduce the model weights and activation values ​​from float32 to float16 to reduce the storage and computational requirements of the model.

[0081] In this embodiment, combined with Figure 3 As shown in the figure, when the patient's medical record information is obtained, the NLP technology is used to identify the patient's complete medical record information in the hospital, including text medical record information field parsing, medical entity and entity relationship identification, and the results are standardized according to national clinical standards. Among them, the medical record information is pre-processed in real time based on NLP technology, including the current medical record information and the same patient's historical medical record information, which can fully identify the patient's medical record information in the hospital and provide more complete and accurate data for the subsequent prediction of the HAE rare disease model.

[0082] In this embodiment, the small model can be used for actual risk prediction after the first step of training and verification. According to the preprocessing, the patient's current + historical medical record information is processed to obtain the data set results, and the data results are used as the input of the model to generate a risk score (such as a probability value between 0-1) to indicate the possibility of the patient's rare disease. Subsequently, the probability output by the model can be calibrated using calibration techniques (such as Platt calibration, Isotonic regression) to ensure the accuracy of the risk score. This embodiment uses Isotonic regression for calibration.

[0083] Among them, Isotonic regression is a non-parametric method that is suitable for situations where the relationship between the probability of model output and the true probability is more complicated. It does not assume a specific probability distribution form, but directly calibrates through sorted data. First, the probability values ​​of the model output are sorted, and a monotonic non-decreasing function is fitted through Isotonic regression to map the uncalibrated output probability to the true probability, which means that the calibrated probability will not be reversed (that is, a higher original score corresponds to a higher calibrated probability). The generated monotonic function is used to calibrate the model output of new data so that the output probability is more consistent with the true distribution.

[0084] In this embodiment, patients can be divided into different risk levels (such as high, medium, and low risk) according to the risk score to help doctors make decisions. By intelligently grading the patient's HAE risk results, the medical team can identify patients with high risk of rare diseases earlier, provide early intervention, and optimize treatment effects, which not only improves diagnostic efficiency, but also provides a scientific basis for personalized medicine.

[0085] Based on the risk score generated by the model, the risk level is determined, for example, the grading standards are determined as follows: high risk (>0.8), medium risk (0.5-0.8), low risk (<0.5), etc. These thresholds are determined based on clinical needs and statistical analysis. In addition, users can also be supported to optimize the balance between different levels by adjusting the grading thresholds to ensure that the number of patients and risks at each level are balanced.

[0086] In this embodiment, the HAE rare disease screening model service can be integrated into the CDSS system and embedded in the HIS page. During the doctor's normal consultation and medical record writing process, a pop-up window will be displayed to warn the patient. The risk will be graded according to the patient's actual situation, and card control reminders will be made based on different risk levels. The pop-up window supports clinicians to confirm the warning results, ignore the warning, adjust the warning level, etc. The user behavior of the pop-up page supports point embedding, which is convenient for subsequent statistical analysis. The HAE rare disease screening model is calculated in real time based on the patient's latest medical record information, and the pop-up window displays the latest warning content. The pop-up window interacts with the doctor, and the system will collect the doctor's feedback and use this feedback to optimize the model. The HAE rare disease model identifies suspected HAE patients, and a pop-up warning will be displayed on the CDSS system page. While viewing / editing the medical record, the doctor can confirm the warning patient classification details to ensure the accuracy of the patient sent for examination. The data generated by patients diagnosed through warnings can also be used as training data to continuously improve the model effect.

[0087] In this embodiment, the pre-trained domain model is used to train the HAE screening model, and rare disease screening is performed based on the patient's complete medical history information in the hospital. By analyzing a large amount of patient data, medical literature and clinical records, the characteristic patterns of rare diseases are identified, thereby improving the accuracy and speed of diagnosis. The diagnosis of rare diseases often requires the integration of multiple information such as genetic data, clinical data, and imaging data. AI can effectively integrate and analyze this data, provide comprehensive disease analysis, and thus help doctors make more accurate judgments.

[0088] like Figure 4 As shown, in one embodiment, the AI-based HAE rare disease risk prediction method provided by the present invention obtains medical data publicly available on the Internet and preprocesses the medical data to train and fine-tune the open source Llama3 base model through the preprocessed medical data, specifically including the following steps:

[0089] Step S111, obtaining the target task type of the preset open source Llama3 base model to select target medical data related to the target task type from the medical data.

[0090] Step S112, removing noise, duplicate data and useless data in the target medical data to clean the target medical data, and dividing the cleaned target medical data into a training set, a test set and a validation set.

[0091] Among them, the target task type is to give the corresponding patient a disease risk score based on the medical record data, and to grade the risk score results. The training set is used to train and fine-tune the open source Llama3 base model by fine-tuning the hyperparameters. The validation set is used to monitor and evaluate the indicator performance of the open source Llama3 base model. The test set is used to evaluate the performance of the fine-tuned open source Llama3 base model.

[0092] like Figure 5 As shown, in one embodiment, the AI-based HAE rare disease risk prediction method provided by the present invention extracts HAE disease medical knowledge from the open source Llama3 base model, and inputs the HAE disease medical knowledge as training data into the fine-tuned open source Llama3 base model to output a first risk probability distribution corresponding to the HAE disease medical knowledge, specifically comprising the following steps:

[0093] Step S121, taking the HAE disease risk prediction task as the target task of the TinyBERT model, performing knowledge distillation on the medical data in the fine-tuned open source Llama3 base model to extract the corresponding HAE disease medical knowledge.

[0094] Step S122, calling the fine-tuned open source Llama3 base model to predict HAE disease medical knowledge to output the hard label and soft label corresponding to each input data.

[0095] Among them, the hard label is the risk score result corresponding to each input data, and the soft label is the probability distribution corresponding to the risk score result. The probability distribution is used to characterize the probability that each input data belongs to the preset risk score interval.

[0096] like Figure 6 As shown, in one embodiment, the AI-based HAE rare disease risk prediction method provided by the present invention calculates the gradient of the difference through the back propagation algorithm, and uses the optimizer to update the parameters of the TinyBERT model to iteratively train and fine-tune the TinyBERT model to obtain the HAE rare disease risk prediction model, which specifically includes the following steps:

[0097] Step S141, training the last layer of the TinyBERT model, and evaluating the performance of the TinyBERT model using a validation set of HAE disease medical knowledge every two epochs.

[0098] Step S142: When the difference between the performance evaluation results is lower than the first threshold, the previous layer of the TinyBERT model is gradually unfrozen to avoid initial overfitting.

[0099] Each period is a process in which HAE disease medical knowledge undergoes one forward propagation and one backward propagation in the TinyBERT model.

[0100] like Figure 7 As shown, in one embodiment, the AI-based HAE rare disease risk prediction method provided by the present invention uses the pre-processed medical record data of the current patient as the input of the HAE rare disease risk prediction model to output the risk prediction score corresponding to the medical record data, specifically comprising the following steps:

[0101] Step S151, obtaining the current medical record data and historical medical record data corresponding to the current patient, and preprocessing the current medical record data and historical medical record data to obtain a data set result.

[0102] Step S152, calling the HAE rare disease risk prediction model to predict the data set results to output the probability value corresponding to the HAE rare disease risk corresponding to the current patient.

[0103] like Figure 8 As shown, in one embodiment, the AI-based HAE rare disease risk prediction method provided by the present invention further includes the following steps:

[0104] Step S810, sorting the probability values ​​according to their magnitude to obtain a probability value sequence, and fitting the probability value sequence to a corresponding monotone non-decreasing function through Isotonic regression to calibrate the probability values.

[0105] Step S820, based on the HAE rare disease risk prediction score, the current patient is divided into different risk levels, and the risk levels include at least a first risk level, a second risk level and a third risk level.

[0106] Among them, the first risk level is the risk level corresponding to when the HAE rare disease risk prediction score exceeds the third threshold, the second risk level is the risk level corresponding to when the HAE rare disease risk prediction score is between the third threshold and the fourth threshold, and the third risk level is the risk level corresponding to when the HAE rare disease risk prediction score is lower than the fourth threshold, and the third threshold and the fourth threshold are both preset according to clinical needs and statistics.

[0107] The AI-based HAE rare disease risk prediction device provided by the present invention is described below. The AI-based HAE rare disease risk prediction device described below and the AI-based HAE rare disease risk prediction method described above can be referenced to each other.

[0108] like Fig. 9 As shown, in one embodiment, an AI-based HAE rare disease risk prediction device includes a large model fine-tuning module 910, a knowledge distillation module 920, an output difference calculation module 930, a small model fine-tuning module 940 and an HAE rare disease risk prediction module 950.

[0109] The large model fine-tuning module 910 is used to obtain medical data publicly available on the Internet and pre-process the medical data so as to train and fine-tune the open source Llama3 base model through the pre-processed medical data. The pre-processing includes data cleaning and data partitioning.

[0110] The knowledge distillation module 920 is used to extract HAE disease medical knowledge from the open source Llama3 base model, and input the HAE disease medical knowledge as training data into the fine-tuned open source Llama3 base model to output the first risk probability distribution corresponding to the HAE disease medical knowledge.

[0111] The output difference calculation module 930 is used to use the training data as the input of the TinyBERT model to output the second risk probability distribution corresponding to the HAE disease medical knowledge, and call the cross entropy loss function to calculate the difference between the first risk probability distribution and the second risk probability distribution.

[0112] The small model fine-tuning module 940 is used to calculate the gradient of the difference through the back-propagation algorithm, and use the optimizer to update the parameters of the TinyBERT model to iteratively train and fine-tune the TinyBERT model to obtain the HAE rare disease risk prediction model.

[0113] The HAE rare disease risk prediction module 950 is used to use the pre-processed medical record data of the current patient as the input of the HAE rare disease risk prediction model to output the risk prediction score corresponding to the medical record data. The risk prediction score is used to characterize the probability of the current patient having the HAE rare disease.

[0114] In this embodiment, the AI-based HAE rare disease risk prediction device provided by the present invention, the large model fine-tuning module is specifically used for:

[0115] Get the target task type of the preset open source Llama3 base model to select target medical data related to the target task type from the medical data.

[0116] Noise, duplicate data and useless data in the target medical data are removed to clean the target medical data, and the cleaned target medical data is divided into a training set, a test set and a validation set.

[0117] Among them, the target task type is to give the corresponding patient a disease risk score based on the medical record data, and to grade the risk score results. The training set is used to train and fine-tune the open source Llama3 base model by fine-tuning the hyperparameters. The validation set is used to monitor and evaluate the indicator performance of the open source Llama3 base model. The test set is used to evaluate the performance of the fine-tuned open source Llama3 base model.

[0118] In this embodiment, the AI-based HAE rare disease risk prediction device provided by the present invention, the knowledge distillation module is specifically used for:

[0119] Taking the HAE disease risk prediction task as the target task of the TinyBERT model, knowledge distillation is performed on the medical data in the fine-tuned open source Llama3 base model to extract the corresponding HAE disease medical knowledge.

[0120] The fine-tuned open source Llama3 base model is called to predict HAE disease medical knowledge to output the hard label and soft label corresponding to each input data.

[0121] Among them, the hard label is the risk score result corresponding to each input data, and the soft label is the probability distribution corresponding to the risk score result. The probability distribution is used to characterize the probability that each input data belongs to the preset risk score interval.

[0122] In this embodiment, the HAE rare disease risk prediction device based on AI provided by the present invention, the small model fine-tuning module is specifically used for:

[0123] The last layer of the TinyBERT model is trained, and the performance of the TinyBERT model is evaluated on a validation set of HAE disease medical knowledge every two epochs.

[0124] When the difference between the performance evaluation results is lower than the first threshold, the previous layer of the TinyBERT model is gradually unfrozen to avoid initial overfitting.

[0125] Each period is a process in which HAE disease medical knowledge undergoes one forward propagation and one backward propagation in the TinyBERT model.

[0126] In this embodiment, the AI-based HAE rare disease risk prediction device provided by the present invention further includes a neuron removal module for:

[0127] The activation frequency of each neuron in the TinyBERT model in the training set is counted, and neurons whose activation frequency is lower than the second threshold are removed. The second threshold is the average activation frequency of neurons in the training set.

[0128] In this embodiment, the HAE rare disease risk prediction device based on AI provided by the present invention, the HAE rare disease risk prediction module is specifically used for:

[0129] Obtain the current medical record data and historical medical record data corresponding to the current patient, and preprocess the current medical record data and historical medical record data to obtain the data set results.

[0130] The HAE rare disease risk prediction model is called to predict the data set results to output the probability value corresponding to the HAE rare disease risk corresponding to the current patient.

[0131] In this embodiment, the AI-based HAE rare disease risk prediction device provided by the present invention further includes a probability value calibration module for:

[0132] The probability values ​​are sorted according to their sizes to obtain a probability value sequence, and the probability value sequence is fitted into a corresponding monotone non-decreasing function through Isotonic regression to calibrate the probability values.

[0133] In this embodiment, the AI-based HAE rare disease risk prediction device provided by the present invention further includes a HAE rare disease risk rating module, which is used to:

[0134] Based on the HAE rare disease risk prediction score, the current patients are divided into different risk levels, and the risk levels include at least the first risk level, the second risk level and the third risk level.

[0135] In this embodiment, the AI-based HAE rare disease risk prediction device provided by the present invention has a first risk level that corresponds to when the HAE rare disease risk prediction score exceeds the third threshold, a second risk level that corresponds to when the HAE rare disease risk prediction score is between the third threshold and the fourth threshold, and a third risk level that corresponds to when the HAE rare disease risk prediction score is lower than the fourth threshold, and both the third threshold and the fourth threshold are preset according to clinical needs and statistics.

[0136] Fig.10 An example of a physical structure diagram of an electronic device is shown. The electronic device may be a smart terminal, and its internal structure diagram may be as follows: Fig.10As shown. The electronic device includes a processor, a memory and a network interface connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above-mentioned AI-based HAE rare disease risk prediction method is implemented.

[0137] On the other hand, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned AI-based HAE rare disease risk prediction method.

[0138] In another aspect, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, the above-mentioned AI-based HAE rare disease risk prediction method is implemented.

[0139] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0140] By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0141] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for predicting the risk of HAE rare disease based on AI, characterized in that: The method comprises: Obtaining public medical data on the Internet and preprocessing the medical data to train and fine-tune the open source Llama3 base model through the preprocessed medical data, wherein the preprocessing includes data cleaning and data partitioning; Extracting HAE disease medical knowledge from the open source Llama3 base model, and inputting the HAE disease medical knowledge as training data into the fine-tuned open source Llama3 base model to output a first risk probability distribution corresponding to the HAE disease medical knowledge; Using the training data as input of the TinyBERT model to output a second risk probability distribution corresponding to the HAE disease medical knowledge, and calling a cross entropy loss function to calculate the difference between the first risk probability distribution and the second risk probability distribution; The gradient of the difference is calculated by a back-propagation algorithm, and the parameters of the TinyBERT model are updated using an optimizer to iteratively train and fine-tune the TinyBERT model to obtain a HAE rare disease risk prediction model; Using the pre-processed medical record data of the current patient as the input of the HAE rare disease risk prediction model to output the HAE rare disease risk prediction score corresponding to the medical record data, wherein the risk prediction score is used to characterize the probability of the current patient having the HAE rare disease; The obtaining of medical data publicly available on the Internet and preprocessing of the medical data to train and fine-tune the open source Llama3 base model through the preprocessed medical data includes: Acquire a preset target task type of the open source Llama3 base model to select target medical data related to the target task type from the medical data; Removing noise, duplicate data, and useless data from the target medical data to clean the target medical data, and dividing the cleaned target medical data into a training set, a test set, and a validation set; Among them, the target task type is to score the disease risk of the corresponding patient based on the medical record data, and to grade the risk score results. The training set is used to train and fine-tune the open source Llama3 base model by fine-tuning the hyperparameters. The validation set is used to monitor and evaluate the indicator performance of the open source Llama3 base model. The test set is used to evaluate the performance of the fine-tuned open source Llama3 base model.

2. The AI-based HAE rare disease risk prediction method according to claim 1, characterized in that: The extracting of HAE disease medical knowledge from the open source Llama3 base model and inputting the HAE disease medical knowledge as training data into the fine-tuned open source Llama3 base model to output a first risk probability distribution corresponding to the HAE disease medical knowledge includes: Taking the HAE disease risk prediction task as the target task of the TinyBERT model, knowledge distillation is performed on the medical data in the fine-tuned open source Llama3 base model to extract the corresponding HAE disease medical knowledge; Calling the fine-tuned open source Llama3 base model to predict the HAE disease medical knowledge to output hard labels and soft labels corresponding to each input data; The hard label is the risk score result corresponding to each input data, and the soft label is the probability distribution corresponding to the risk score result. The probability distribution is used to characterize the probability that each input data belongs to a preset risk score interval.

3. The AI-based HAE rare disease risk prediction method according to claim 2, characterized in that: The gradient of the difference is calculated by the back propagation algorithm, and the parameters of the TinyBERT model are updated using an optimizer to iteratively train and fine-tune the TinyBERT model to obtain a HAE rare disease risk prediction model, including: Training the last layer of the TinyBERT model and evaluating the performance of the TinyBERT model using a validation set of HAE disease medical knowledge every two epochs; When the difference between the performance evaluation results is lower than a first threshold, the previous layer of the TinyBERT model is gradually unfrozen to avoid initial overfitting; Each period is a process in which the HAE disease medical knowledge undergoes one forward propagation and one backward propagation in the TinyBERT model.

4. The AI-based HAE rare disease risk prediction method according to claim 3, characterized in that: The method further comprises: The activation frequency of each neuron in the TinyBERT model in the training set is counted, and neurons whose activation frequency is lower than a second threshold are removed, where the second threshold is the average activation frequency of neurons in the training set.

5. The AI-based HAE rare disease risk prediction method according to claim 4, characterized in that: The method of using the pre-processed medical record data of the current patient as the input of the HAE rare disease risk prediction model to output the HAE rare disease risk prediction score corresponding to the medical record data includes: Obtaining current medical record data and historical medical record data corresponding to the current patient, and preprocessing the current medical record data and historical medical record data to obtain a data set result; The HAE rare disease risk prediction model is called to predict the data set results to output a probability value corresponding to the HAE rare disease risk corresponding to the current patient.

6. The AI-based HAE rare disease risk prediction method according to claim 5, characterized in that: The method further comprises: The probability values ​​are sorted according to their sizes to obtain a probability value sequence, and the probability value sequence is fitted to a corresponding monotone non-decreasing function through Isotonic regression to calibrate the probability value.

7. The AI-based HAE rare disease risk prediction method according to claim 1, characterized in that: The method uses the pre-processed medical record data of the current patient as the input of the HAE rare disease risk prediction model to output the HAE rare disease risk prediction score corresponding to the medical record data, and then includes: Based on the HAE rare disease risk prediction score, the current patient is divided into different risk levels, and the risk levels include at least a first risk level, a second risk level and a third risk level.

8. The AI-based HAE rare disease risk prediction method according to claim 7, characterized in that: The first risk level is the risk level corresponding to when the HAE rare disease risk prediction score exceeds the third threshold, the second risk level is the risk level corresponding to when the HAE rare disease risk prediction score is between the third threshold and the fourth threshold, the third risk level is the risk level corresponding to when the HAE rare disease risk prediction score is lower than the fourth threshold, and the third threshold and the fourth threshold are both preset according to clinical needs and statistics.

9. An AI-based HAE rare disease risk prediction device, characterized in that: The device comprises: A large model fine-tuning module, which is used to obtain public medical data on the Internet and pre-process the medical data, so as to train and fine-tune the open source Llama3 base model through the pre-processed medical data, wherein the pre-processing includes data cleaning and data partitioning; A knowledge distillation module, for extracting HAE disease medical knowledge from the open source Llama3 base model, and inputting the HAE disease medical knowledge as training data into the fine-tuned open source Llama3 base model to output a first risk probability distribution corresponding to the HAE disease medical knowledge; An output difference calculation module is used to use the training data as an input of the TinyBERT model to output a second risk probability distribution corresponding to the HAE disease medical knowledge, and call a cross entropy loss function to calculate the difference between the first risk probability distribution and the second risk probability distribution; A small model fine-tuning module, used to calculate the gradient of the difference by a back-propagation algorithm, and to update the parameters of the TinyBERT model using an optimizer, so as to iteratively train and fine-tune the TinyBERT model to obtain a HAE rare disease risk prediction model; An HAE rare disease risk prediction module, used to use the pre-processed medical record data of the current patient as the input of the HAE rare disease risk prediction model to output a risk prediction score corresponding to the medical record data, wherein the risk prediction score is used to characterize the probability of the current patient having the HAE rare disease; The large model fine-tuning module is specifically used for: Obtain a target task type of a preset open source Llama3 base model to select target medical data related to the target task type from the medical data; Removing noise, duplicate data, and useless data from the target medical data to clean the target medical data, and dividing the cleaned target medical data into a training set, a test set, and a validation set; Among them, the target task type is to give the corresponding patient a disease risk score based on the medical record data, and to grade the risk score results. The training set is used to train and fine-tune the open source Llama3 base model by fine-tuning the hyperparameters. The validation set is used to monitor and evaluate the indicator performance of the open source Llama3 base model. The test set is used to evaluate the performance of the fine-tuned open source Llama3 base model.

Images