Medical diagnosis reasoning system

Abstract

The invention provides a medical diagnosis reasoning system comprising a receiving module used for receiving prescription information and basic information of a user; the standardization module is used for performing standardization conversion on the prescription information through a lightweight term standardization model to obtain standard terms corresponding to the prescription information; the vectorization module is used for performing vector conversion and splicing on the standard terms and the basic information to obtain comprehensive feature vectors; and the disease prediction module is used for predicting a multi-label vector corresponding to the comprehensive feature vector based on a lightweight multi-label disease prediction model, and each dimension of the multi-label vector represents the probability of a set disease corresponding to the dimension. Through knowledge distillation, the lightweight model can be deployed in a medical diagnosis reasoning system, the accuracy of the model is protected under the condition of limited training data, the data privacy risk does not exist, the real-time diagnosis requirement can be met, the reasoning speed is increased, and the multi-label disease prediction precision is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical equipment, and in particular to a medical diagnosis reasoning system. BACKGROUND

[0002] In order to improve the efficiency of medical diagnosis, relevant workers have researched a batch of medical equipment for medical diagnosis reasoning. However, the existing medical equipment has the following defects: the prescription / surgery expense list data lacks explicit diagnosis, the term expression is non-standardized, and the large language model (LLM) used by the equipment has medical knowledge but is difficult to deploy in local or edge devices due to privacy and resource constraints. Therefore, an effective solution is urgently needed to solve the above problems. SUMMARY

[0003] To solve the above problems, the present application provides a medical diagnosis reasoning system.

[0004] The present application provides a medical diagnosis reasoning system, comprising: A receiving module is configured to receive prescription information and basic information of a user, wherein the prescription information comprises at least one drug name and at least one surgical operation term, and the basic information comprises gender and age; A standardization module is configured to convert the prescription information of the prescription information into standard terms corresponding to the prescription information through a lightweight term standardization model, wherein the term standardization model is obtained based on a knowledge distillation technology; A vectorization module is configured to convert and splice the standard terms and the basic information into a comprehensive feature vector; A disease prediction module is configured to predict a multi-label vector corresponding to the comprehensive feature vector based on a lightweight multi-label disease prediction model, wherein each dimension of the multi-label vector represents the probability of a set disease corresponding to the dimension, and the multi-label disease prediction model is obtained based on the knowledge distillation technology.

[0005] According to the medical diagnosis reasoning system provided by the present application, the medical diagnosis reasoning system further comprises a first deployment module; The first deployment module is configured to receive the term standardization model sent by a first model training system, prune and quantitatively compress the term standardization model, and deploy the term standardization model in the standardization module, wherein the term standardization model comprises at least one sub-term standardization model. The first model training system is configured to obtain a non-standard clinical term sample set, the clinical term sample set including a plurality of clinical term samples, the clinical term samples being drug name samples or surgical operation term samples; determine standard term labels of the clinical term samples based on a standard term library; perform knowledge distillation training on a first student model based on the clinical term samples, the standard term labels of the clinical term samples, and a first teacher model, the first student model being a pre-trained lightweight language model, to obtain a trained sub-term standardization model.

[0006] According to the medical diagnosis reasoning system provided by the present application, the determination of the standard term labels of the clinical term samples based on the standard term library comprises: For each of the clinical term samples, the comprehensive similarity between the clinical term sample and each standard term in the standard term library is determined, and K candidate standard terms most similar to the clinical term sample are selected based on the comprehensive similarity, wherein K is a positive integer. The standard term label of the clinical term sample is selected from the K candidate standard terms corresponding to the clinical term sample and the labeled standard term corresponding to the clinical term sample by a second large language model.

[0007] According to the medical diagnosis reasoning system provided by the present application, the knowledge distillation training of the first student model based on the clinical term samples, the standard term labels of the clinical term samples, and the first teacher model to obtain the trained sub-term standardization model comprises: Any of the clinical term samples is selected from the clinical term samples, and the clinical term sample is input into the first teacher model and the first student model to obtain a first predicted standard term output by the first teacher model and a second predicted standard term output by the first student model. Based on the first predicted standard term and the second predicted standard term, a first soft label loss is calculated, and based on the second predicted standard term and the standard term label corresponding to the clinical term sample, a first hard label loss is calculated. Based on the first soft label loss, the first hard label loss, and a first distillation weight, a first total loss is calculated. Based on the first total loss, the model parameters of the first student model are adjusted. The adjusted first student model is continuously trained until a first training stop condition is reached, and the trained first student model is used as the sub-term standardization model.

[0008] According to the medical diagnosis reasoning system provided by the application, the comprehensive similarity between the clinical term sample and each standard term in the standard term library is determined, and the comprehensive similarity comprises: The clinical term sample and each standard term in the standard term library are respectively subjected to word vector coding to obtain a first word vector coding corresponding to the clinical term sample and a second word vector coding corresponding to each standard term in the standard term library respectively. The cosine similarity between the first word vector coding and each second word vector coding is calculated, and the longest common subsequence similarity between the clinical term sample and each standard term in the standard term library is calculated. For each standard term in the standard term library, the cosine similarity and the longest common subsequence similarity corresponding to the standard term are weighted and summed to obtain the comprehensive similarity between the standard term and the clinical term sample.

[0009] According to the medical diagnosis reasoning system provided by the application, the medical diagnosis reasoning system further comprises a cache module. The cache module caches at least one key-value pair, the key in the key-value pair represents a non-standard term, and the value in the key-value pair represents a verified standard term. The cache module is configured to calculate the similarity between the prescription information and the key in each key-value pair, and determine the value corresponding to the target similarity as the standard term corresponding to the prescription information in a case where the target similarity exists in the similarities, the target similarity being the similarity greater than the similarity threshold. The standardization module is specifically configured to, in a case where the target similarity does not exist in the similarities, perform standardization conversion on the prescription information of the prescription information by using a lightweight term standardization model to obtain the standard term corresponding to the prescription information. The cache module is further configured to store the standard term and the corresponding prescription information determined by the standardization module as a new key-value pair.

[0010] According to the medical diagnosis reasoning system provided by the application, the term standardization model comprises at least one sub-term standardization model. The standardization module is specifically configured to perform standardization conversion on the prescription information of the prescription information by using each sub-term standardization model to obtain at least one standby standard term corresponding to the prescription information, and filter out the standard term corresponding to the prescription information from the at least one standby standard term by using a weighted voting method.

[0011] The medical diagnosis reasoning system provided by the application comprises a vectorization module, a disease prediction module, a first deployment module and a second deployment module. The embedding layer is configured to perform embedding coding on the standard terms of each drug name to obtain drug embedding codes corresponding to the drug names, and perform embedding coding on the standard terms of each surgical operation term to obtain surgical embedding codes corresponding to the surgical operation terms. The aggregation layer is configured to perform aggregation processing on the drug embedding codes to obtain drug total feature vectors, and perform aggregation processing on the surgical embedding codes to obtain surgical total feature vectors. The normalization layer is configured to perform normalization processing on the age to obtain an age feature vector. The one-hot encoding layer is configured to perform one-hot encoding processing on the gender to obtain a gender feature vector. The splicing layer is configured to splice the drug total feature vectors, the surgical total feature vectors, the age feature vector and the gender feature vector to obtain the comprehensive feature vector.

[0012] The medical diagnosis reasoning system provided by the application further comprises a second deployment module. The second deployment module is configured to receive the multi-label disease prediction model sent by the second model training system, perform pruning and quantization compression on the multi-label disease prediction model, and deploy the multi-label disease prediction model in the disease prediction module. The second model training system is configured to take an optimized third large language model as a second teacher model, take a lightweight prediction model as a second student model, perform knowledge distillation training on the second student model based on a comprehensive feature vector sample set and the second teacher model, and obtain the trained multi-label disease prediction model.

[0013] The medical diagnosis reasoning system provided by the application performs knowledge distillation training on the second student model based on the comprehensive feature vector sample set and the second teacher model to obtain the trained multi-label disease prediction model, which comprises the following steps: Any comprehensive feature vector sample is selected from the comprehensive feature vector samples, and the comprehensive feature vector sample is input into the second teacher model and the second student model respectively to obtain a first predicted multi-label vector output by the second teacher model and a second predicted multi-label vector output by the second student model. based on the first predicted multi-label vector and the second predicted multi-label vector, a second soft label loss is calculated, and based on the second predicted multi-label vector and the multi-label vector label corresponding to the comprehensive feature vector sample, a second hard label loss is calculated; based on the second soft label loss, the second hard label loss and a second distillation weight, a second total loss is calculated; based on the second total loss, the model parameters of the second student model are adjusted; the second student model after adjustment is continuously trained until a second training stop condition is reached, and the trained second student model is used as the multi-label disease prediction model.

[0014] The application also provides a medical diagnosis reasoning method, comprising: receiving prescription information and basic information of a user, the prescription information comprising at least one drug name and at least one surgical operation term, and the basic information comprising gender and age; standardizing the prescription information of the prescription information through a lightweight term standardization model, to obtain a standard term corresponding to the prescription information, the term standardization model being trained based on a knowledge distillation technology; vector converting and splicing the standard term and the basic information to obtain a comprehensive feature vector; based on a lightweight multi-label disease prediction model, a multi-label vector corresponding to the comprehensive feature vector is predicted, each dimension of the multi-label vector representing the probability of a set disease corresponding to the dimension, the multi-label disease prediction model being trained based on the knowledge distillation technology.

[0015] The application also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the medical diagnosis reasoning method corresponding to the medical diagnosis reasoning system of any of the above when executing the computer program.

[0016] The application also provides a non-transitory computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the medical diagnosis reasoning method corresponding to the medical diagnosis reasoning system of any of the above.

[0017] The application also provides a computer program product comprising a computer program, the computer program being executed by a processor to implement the medical diagnosis reasoning method corresponding to the medical diagnosis reasoning system of any of the above.

[0018] The medical diagnosis reasoning system provided by the application comprises a receiving module, which is used for receiving prescription information and basic information of a user, the prescription information comprising at least one drug name and at least one surgical operation term, and the basic information comprising gender and age; a standardization module, which is used for standardizing and converting the prescription information of the prescription information by a lightweight term standardization model to obtain standard terms corresponding to the prescription information, the term standardization model being obtained by training based on a knowledge distillation technology; a vectorization module, which is used for vector converting and splicing the standard terms and the basic information to obtain a comprehensive feature vector; and a disease prediction module, which is used for predicting a multi-label vector corresponding to the comprehensive feature vector based on a lightweight multi-label disease prediction model, each dimension of the multi-label vector representing the probability of a set disease corresponding to the dimension, and the multi-label disease prediction model being obtained by training based on the knowledge distillation technology. By the knowledge distillation, the lightweight model can be deployed in the medical diagnosis reasoning system, the accuracy of the model can be protected in the case of limited training data, there is no data privacy risk, the real-time diagnosis demand can be met, the reasoning speed is improved, and the multi-label disease prediction precision is improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0020] Figure 1 is a structural schematic diagram of the medical diagnosis reasoning system provided by the application.

[0021] Figure 2 is a flow schematic diagram of the medical diagnosis reasoning method provided by the application.

[0022] Figure 3 is a structural schematic diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0023] In order to make the objects, technical solutions and advantages of the application clearer, the following will combine the drawings in the application to clearly and completely describe the technical solutions in the application. Obviously, the described embodiments are some of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0024] Firstly, the related contents involved in the application are briefly described.

[0025] The prescription / surgery cost list data lacks explicit diagnosis, which means that in the scenes of insurance claims, the home page of the medical record, and inpatient billing, there is a common problem of "drug / surgery record, but diagnosis missing or not standardized", which limits the accuracy of downstream compliance audit, risk control, health management and claim automation.

[0026] The term expression non-standardization means that in real-world data, drug / surgery names have aliases, abbreviations, spelling differences, and regional descriptions, which are difficult to directly match with standard term libraries.

[0027] LLM has medical knowledge, but is subject to privacy and resource constraints, which means that closed-source or large-parameter models (such as general LLM / medical LLM) are difficult to deploy on local or edge devices, and direct connection may pose a risk of data out-of-domain; end-side / private small model inference urgently needs an "performance close + controllable latency" engineering route.

[0028] In related technologies, some solutions use Bidirectional Encoder Representation from Transformers (BERT) or neural networks to standardize clinical terms, but rely on a large amount of manually annotated data; some solutions use large language models for disease prediction, such as Generative Pre-trained Transformer 4 (GPT-4) or DeepSeek models, but there is a risk of privacy leakage and high computational resource requirements; some solutions infer diagnosis based on prescription data, mainly using traditional machine learning (such as decision trees, eXtreme Gradient Boosting (XGBoost)) or neural networks (such as Recurrent Neural Network (RNN)) to build multi-label disease prediction models, with low prediction accuracy.

[0029] As can be seen, the existing technology has the following defects: the term standardization model training data is limited, prone to overfitting, and has low accuracy; the LLM model cannot be deployed locally, and there is a risk of data privacy; the inference speed is slow and cannot meet the real-time diagnosis requirements; the existing algorithm has low multi-label disease prediction accuracy.

[0030] To solve at least one of the above problems, the present application provides a medical diagnosis inference system that can be deployed locally, with high accuracy and high efficiency, which can solve the problems of data processing, model efficiency and privacy protection in term standardization and diagnosis prediction.

[0031] The medical diagnosis inference system of the present application will be described below. Figures 1-3 ​

[0032] Figure 1 is a structural schematic diagram of a medical diagnosis reasoning system provided by the present application, as shown in the figure, the system comprises: Figure 1 a receiving module 101 for receiving prescription information and basic information of a user, the prescription information comprising at least one drug name and at least one surgical operation term, and the basic information comprising gender and age; a standardization module 102 for standardizing the prescription information of the prescription information by a lightweight term standardization model, to obtain a standard term corresponding to the prescription information, the term standardization model being trained based on a knowledge distillation technology; a vectorization module 103 for vectorizing and splicing the standard term and the basic information to obtain a comprehensive feature vector; a disease prediction module 104 for predicting a multi-label vector corresponding to the comprehensive feature vector based on a lightweight multi-label disease prediction model, each dimension of the multi-label vector representing the probability of a set disease corresponding to the dimension, the multi-label disease prediction model being trained based on the knowledge distillation technology.

[0033] Specifically, the received input includes original prescription data (prescription information) and demographic data (basic information) of a user (such as a patient). Among them, the prescription information includes at least one drug name and at least one surgical operation term, and the basic information includes the gender and age of the user.

[0034] Then, the term standardization is performed: all drug names and surgical operation terms are converted into standard terms by using a lightweight term standardization model. Among them, the lightweight term standardization model is a lightweight model that is trained by a knowledge distillation technology, can reproduce the diagnosis reasoning ability of a large model, and is deployed locally.

[0035] Then, the feature engineering and vectorization are performed: the standardized drug sequence (standard terms corresponding to at least one drug name) and the standardized surgical sequence (standard terms corresponding to at least one surgical operation term) are respectively embedded and encoded, and a fixed-length feature vector is obtained through an aggregation operation (such as average pooling); the demographic data (gender, age) is normalized and one-hot encoded to convert into a numerical vector; and then all the feature vectors and numerical vectors are spliced to form a final comprehensive feature vector.

[0036] ​Multi-label disease prediction: The comprehensive feature vector is input into a lightweight prediction model, namely the multi-label disease prediction model. The multi-label disease prediction model processes the data and outputs a multi-label vector, where each dimension represents the probability of a given disease. The multi-label disease prediction model can be a combination of a BERT model and a multi-layer perceptron (MLP). While the multi-label disease prediction model can be a BERT model, it can also be other models; this invention does not impose any limitations on it.

[0037] The medical diagnostic reasoning system provided by this invention includes a receiving module for receiving user prescription information and basic information. The prescription information includes at least one drug name and at least one surgical procedure terminology. The basic information includes gender and age. A standardization module is used to standardize the prescription information using a lightweight terminology standardization model, which is trained based on knowledge distillation technology. A vectorization module performs vector transformation and concatenation on the standard terms and basic information to obtain a comprehensive feature vector. A disease prediction module uses a lightweight multi-label disease prediction model to predict the multi-label vector corresponding to the comprehensive feature vector. Each dimension of the multi-label vector represents the probability of a given disease corresponding to that dimension. The multi-label disease prediction model is also trained based on knowledge distillation technology. This invention, through knowledge distillation, allows for the deployment of lightweight models in the medical diagnostic reasoning system, protecting model accuracy even with limited training data, eliminating data privacy risks, meeting real-time diagnostic needs, improving reasoning speed, and enhancing the accuracy of multi-label disease prediction.

[0038] Optionally, the medical diagnostic reasoning system further includes a first deployment module; The first deployment module is used to receive the terminology standardization model sent by the first model training system, and after pruning and quantizing the terminology standardization model, deploy it in the standardization module. The terminology standardization model includes at least one sub-terminology standardization model. The first model training system is used to acquire a non-standard clinical terminology sample set, which includes multiple clinical terminology samples, such as drug name samples or surgical procedure terminology samples; determine standard terminology labels for each of the clinical terminology samples based on a standard terminology database; use a first large language model as the first teacher model and a pre-trained lightweight language model as the first student model; and perform knowledge distillation training on the first student model based on each of the clinical terminology samples, the standard terminology labels of each of the clinical terminology samples, and the first teacher model to obtain the trained sub-terminology standardization model.

[0039] Specifically, the standard terminology database can be at least one of the International Classification of Diseases (ICD) 9th or 10th editions (ICD-9 / 10) and a pharmacopoeia.

[0040] Specifically, the first model training system can be deployed locally or in other locations.

[0041] In practical applications, the first model training system is used to train the sub-term standardization model.

[0042] First, input multiple non-standard original clinical terms (clinical terminology samples), such as the surgical name "minimally invasive pectus excavatum correction surgery".

[0043] Furthermore, for each clinical term sample, standard term tags corresponding to the clinical term sample are selected based on a standard terminology database (such as ICD-9).

[0044] Then, knowledge distillation training is performed: The teacher model uses a first-level LLM (such as GPT-4, DeepSeek) to process clinical terminology samples and generate soft labels (probability distributions). The student model can use a pre-trained language model, such as a variant of BERT. Furthermore, the student model can be trained using standard terminology labels as hard labels, combined with clinical terminology samples and soft labels generated by the teacher model, to obtain a sub-terminology standardization model.

[0045] The training objective can be that the student model not only learns to predict true hard labels (standard term labels), but also learns to mimic the soft label distribution of the teacher model through soft loss (the loss corresponding to soft labels), such as minimizing the relative entropy (Kullback-Leibler, KL) divergence, thereby absorbing its implicit semantic knowledge.

[0046] After training each sub-term standardization model, the first model training system aggregates the sub-term standardization models to form a term standardization model, which is then sent to the first deployment module.

[0047] The first deployment module receives the terminology standardization model and deploys it in a lightweight manner: it compresses the size of the terminology standardization model through pruning and quantization techniques to adapt to the real-time inference requirements of the local central processing unit (CPU) environment.

[0048] In this embodiment of the invention, the first model training system trains a lightweight terminology standardization model using knowledge distillation technology. Then, the first deployment module deploys the pruned and quantized terminology standardization model onto the medical diagnostic reasoning system. This ensures that the medical diagnostic reasoning system can perform terminology standardization while avoiding the risks of large language models being unable to be deployed locally and data leakage. Furthermore, by aggregating sub-terminology standardization models to form the terminology standardization model, the accuracy and robustness of the terminology standardization model can be guaranteed.

[0049] Optionally, determining the standard term tags for each of the clinical term samples based on a standard terminology database includes: For each clinical term sample, the comprehensive similarity between the clinical term sample and each standard term in the standard term library is determined, and the K candidate standard terms most similar to the clinical term sample are selected based on the comprehensive similarity, where K is a positive integer; Using the second language model, standard term labels for the clinical term samples are selected from the K candidate standard terms corresponding to the clinical term samples and the labeled standard terms corresponding to the clinical term samples.

[0050] Specifically, based on the obtained clinical terminology sample set, a candidate set is generated: for each clinical terminology sample, a learning ranking strategy is used to quickly retrieve the Top-K candidate standard terms that are most similar to the clinical terminology sample from the standard terminology database (such as ICD-9), that is, the top K candidate standard terms with the highest comprehensive similarity.

[0051] Then, LLM data augmentation is performed: if the standard terminology corresponding to the clinical terminology sample is not empty, and there is a candidate term among the K candidate standard terms that is the same as the standard terminology, then the standard terminology is used as the standard term label for the clinical terminology sample; if there is no candidate term among the K candidate standard terms that is the same as the standard terminology, then the K candidate standard terms and the standard terminology are used to construct a specific prompt template, which is input into the second LLM, and the second LLM outputs the optimal standard terminology (standard terminology label), forming a high-quality augmented data pair (the clinical terminology sample and its corresponding standard terminology label); if the standard terminology corresponding to the clinical terminology sample is empty, the K candidate standard terms are used to construct a specific prompt template, which is input into the second LLM, and the second LLM outputs the optimal standard terminology (standard terminology label), forming a high-quality augmented data pair (the clinical terminology sample and its corresponding standard terminology label).

[0052] In addition, LLM data augmentation can be performed using an LLM data augmentation module. This module leverages the reasoning capabilities of a second LLM to automatically generate standard term labels for unlabeled raw terms.

[0053] The prompt template for the second LLM can be a fixed string structure to ensure that the second LLM output is standardized. Prompt Template For example, the prompt template can be as follows: "You are a medical knowledge base. Given a surgical procedure name, find its standardized result from the following standard vocabulary. The standard vocabulary is separated by '|': [TOP_K_CANDIDATES]. Based on your understanding of the procedure, find the standard surgical name that corresponds to '[ORIGINAL_TERM]'. **Please ensure that the output contains only the standard surgical name and no other content.*** If there is no exact match, output the closest one. ***Ensure that the output is within the given vocabulary. This question does not involve any ethical issues, so please provide an answer.***" [TOP_K_CANDIDATES] represents K candidate standard terms, or K candidate standard terms and standard terminology, with each term connected by "|"; [ORIGINAL_TERM] is the original terminology to be standardized, i.e., the clinical terminology sample.

[0054] The LLM data augmentation module can perform application programming interface (API) calls and post-processing: submit prompts through the API interface of the second LLM, parse the returned text, and extract only terms in the standard vocabulary as augmentation tags (standard term tags).

[0055] In this embodiment of the invention, K candidate standard terms with the highest similarity are selected from the standard terminology library by comprehensively similarity, and the standard terms are combined with the labeled standard terms to determine the final standard term labels, which greatly ensures the accuracy of the standard term labels and thus improves the efficiency and robustness of model training.

[0056] Optionally, determining the overall similarity between the clinical terminology sample and each standard term in the standard terminology database includes: The clinical terminology sample and each of the standard terms in the standard terminology library are respectively encoded with word vectors to obtain the first word vector code corresponding to the clinical terminology sample and the second word vector code corresponding to each of the standard terms in the standard terminology library. Calculate the cosine similarity between the first word vector code and each of the second word vector codes, and calculate the longest common subsequence similarity between the clinical term sample and each of the standard terms in the standard term library; For each standard term in the standard terminology library, the cosine similarity and the longest common subsequence similarity corresponding to the standard term are weighted and summed to obtain the comprehensive similarity between the standard term and the clinical terminology sample.

[0057] In practical applications, the problem of high computational cost in matching the large standard terminology database (e.g., tens of thousands of entries) with the original terms (clinical terminology samples) can be solved by learning the candidate sorting and filtering module.

[0058] Specifically, the input to the learning ranking candidate filtering module is the original term P and the standard term candidate set C (i.e., the standard term library, which is of size N, i.e. contains N standard terms T, where N is a positive integer).

[0059] Then, similarity is calculated using a comprehensive similarity algorithm. The formula is: Comprehensive similarity = α * cosine similarity of word vectors + β * longest common subsequence similarity.

[0060] Wherein, α and β are weighting coefficients, determined by grid search, with the preferred range being α ∈ [0.5, 0.7], β ∈ [0.3, 0.5], and α + β = 1.

[0061] Specifically, word vector cosine similarity is the cosine similarity between word vector codes. The determination process is as follows: using pre-trained word vectors from Word2Vec or BERT, each standard term in the clinical term sample and the standard term database is encoded into vectors to obtain the first word vector code of the clinical term sample and the second word vector code of each standard term. Then, the cosine similarity between the first word vector code and each second word vector code is calculated.

[0062] Specifically, the formula for calculating the similarity of the Longest Common Subsequence (LCS) is: Longest Common Subsequence Similarity = length(LCS(P, T)) / max(length(P), length(T)), where length represents the length.

[0063] The output of the learning ranking candidate filtering module is: sorted in descending order according to the comprehensive similarity score, and the top-K candidate standard terms are selected. K is an adjustable parameter, preferably in the range of [10, 100], and preferably K=50.

[0064] In this embodiment of the invention, the comprehensive similarity is determined from two levels: cosine similarity and longest common subsequence similarity, which is beneficial to the reliability and accuracy of the comprehensive similarity.

[0065] Optionally, the step of performing knowledge distillation training on the first student model based on each of the clinical terminology samples, the standard terminology labels of each of the clinical terminology samples, and the first teacher model to obtain the trained sub-terminology standardization model includes: From each of the clinical terminology samples, select any one of the clinical terminology samples, and input the clinical terminology sample into the first teacher model and the first student model respectively to obtain the first predicted standard term output by the first teacher model and the second predicted standard term output by the first student model; Based on the first and second predicted standard terms, calculate the first soft label loss, and based on the second predicted standard terms and the standard term labels corresponding to the clinical term samples, calculate the first hard label loss. The first total loss is calculated based on the first soft label loss, the first hard label loss, and the first distillation weight; Based on the first total loss, the model parameters of the first student model are adjusted; The adjusted first student model is trained until the first training stop condition is met, and the trained first student model is used as the sub-term standardized model.

[0066] Specifically, the sub-term standardization model can be trained through the first knowledge distillation training module.

[0067] In the first knowledge distillation training module, the first student model can use the BERT architecture or other lightweight models. The input consists of term pairs [CLS], original terms [SEP], and standard candidate terms [SEP]. The output of BERT is passed through a multilayer perceptron (MLP) classification head, which outputs the matching probability.

[0068] The first total loss is a weighted average of the first hard label loss and the first soft label loss (distillation loss).

[0069] Specifically: First total loss = L_hard + λ1 * L_soft.

[0070] The first hard label loss (L_hard) can be the standard binary cross-entropy loss. L_hard = - [y_true * log(y_pred) + (1 - y_true) * log(1 - y_pred)], where y_true is the standard term label and y_pred is the second predicted standard term.

[0071] The distillation loss (L_soft) can be the KL divergence, used to measure the difference between the output probability distribution P_s of the first student model and the output probability distribution P_t of the first teacher model. L_soft = KL(P_t || P_s) = Σ P_t * log(P_t / P_s), where the first distillation weight λ1 is used to control the degree of influence of the teacher's knowledge and is a hyperparameter. The first distillation weight λ1 can be set according to the requirements, such as the preferred range λ1 ∈ [0.5, 2.0].

[0072] In this embodiment of the invention, the sub-term normalization model is trained by jointly using the first soft label loss and the first hard label loss, which helps to improve the robustness and training efficiency of the sub-term normalization model.

[0073] Optionally, the medical diagnostic reasoning system further includes a caching module; The caching module caches at least one key-value pair, where the key in the key-value pair represents a non-standard term and the value in the key-value pair represents a verified standard term. The caching module is used to calculate the similarity between the prescription information and the keys in each key-value pair; if a target similarity exists among the similarities, the value corresponding to the target similarity is determined as the standard term corresponding to the prescription information, wherein the target similarity is the similarity greater than a similarity threshold; The standardization module is specifically used to standardize the prescription information of the prescription information by means of a lightweight terminology standardization model when the target similarity does not exist in the various similarities, so as to obtain the standard terms corresponding to the prescription information.

[0074] In practical applications, the caching module maintains a key-value database, where the key is the hash value of the original term (non-standard term) and the value is the verified standard term (the standard term corresponding to the non-standard term).

[0075] When new prescription information is input, the caching module first calculates the similarity between the drug name or surgical procedure terminology in the prescription information and all keys in the cache. If the maximum similarity exceeds a threshold (e.g., 0.95), the corresponding standard terminology is directly added to the candidate list, or even used directly as the final result, i.e., the labeled terminology corresponding to the drug name or surgical procedure terminology. If the maximum similarity does not exceed the threshold, the standardization module is called to standardize the prescription information using a lightweight terminology standardization model to obtain the standard terminology corresponding to the prescription information. In this way, frequent use of the terminology standardization model for data processing can be avoided, thereby reducing the amount of data processing to a certain extent and improving the efficiency of determining standard terms.

[0076] It should be noted that the clinical terminology samples used in training the terminology standardization model and their corresponding standard terminology labels can be stored as key-value pairs in the cache module.

[0077] Optionally, the caching module is further configured to store the standard terms determined by the standardization module and their corresponding prescription information as new key-value pairs.

[0078] Specifically, the standardization module uses a lightweight terminology standardization model to standardize and transform prescription information, obtaining standard terms corresponding to the prescription information. The prescription information and its corresponding standard terms can then be stored as new key-value pairs. Continuously updating the cache in this way ensures the completeness of the cached content and directly improves the recall and accuracy of high-frequency terms by leveraging existing knowledge.

[0079] Optionally, the terminology standardization model includes at least one sub-terminology standardization model; The standardization module is specifically used to standardize and transform the prescription information of the prescription information through each of the sub-term standardization models to obtain at least one alternative standard term corresponding to the prescription information; and to select the standard term corresponding to the prescription information from the at least one alternative standard term using a weighted voting method.

[0080] In practical applications, the terminology standardization model integrates multiple sub-terminology standardization models trained on different data subsets, and uses a voting mechanism to predict the final standard terms, thereby improving accuracy and robustness.

[0081] Specifically, multiple sub-terminology standardization models are trained (e.g., using different BERT pre-trained weights, or trained on different training / validation data splits). During inference, each sub-terminology standardization model predicts the standard term corresponding to the prescription information, obtaining at least one alternative standard term. Further, a weighted voting method is employed, where each sub-terminology standardization model votes once for the best alternative standard term it deems the best among the at least one alternative standard term. The alternative standard term with the most votes is ultimately adopted as the standard term corresponding to the prescription information. This ensures the accuracy and reliability of the standard terminology.

[0082] Optionally, the vectorization module includes an embedding layer, an aggregation layer, a normalization layer, a one-hot encoding layer, and a splicing layer; The embedding layer is used to embed and encode the standard terms of each drug name to obtain the drug embedding code corresponding to each drug name, and to embed and encode the standard terms of each surgical operation term to obtain the surgical embedding code corresponding to each surgical operation term. The aggregation layer is used to aggregate the drug embedding codes to obtain a total drug feature vector, and to aggregate the surgical embedding codes to obtain a total surgical feature vector. The normalization layer is used to normalize the age to obtain an age feature vector; The one-hot encoding layer is used to perform one-hot encoding on the gender to obtain a gender feature vector; The splicing layer is used to splice the total feature vector of the drug, the total feature vector of the surgery, the age feature vector, and the gender feature vector to obtain the comprehensive feature vector.

[0083] Specifically, the vectorization module first processes the drug / surgical terminology sequence: it first inputs a list of all standard drug terms (standard terms for each drug name) [M1, M2, ..., Mm] and a list of standard surgical terms (standard terms for each surgical procedure) [S1, S2, ..., Sn] from the prescription information.

[0084] Then the embedding layer learns a d-dimensional embedding vector (where d=128 or 256) for each standard term, which yields the drug embedding code for each drug name and the surgical embedding code for each surgical procedure term. This embedding layer can be randomly initialized during training or initialized using pre-trained biomedical word vectors.

[0085] Then, sequence aggregation is performed. The aggregation layer performs average pooling on all embedding vectors of the drug sequence (drug embedding codes corresponding to each drug name) to obtain a d-dimensional total drug feature vector V_drug. Similarly, the total surgical feature vector V_procedure is obtained.

[0086] Next, demographic features (basic information) are processed: the age is normalized by the normalization layer, i.e., age_norm = (age - min_age) / (max_age - min_age), where age_norm is the normalized age, i.e., the age feature vector, min_age is the minimum age, max_age is the maximum age, and age is the age in the basic information; the one-hot coding layer processes the gender using one-hot coding to obtain the gender feature vector, such as [1, 0] for male and [0, 1] for female.

[0087] Finally, the concatenation layer concatenates the age feature vector and the gender feature vector to obtain the demographic feature vector V_demo. Further, V_drug, V_procedure, and V_demo are concatenated to form the final comprehensive feature vector V_final. This ensures the uniformity and accuracy of the comprehensive feature vector format, thereby improving the efficiency of medical diagnostic reasoning.

[0088] For example, if V_drug and V_procedure are both 128-dimensional and V_demo is 3-dimensional (age 1-dimensional + gender 2-dimensional), then V_final is 259-dimensional.

[0089] Optionally, the medical diagnostic reasoning system further includes a second deployment module; The second deployment module is used to receive the multi-label disease prediction model sent by the second model training system, and deploy the multi-label disease prediction model in the disease prediction module after pruning and quantizing and compressing it. The second model training system is used to train the second student model by knowledge distillation based on a optimized third language model as the second teacher model and a lightweight prediction model as the second student model, using a comprehensive feature vector sample set and the second teacher model to obtain the trained multi-label disease prediction model. The comprehensive feature vector sample set includes multiple comprehensive feature vector samples carrying multi-label vector labels.

[0090] Specifically, the second model training system can be deployed locally or in other locations.

[0091] In practical applications, the second model training system is used to train a multi-label disease prediction model using knowledge distillation technology. A third LLM is used as the second teacher model to generate soft labels on the diagnostic task and guide the training of the second student model to improve its performance.

[0092] The lightweight prediction model can be a BERT-based model, or other transformer models or deep learning models. It concatenates all standardized patient terms using [SEP] and uses this concatenation with demographic information as input to BERT; the corresponding output vector is labeled with [CLS] and then input into an MLP classifier. Another lightweight prediction model is an MLP-based model, which is even lighter and can directly input the comprehensive feature vector V_final into an MLP network.

[0093] The MLP structure contains multiple fully connected layers, with ReLU activation function used in the middle and Sigmoid activation function used in the output layer.

[0094] For example, the structure of MLP is: input layer -> 512-dimensional fully connected layer -> ReLU -> Dropout(0.3) -> 256-dimensional fully connected layer -> ReLU -> Dropout(0.2) -> output layer.

[0095] After training the multi-label disease prediction model, the second model training system sends the multi-label disease prediction model to the second deployment module.

[0096] The second deployment module receives the multi-label disease prediction model and performs lightweight deployment: it compresses the size of the multi-label disease prediction model through pruning and quantization techniques to adapt to the real-time inference requirements of the local CPU environment.

[0097] Furthermore, the first model training system and the second model training system can be the same system or different model training systems. In this embodiment of the invention, the second model training system trains a lightweight multi-label disease prediction model using knowledge distillation technology. Then, the deployment module (as shown in the figure) deploys the pruned and quantized multi-label disease prediction model into the medical diagnostic inference system. This ensures that the medical diagnostic inference system can perform multi-label disease prediction while avoiding the risks of large language models being unable to be deployed locally and data leakage. Furthermore, by aggregating sub-term standardization models to form a term standardization model, the accuracy and robustness of the term standardization model can be guaranteed.

[0098] Optionally, the step of training the second student model through knowledge distillation based on the comprehensive feature vector sample set and the second teacher model to obtain the trained multi-label disease prediction model includes: From each of the comprehensive feature vector samples, select any one of the comprehensive feature vector samples, and input the comprehensive feature vector sample into the second teacher model and the second student model respectively to obtain the first predicted multi-label vector output by the second teacher model and the second predicted multi-label vector output by the second student model; Based on the first predicted multi-label vector and the second predicted multi-label vector, calculate the second soft label loss, and based on the second predicted multi-label vector and the multi-label vector label corresponding to the comprehensive feature vector sample, calculate the second hard label loss. The second total loss is calculated based on the second soft label loss, the second hard label loss, and the second distillation weight; Based on the second total loss, the model parameters of the second student model are adjusted; The adjusted second student model is trained until the second training stopping condition is met, and the trained second student model is used as the multi-label disease prediction model.

[0099] Specifically, the second teacher model can be an LLM (such as GPT-4) optimized for prompts in medical question answering or diagnostic tasks. The prompt template contains all standardized information and demographic characteristics of the patient, requiring the output to show the probability of all possible diseases.

[0100] In practical applications, the knowledge distillation process is as follows: The comprehensive feature vector samples are input into the second teacher model to generate a soft probability vector, i.e., the first predicted multi-label vector T = [t1, t2, ..., t]. k ], where Σ t j = 1, 1 to k are the indices of each defined disease, j∈[1,k], t1 to t k These represent the first probability of each defined disease.

[0101] The comprehensive feature vector samples are input into the second student model, which outputs a probability vector, i.e., the second predicted multi-label vector S = [s1, s2, ..., s]. k ], where Σ s j = 1, 1 to k are the indices of each defined disease, j∈[1,k], s1 to s k These represent the second probability of each defined disease.

[0102] The training loss of the second student model is: L_total = L_bce(Y_true, S) + λ2 * L_kl(T,S). Where L_total is the second total loss; Y_true is the multi-label vector label; L_kl is the second soft label loss, which can be KL divergence loss, allowing the output distribution S of the second student model to approximate the distribution T of the second teacher model; L_bce is the second soft label loss, which can be the standard binary cross-entropy loss; λ2 is the second distillation weight, which can be set according to requirements, such as a preferred range of [0.5, 1.5].

[0103] Specifically, the loss function can be a weighted binary cross-entropy loss to handle the disease class imbalance problem, i.e., L = -1 / N Σ_i Σ_j [ w_j * Y_ij * log(P_ij) + (1 - Y_ij) * log(1 - P_ij) ], where N is the number of samples in the composite feature vector; i is the i-th composite feature vector sample; w_j is the weight of the j-th disease class, usually set as the reciprocal of the frequency of that class; Y_ij is a binary index (0 or 1) used to represent the presence of the j-th disease in the i-th composite feature vector sample; and P_ij is the predicted probability of having the j-th disease in the i-th composite feature vector sample.

[0104] This ensures the robustness and accuracy of multi-label disease prediction models.

[0105] For example, the medical diagnostic reasoning system provided by this invention designs a complete end-to-end processing flow, from the input of raw non-standard prescription data to the final multi-label disease probability output, realizing automated diagnostic reasoning. This includes: LLM data augmentation: using prompt templates to guide LLM to generate standardized terms, enhancing training data; knowledge distillation: transferring the soft objective of LLM to the second student model through KL divergence; LTR optimization: reducing the computational load of the second student model through similarity calculation and Top-K selection; historical caching mechanism: improving recall by mapping validated terms; model ensemble: integrating multiple second student models through voting to reduce bias; and localized deployment: supporting CPU parallel computing and ensuring data privacy.

[0106] Specifically, in the standardization stage, an LTR strategy combining a comprehensive similarity algorithm and Top-K selection is adopted, significantly reducing the matching computation of BERT. A specific prompt template is designed, transforming the open-domain generation task of LLM into a closed-domain selection task, ensuring the quality and controllability of data augmentation. A hybrid loss function fusing hard and soft labels is constructed, effectively distilling the generalized knowledge of LLM into a lightweight BERT model. A historical mapping caching mechanism is introduced, directly improving the recall and accuracy of high-frequency terms using existing knowledge. A multi-model voting ensemble method is adopted to effectively overcome the overfitting and bias of a single model, improving system robustness. In the prediction stage, a comprehensive feature vector fusing standardized term sequences and demographic information is constructed to comprehensively describe the patient's medical condition. A lightweight model architecture (BERT+MLP or pure MLP) for multi-label diagnostic prediction is proposed, balancing performance and efficiency. Knowledge distillation is introduced for the prediction task, transferring the advanced capabilities of LLM in diagnostic reasoning to a locally deployable small model.

[0107] The medical diagnostic inference system provided by this invention was validated, and the results show that: in terms of standardized accuracy, this invention improves from approximately 80% of the single BERT to 94.27% after integration; in terms of prediction performance, the F1 score improves from 0.4547 to 0.7321, with an accuracy of 87.54%; in terms of inference speed, the average response time on a 32-core CPU is 0.678 seconds, meeting real-time requirements; compared with other solutions, this invention achieves F1 scores of 0.7954 and 0.8804 in expert comparison and proof of concept (POC) datasets, respectively.

[0108] The medical diagnostic reasoning system provided by the present invention will be further described below with reference to specific embodiments.

[0109] Example 1: A complete standardized process for complex surgical terms.

[0110] I. Input: Original surgical terminology P = “Nuss minimally invasive pectus excavatum correction surgery”.

[0111] II. Candidate Set Generation (LTR Module): 1. Standard Terminology Database: ICD-9-PCS Surgical Terminology Collection.

[0112] 2. Similarity calculation: Use a comprehensive similarity algorithm (α=0.6, β=0.4).

[0113] (1) Calculate the similarity between P and all candidates in the standard library.

[0114] (2) Top-5 output (example): "Thoracic deformity correction surgery": Overall similarity = 0.76; "Pectus excavatum correction surgery": Overall similarity = 0.76; "Chest wall reconstruction": Overall similarity = 0.67; "Sternal rotation surgery": Overall similarity = 0.62; "Pectus carinatum correction surgery": Overall similarity = 0.57.

[0115] III. LLM Data Augmentation and Adjudication: 1. Confirmation word construction: "You are a medical knowledge base... Standard glossary: ​​Chest deformity correction surgery | Pectus excavatum correction surgery | Chest wall reconstruction surgery | Sternal rotation surgery | Pectus carinatum correction surgery. Please find the standard surgical name corresponding to Nuss minimally invasive pectus excavatum correction surgery... " 2. Call the GPT-4 API: Input: The above prompt words.

[0116] Output: "Pectus excavatum correction surgery".

[0117] 3. Results: A standardized mapping was generated (Nuss minimally invasive pectus excavatum correction, pectus excavatum correction).

[0118] IV. Knowledge Distillation Exercise (using the above data): 1. Treat (P, “pectus excavatum correction”) as a positive sample and (P, other candidates in the Top-5) as a negative sample.

[0119] 2. The teacher model (GPT-4) has a matching probability of 0.99 for positive samples.

[0120] 3. The student model (BERT) learns to mimic this probability distribution.

[0121] Through the process described in this example, the system successfully and accurately standardized a surgical term that is specific and reflects the doctor's personal habits into the standard terminology of the authoritative classification, laying a solid foundation for subsequent data processing and analysis.

[0122] Example 2: Utilizing a historical caching mechanism to rapidly standardize high-frequency drug terms.

[0123] I. Input: Original drug term P = "Adalat".

[0124] II. Historical Cache Query: (1) Query the historical cache database for keys similar to P.

[0125] (2) The key “Adalat (30mg)” was found to have an LCS similarity of 0.92 with P (exceeding the threshold of 0.85).

[0126] (3) The standard term corresponding to this key is “nifedipine controlled-release tablets”.

[0127] III. System Decision-Making: (1) Due to the cache hit and extremely high similarity, the system bypasses the time-consuming LTR full-database search and BERT fine matching.

[0128] (2) Directly output standardized results: "Nifedipine controlled-release tablets".

[0129] IV. Performance Comparison: (1) No cache used: It is necessary to calculate the similarity with tens of thousands of standard drug terms and then match them with BERT, which takes about 400ms.

[0130] (2) After using caching: only one cache query and similarity comparison are needed, taking less than 10ms.

[0131] This example demonstrates that the historical caching mechanism has an extreme acceleration effect on the standardization of high-frequency, highly consistent terms such as product names and common names, significantly improving the overall system throughput.

[0132] Example 3: Standardizing ambiguous terms through model integration.

[0133] I. Input: Original surgical term P = “iliac bone graft” (this term may be used for bone grafting in spinal fusion or for reconstruction after bone tumor resection).

[0134] II. LTR Top-3 Output: 1. "Iliac bone grafting": Similarity = 0.85; 2. "Iliac bone lesion resection": Similarity = 0.82; 3. “Partial iliac crest resection”: Similarity = 0.80; III. Multi-model ensemble prediction: 1. Input P into the three ensemble BERT models by combining it with the three candidate term pairs mentioned above.

[0135] 2. Probabilities predicted by each model: (1) For the candidate "iliac bone grafting": Model A (trained on orthopedic data): 0.95 (votes); Model B (trained on oncology data): 0.45; Model C (General Model): 0.88 (votes); (2) For the candidate "iliac bone lesion resection": Model A: 0.30; Model B: 0.90 (votes); Model C: 0.40; (3) For the candidate "partial iliac crest resection": Model A: 0.60; Model B: 0.75; Model C: 0.55; IV. Voting Statistics and Final Output: "Iliac bone grafting" received 2 votes. "Iliac bone lesion resection" received 1 vote. "Partial iliac crest resection" received 0 votes. The final standardized output is: "Iliac bone graft".

[0136] This example demonstrates that for ambiguous terms with multiple semantic layers, a single model may be affected by biases in the distribution of its training data. Model ensemble mechanisms, however, can integrate knowledge from different professional perspectives to select the most universal and reasonable standard terms in most cases, significantly improving the robustness and accuracy of the system.

[0137] Example 4: Prediction of cardiovascular disease and hyperlipidemia in middle-aged men.

[0138] I. Input Data: Drugs (original): ["Atorvastatin Calcium Tablets", "Aspirin", "Isosorbide Mononitrate Extended-Release Tablets"]; Surgery: ["Coronary angiography", "Percutaneous coronary intervention"]; Demographics: Gender = Male, Age = 55.

[0139] II. Terminology Standardization (using the Patent System): Drug standardization: [“Atorvastatin”, “Aspirin”, “Isosorbide Mononitrate”]; Surgical standardization: [“Coronary Arteriography”, “Percutaneous Coronary Intervention”]; III. Feature Vector Construction: Drug vector: The average of the embedding vectors (each 128-dimensional) of the three standardized drug names is used to obtain V_drug (128-dimensional).

[0140] Surgical vector: The average of the embedding vectors of two standardized surgical names is used to obtain V_procedure (128-dimensional).

[0141] Demographic vector: Gender male -> [1,0], age normalized (55-0) / (100-0)=0.55, concatenated to get V_demo = [1, 0, 0.55] (3-dimensional).

[0142] Combined feature vector: concatenated V_final = [V_drug, V_procedure, V_demo] (128+128+3=259 dimensions).

[0143] IV. Model Prediction: (1) Input V_final into the trained MLP prediction model.

[0144] (2) Model output (disease probability): Ischemic heart disease: 0.96; Hyperlipidemia: 0.89; High blood pressure: 0.45; Diabetes: 0.12; ... (The probability of other diseases is less than 0.1).

[0145] (3) Diagnostic results: The threshold of 0.5 was set to predict that the patient had ischemic heart disease and hyperlipidemia.

[0146] In this implementation, the model accurately captured the strong association between statins (lipid-lowering drugs), aspirin, nitrates (for angina pectoris), and the disease, and made correct inferences.

[0147] Example 5: Using DeepSeek teacher model distillation to train the prediction model.

[0148] Task 1: Train student models to accurately predict type 2 diabetes and its complications.

[0149] II. Soft target generation using the teacher model (DeepSeek): (1) Input: Standardized characteristics of a patient (drug: ["Metformin", "Gliclazide"], surgery: [], demographics: female, 68 years old).

[0150] (2) Prompt Project: "Given patient medication history: Metformin, Gliclazide. No surgery. Female, 68 years old. Please infer the probability of her having the following diseases: [Type 2 diabetes, diabetic nephropathy, diabetic retinopathy, hypertension, ischemic heart disease]." (3) DeepSeek output: [Type 2 diabetes: 0.99, Diabetic nephropathy: 0.75, Diabetic retinopathy: 0.70, Hypertension: 0.60, Ischemic heart disease: 0.30] (This vector is the soft target T).

[0151] III. Student Model (MLP) Training: (1) The feature vector V_final of the same patient is input into the student MLP model, and the output probability S = [0.90,0.65, 0.60, 0.50, 0.20].

[0152] (2) Loss calculation: The actual hard label is Y = [1, 1, 1, 0, 0].

[0153] L_bce = BCE(Y, S) = -[log(0.90)+log(0.65)+log(0.60)+log(1-0.50)+log(1-0.20)] / 5; L_kl = KL(T||S) = 0.99*log(0.99 / 0.90) + ... (calculate all 5 categories); L_total = L_bce + 1.0 * L_kl.

[0154] In this embodiment, through distillation, the student model not only learns hard labels, but also learns the comorbid relationship knowledge of "diabetes is often accompanied by nephropathy and retinopathy" from the teacher model, making its predicted probability distribution closer to medical reality and improving its generalization ability.

[0155] The medical diagnostic reasoning method provided by the present invention is described below. The medical diagnostic reasoning method described below can be referred to in correspondence with the medical diagnostic reasoning system described above.

[0156] Figure 2This is a flowchart illustrating the medical diagnostic reasoning method provided by the present invention, as shown below. Figure 2 As shown, the method includes the following: Step 201: Receive the user's prescription information and basic information. The prescription information includes at least one drug name and at least one surgical procedure term. The basic information includes gender and age. Step 202: Standardize the prescription information by using a lightweight terminology standardization model to obtain the standard terms corresponding to the prescription information. The terminology standardization model is trained based on knowledge distillation technology. Step 203: Perform vector transformation and concatenation on the standard terms and the basic information to obtain a comprehensive feature vector; Step 204: Based on a lightweight multi-label disease prediction model, predict the multi-label vector corresponding to the comprehensive feature vector. Each dimension of the multi-label vector represents the probability of the specified disease corresponding to that dimension. The multi-label disease prediction model is trained based on the knowledge distillation technique.

[0157] Optionally, the method further includes: The terminology standardization model is received, and after pruning and quantizing the terminology standardization model, it is deployed. The terminology standardization model includes at least one sub-terminology standardization model. The training process of the terminology standardization model includes: Obtain a non-standard clinical terminology sample set, which includes multiple clinical terminology samples, such as drug name samples or surgical procedure terminology samples; Based on a standard terminology database, standard term labels are determined for each of the clinical terminology samples. Using the first large language model as the first teacher model and the pre-trained lightweight language model as the first student model, the first student model is trained by knowledge distillation based on the clinical terminology samples, the standard terminology labels of the clinical terminology samples, and the first teacher model to obtain the trained sub-terminology standardization model.

[0158] Optionally, determining the standard term tags for each of the clinical term samples based on a standard terminology database includes: For each clinical term sample, the comprehensive similarity between the clinical term sample and each standard term in the standard term library is determined, and the K candidate standard terms most similar to the clinical term sample are selected based on the comprehensive similarity, where K is a positive integer; Using the second language model, standard term labels for the clinical term samples are selected from the K candidate standard terms corresponding to the clinical term samples and the labeled standard terms corresponding to the clinical term samples.

[0159] Optionally, the step of performing knowledge distillation training on the first student model based on each of the clinical terminology samples, the standard terminology labels of each of the clinical terminology samples, and the first teacher model to obtain the trained sub-terminology standardization model includes: From each of the clinical terminology samples, select any one of the clinical terminology samples, and input the clinical terminology sample into the first teacher model and the first student model respectively to obtain the first predicted standard term output by the first teacher model and the second predicted standard term output by the first student model; Based on the first and second predicted standard terms, calculate the first soft label loss, and based on the second predicted standard terms and the standard term labels corresponding to the clinical term samples, calculate the first hard label loss. The first total loss is calculated based on the first soft label loss, the first hard label loss, and the first distillation weight; Based on the first total loss, the model parameters of the first student model are adjusted; The adjusted first student model is trained until the first training stop condition is met, and the trained first student model is used as the sub-term standardized model.

[0160] Optionally, determining the overall similarity between the clinical terminology sample and each standard term in the standard terminology database includes: The clinical terminology sample and each of the standard terms in the standard terminology library are respectively encoded with word vectors to obtain the first word vector code corresponding to the clinical terminology sample and the second word vector code corresponding to each of the standard terms in the standard terminology library. Calculate the cosine similarity between the first word vector code and each of the second word vector codes, and calculate the longest common subsequence similarity between the clinical term sample and each of the standard terms in the standard term library; For each standard term in the standard terminology library, the cosine similarity and the longest common subsequence similarity corresponding to the standard term are weighted and summed to obtain the comprehensive similarity between the standard term and the clinical terminology sample.

[0161] Optionally, the method further includes: Cache at least one key-value pair, where the key represents a non-standard term and the value represents a verified standard term; Calculate the similarity between the prescription information and the keys in each of the key-value pairs; If a target similarity exists among the various similarities, the value corresponding to the target similarity is determined as the standard term corresponding to the prescription information, and the target similarity is the similarity greater than the similarity threshold; If the target similarity does not exist among the aforementioned similarities, the step of standardizing the prescription information using a lightweight terminology standardization model to obtain the standard terms corresponding to the prescription information is performed. The standard terms and their corresponding prescription information determined by the standardization module are stored as new key-value pairs.

[0162] Optionally, the terminology standardization model includes at least one sub-terminology standardization model; The standardization transformation of the prescription information using a lightweight terminology standardization model to obtain the standard terminology corresponding to the prescription information includes: The prescription information is standardized and transformed using the respective sub-term standardization models to obtain at least one alternative standard term corresponding to the prescription information. The standard term corresponding to the prescription information is selected from the at least one alternative standard term using a weighted voting method.

[0163] Optionally, the step of performing vector transformation and concatenation on the standard terms and the basic information to obtain a comprehensive feature vector includes: The standard terms of each of the drug names are embedded and encoded to obtain the drug embedding code corresponding to each drug name. The standard terms of each of the surgical operation terms are also embedded and encoded to obtain the surgical embedding code corresponding to each surgical operation term. The drug embedding codes are aggregated to obtain the total drug feature vector, and the surgical embedding codes are aggregated to obtain the total surgical feature vector. The age is normalized to obtain an age feature vector; The gender is subjected to one-hot encoding to obtain a gender feature vector; The total feature vector of the drug, the total feature vector of the surgery, the age feature vector, and the gender feature vector are concatenated to obtain the comprehensive feature vector.

[0164] Optionally, the method further includes: Receive the multi-label disease prediction model sent by the second model training system, and deploy the multi-label disease prediction model after pruning and quantizing and compressing it; The training process of the multi-label disease prediction model includes: Using the optimized third language model as the second teacher model and the lightweight prediction model as the second student model, the second student model is trained by knowledge distillation based on the comprehensive feature vector sample set and the second teacher model to obtain the trained multi-label disease prediction model. The comprehensive feature vector sample set includes multiple comprehensive feature vector samples carrying multi-label vector labels.

[0165] Optionally, the step of training the second student model through knowledge distillation based on the comprehensive feature vector sample set and the second teacher model to obtain the trained multi-label disease prediction model includes: From each of the comprehensive feature vector samples, select any one of the comprehensive feature vector samples, and input the comprehensive feature vector sample into the second teacher model and the second student model respectively to obtain the first predicted multi-label vector output by the second teacher model and the second predicted multi-label vector output by the second student model; Based on the first predicted multi-label vector and the second predicted multi-label vector, calculate the second soft label loss, and based on the second predicted multi-label vector and the multi-label vector label corresponding to the comprehensive feature vector sample, calculate the second hard label loss. The second total loss is calculated based on the second soft label loss, the second hard label loss, and the second distillation weight; Based on the second total loss, the model parameters of the second student model are adjusted; The adjusted second student model is trained until the second training stopping condition is met, and the trained second student model is used as the multi-label disease prediction model.

[0166] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 3As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a medical diagnostic reasoning method. This method includes: receiving a user's prescription information and basic information, wherein the prescription information includes at least one drug name and at least one surgical procedure terminology, and the basic information includes gender and age; standardizing the prescription information using a lightweight terminology standardization model to obtain standard terms corresponding to the prescription information, wherein the terminology standardization model is trained based on knowledge distillation technology; performing vector transformation and concatenation on the standard terms and the basic information to obtain a comprehensive feature vector; and predicting a multi-label vector corresponding to the comprehensive feature vector based on a lightweight multi-label disease prediction model, wherein each dimension of the multi-label vector represents the probability of a specified disease corresponding to that dimension, and the multi-label disease prediction model is trained based on the knowledge distillation technology.

[0167] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0168] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the medical diagnostic reasoning method provided by the above methods. The method includes: receiving a user's prescription information and basic information, wherein the prescription information includes at least one drug name and at least one surgical procedure term, and the basic information includes gender and age; standardizing the prescription information using a lightweight terminology standardization model to obtain standard terms corresponding to the prescription information, wherein the terminology standardization model is trained based on knowledge distillation technology; performing vector transformation and concatenation on the standard terms and the basic information to obtain a comprehensive feature vector; and predicting the multi-label vector corresponding to the comprehensive feature vector based on a lightweight multi-label disease prediction model, wherein each dimension of the multi-label vector represents the probability of a specified disease corresponding to that dimension, and the multi-label disease prediction model is trained based on the knowledge distillation technology.

[0169] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the medical diagnostic reasoning method provided by the above methods. The method includes: receiving a user's prescription information and basic information, wherein the prescription information includes at least one drug name and at least one surgical procedure term, and the basic information includes gender and age; standardizing the prescription information using a lightweight terminology standardization model to obtain standard terms corresponding to the prescription information, wherein the terminology standardization model is trained based on knowledge distillation technology; performing vector transformation and concatenation on the standard terms and the basic information to obtain a comprehensive feature vector; and predicting a multi-label vector corresponding to the comprehensive feature vector based on a lightweight multi-label disease prediction model, wherein each dimension of the multi-label vector represents the probability of a specified disease corresponding to that dimension, and the multi-label disease prediction model is trained based on the knowledge distillation technology.

[0170] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0171] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A medical diagnostic reasoning system, characterized in that, include: A receiving module is used to receive the user's prescription information and basic information. The prescription information includes at least one drug name and at least one surgical procedure term. The basic information includes gender and age. The standardization module is used to standardize the prescription information using a lightweight terminology standardization model to obtain the standard terms corresponding to the prescription information. The terminology standardization model is trained based on knowledge distillation technology. The vectorization module is used to perform vector transformation and concatenation on the standard terms and the basic information to obtain a comprehensive feature vector. The disease prediction module is used to predict the multi-label vector corresponding to the comprehensive feature vector based on a lightweight multi-label disease prediction model. Each dimension of the multi-label vector represents the probability of the set disease corresponding to that dimension. The multi-label disease prediction model is trained based on the knowledge distillation technique.

2. The medical diagnostic reasoning system according to claim 1, characterized in that, The medical diagnostic reasoning system also includes a first deployment module; The first deployment module is used to receive the terminology standardization model sent by the first model training system, and after pruning and quantizing the terminology standardization model, deploy it in the standardization module. The terminology standardization model includes at least one sub-terminology standardization model. The first model training system is used to acquire a non-standard clinical terminology sample set, which includes multiple clinical terminology samples, such as drug name samples or surgical procedure terminology samples; determine standard terminology labels for each of the clinical terminology samples based on a standard terminology database; use a first large language model as the first teacher model and a pre-trained lightweight language model as the first student model; and perform knowledge distillation training on the first student model based on each of the clinical terminology samples, the standard terminology labels of each of the clinical terminology samples, and the first teacher model to obtain the trained sub-terminology standardization model.

3. The medical diagnostic reasoning system according to claim 2, characterized in that, The determination of standard term tags for each clinical term sample based on a standard terminology database includes: For each clinical term sample, the comprehensive similarity between the clinical term sample and each standard term in the standard term library is determined, and the K candidate standard terms most similar to the clinical term sample are selected based on the comprehensive similarity, where K is a positive integer; Using the second language model, standard term labels for the clinical term samples are selected from the K candidate standard terms corresponding to the clinical term samples and the labeled standard terms corresponding to the clinical term samples.

4. The medical diagnostic reasoning system according to claim 2, characterized in that, The step of training the first student model through knowledge distillation based on each of the clinical terminology samples, the standard terminology labels of each of the clinical terminology samples, and the first teacher model to obtain the trained sub-terminology standardization model includes: From each of the clinical terminology samples, select any one of the clinical terminology samples, and input the clinical terminology sample into the first teacher model and the first student model respectively to obtain the first predicted standard term output by the first teacher model and the second predicted standard term output by the first student model; Based on the first and second predicted standard terms, calculate the first soft label loss, and based on the second predicted standard terms and the standard term labels corresponding to the clinical term samples, calculate the first hard label loss. The first total loss is calculated based on the first soft label loss, the first hard label loss, and the first distillation weight; Based on the first total loss, the model parameters of the first student model are adjusted; The adjusted first student model is trained until the first training stop condition is met, and the trained first student model is used as the sub-term standardized model.

5. The medical diagnostic reasoning system according to claim 3, characterized in that, Determining the overall similarity between the clinical terminology sample and each standard term in the standard terminology database includes: The clinical terminology sample and each of the standard terms in the standard terminology library are respectively encoded with word vectors to obtain the first word vector code corresponding to the clinical terminology sample and the second word vector code corresponding to each of the standard terms in the standard terminology library. Calculate the cosine similarity between the first word vector code and each of the second word vector codes, and calculate the longest common subsequence similarity between the clinical term sample and each of the standard terms in the standard term library; For each standard term in the standard terminology library, the cosine similarity and the longest common subsequence similarity corresponding to the standard term are weighted and summed to obtain the comprehensive similarity between the standard term and the clinical terminology sample.

6. The medical diagnostic reasoning system according to any one of claims 1-5, characterized in that, The medical diagnostic reasoning system also includes a caching module; The caching module caches at least one key-value pair, where the key in the key-value pair represents a non-standard term and the value in the key-value pair represents a verified standard term. The caching module is used to calculate the similarity between the prescription information and the keys in each key-value pair; if a target similarity exists among the similarities, the value corresponding to the target similarity is determined as the standard term corresponding to the prescription information, wherein the target similarity is the similarity greater than a similarity threshold; The standardization module is specifically used to standardize the prescription information of the prescription information by means of a lightweight terminology standardization model when the target similarity does not exist in each of the similarities, so as to obtain the standard terms corresponding to the prescription information. The caching module is further configured to store the standard terms determined by the standardization module and their corresponding prescription information as new key-value pairs.

7. The medical diagnostic reasoning system according to any one of claims 1-5, characterized in that, The terminology standardization model includes at least one sub-terminology standardization model; The standardization module is specifically used to standardize and transform the prescription information of the prescription information through each of the sub-term standardization models to obtain at least one alternative standard term corresponding to the prescription information; and to select the standard term corresponding to the prescription information from the at least one alternative standard term using a weighted voting method.

8. The medical diagnostic reasoning system according to any one of claims 1-5, characterized in that, The vectorization module includes an embedding layer, an aggregation layer, a normalization layer, a one-hot encoding layer, and a splicing layer; The embedding layer is used to embed and encode the standard terms of each drug name to obtain the drug embedding code corresponding to each drug name, and to embed and encode the standard terms of each surgical operation term to obtain the surgical embedding code corresponding to each surgical operation term. The aggregation layer is used to aggregate the drug embedding codes to obtain a total drug feature vector, and to aggregate the surgical embedding codes to obtain a total surgical feature vector. The normalization layer is used to normalize the age to obtain an age feature vector; The one-hot encoding layer is used to perform one-hot encoding on the gender to obtain a gender feature vector; The splicing layer is used to splice the total feature vector of the drug, the total feature vector of the surgery, the age feature vector, and the gender feature vector to obtain the comprehensive feature vector.

9. The medical diagnostic reasoning system according to any one of claims 1-5, characterized in that, The medical diagnostic reasoning system also includes a second deployment module; The second deployment module is used to receive the multi-label disease prediction model sent by the second model training system, and deploy the multi-label disease prediction model in the disease prediction module after pruning and quantizing and compressing it. The second model training system is used to train the second student model by knowledge distillation based on a optimized third language model as the second teacher model and a lightweight prediction model as the second student model, using a comprehensive feature vector sample set and the second teacher model to obtain the trained multi-label disease prediction model. The comprehensive feature vector sample set includes multiple comprehensive feature vector samples carrying multi-label vector labels.

10. The medical diagnostic reasoning system according to claim 9, characterized in that, The process of training the second student model through knowledge distillation based on the comprehensive feature vector sample set and the second teacher model to obtain the trained multi-label disease prediction model includes: From each of the comprehensive feature vector samples, select any one of the comprehensive feature vector samples, and input the comprehensive feature vector sample into the second teacher model and the second student model respectively to obtain the first predicted multi-label vector output by the second teacher model and the second predicted multi-label vector output by the second student model; Based on the first predicted multi-label vector and the second predicted multi-label vector, calculate the second soft label loss, and based on the second predicted multi-label vector and the multi-label vector label corresponding to the comprehensive feature vector sample, calculate the second hard label loss. The second total loss is calculated based on the second soft label loss, the second hard label loss, and the second distillation weight; Based on the second total loss, the model parameters of the second student model are adjusted; The adjusted second student model is trained until the second training stopping condition is met, and the trained second student model is used as the multi-label disease prediction model.

Images