Clinical special disease prediction method and device based on large model instruction prompt
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INST OF AUTOMATION CHINESE ACAD OF SCI
- Filing Date
- 2026-04-03
- Publication Date
- 2026-07-21
Smart Images
Figure CN121983340B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of medical big data processing technology, and in particular to a clinical disease prediction method and device based on large model instruction prompts. Background Technology
[0002] Clinical disease specialization refers to a standardized, full-cycle diagnosis and treatment system built by modern medical institutions based on the traditional subdivision of clinical specialties and integrating multidisciplinary resources around a specific disease. With the increasing demand for precision medicine, the disease-centric clinical disease specialization model has been widely adopted. In this context, predicting disease progression and adverse outcomes early based on structured electronic health records (EHRs) is of great significance for achieving early risk stratification and improving patient prognosis. Real-world EHRs typically consist of tables, sequences, and multi-source clinical events, exhibiting high dimensionality, strong heterogeneity, and complex time dependencies, containing rich information on disease evolution.
[0003] In related technologies, traditional machine learning methods such as support vector machines and random forests are commonly used for disease diagnosis and risk prediction. However, these methods have significant shortcomings when dealing with high-dimensional, multi-source, heterogeneous real-world EHR data: they require extensive expert knowledge for manual feature extraction, resulting in low efficiency; they struggle to capture complex nonlinear relationships in long-sequence clinical events; and they lack the ability to reason about deep medical logic, making them unsuitable for complex clinical disease scenarios.
[0004] Some solutions attempt to introduce Large Language Models (LLMs) for EHR data processing. Compared to traditional machine learning methods, LLMs can capture complex relationships between clinical events with less manual feature engineering, thus demonstrating greater application potential in clinical disease prediction and risk assessment. However, using LLMs for HER data processing also presents the following problems: Due to the context length limitation of LLMs, it is difficult to fully utilize long-sequence EHRs, which may lead to the hiding or loss of key information; general-purpose LLMs often lack deep adaptation to specialized medical knowledge, making it difficult to accurately capture the complex relationships and development patterns unique to clinical diseases, thus exhibiting suboptimal predictive performance in real-world clinical scenarios; and the annotation data for real-world clinical diseases requires significant human and material resources, so the amount of labeled data is usually limited, further increasing the difficulty of training and generalizing predictive models.
[0005] Therefore, there is an urgent need for an efficient and accurate clinical disease prediction method based on LLM. Summary of the Invention
[0006] In view of this, embodiments of this application provide a clinical disease prediction method and apparatus based on large model instruction prompts to solve the problems in the prior art where the high-dimensional heterogeneity of structured electronic health records is difficult to model effectively, and where the scarcity of labeled data in clinical disease scenarios and insufficient adaptation to the general LLM domain make it difficult to accurately predict diseases.
[0007] A first aspect of this application provides a clinical disease prediction method based on large model instruction prompts, including:
[0008] Acquire structured electronic health records (EHRs) for clinical diseases;
[0009] Triple event units are extracted from clinical disease-specific EHR data; each triple event unit includes a unique identifier, event type, and event value.
[0010] Semantic encoding is performed on triplet event units to obtain clinical event coding features;
[0011] QA (Question Answer) question pairs are constructed based on triplet event units, and natural language prompts are constructed using QA question pairs;
[0012] Natural language prompts are embedded and encoded to obtain embedded expression of the instructions;
[0013] A linear projector is used to project clinical event coding features, and the projected clinical event coding features and instruction embedding expressions are then aligned in both dimensions and semantics.
[0014] The projected clinical event coding features and instruction embedding expressions are sequentially concatenated to obtain a dual-modal hybrid embedding sequence.
[0015] The first major model was used to make predictions based on bimodal hybrid embedding sequences, and clinical disease prediction results were obtained.
[0016] A second aspect of this application provides a clinical disease prediction device based on large model instruction prompts, comprising:
[0017] The acquisition module is configured to acquire structured electronic health records (EHRs) for clinical diseases.
[0018] The extraction target is configured to extract triplet event units from clinical disease EHR data; the triplet event unit includes a unique identifier, event type, and event value.
[0019] The semantic encoding module is configured to perform semantic encoding on triplet event units to obtain clinical event encoding features;
[0020] The building module is configured to construct QA question pairs based on triplet event units and use the QA question pairs to construct natural language prompts.
[0021] The embedding encoding module is configured to embed and encode natural language prompts to obtain embedded expression of the instructions;
[0022] The bimodal alignment module is configured to project clinical event coding features using a linear projector to perform dimensional and semantic alignment between the projected clinical event coding features and the instruction embedding representation.
[0023] The bimodal alignment module is also configured to sequentially concatenate the projected clinical event coding features and instruction embedding expressions to obtain a bimodal hybrid embedding sequence;
[0024] The prediction module is configured to use the first major model to make predictions based on bimodal hybrid embedding sequences to obtain clinical disease prediction results.
[0025] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.
[0026] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.
[0027] The beneficial effects of the embodiments in this application compared with the prior art are:
[0028] This application first extracts triple event units from clinical disease EHR data. On one hand, it performs semantic encoding on the triple event units to obtain clinical event encoding features. On the other hand, it constructs QA question pairs based on the triple event units, uses the QA question pairs to construct natural language prompts, and embeds and encodes the natural language prompts to obtain instruction embedding expressions. Then, it projects the clinical event encoding features to achieve dimensional and semantic alignment between the projected clinical event encoding features and the instruction embedding expressions. Finally, it concatenates the projected clinical event encoding features and the instruction embedding expressions to obtain a bimodal hybrid embedding sequence. The first large model derives the clinical disease prediction results based on this bimodal hybrid embedding sequence, which improves the accuracy of extracting clinical disease-related information from EHR data and can generate high-quality instruction samples adapted to clinical disease scenarios. Projecting and aligning the two to obtain a high-quality bimodal hybrid embedding sequence can improve classification prediction accuracy. Attached Figure Description
[0029] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0030] Figure 1 This is a flowchart illustrating a clinical disease prediction method based on large model instruction prompts provided in an embodiment of this application.
[0031] Figure 2 This is a network structure diagram of the clinical disease prediction method based on large model instruction prompts provided in the embodiments of this application.
[0032] Figure 3 This is a schematic diagram of a clinical disease prediction device based on large model instruction prompts provided in an embodiment of this application.
[0033] Figure 4 This is a schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation
[0034] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0035] The following will describe in detail, with reference to the accompanying drawings, a clinical disease prediction method and apparatus based on large model instruction prompts according to embodiments of this application.
[0036] As mentioned above, traditional machine learning methods for disease-specific diagnosis suffer from drawbacks such as difficulty in efficiently and accurately extracting EHR data features and a lack of deep medical logical reasoning capabilities. The solution of introducing LLM for HER data processing also suffers from drawbacks such as insufficient accuracy of LLM in processing long-sequence EHR data and insufficient generalization ability of the model due to the limited amount of disease-specific HER data.
[0037] In view of this, the embodiments of this application provide a clinical disease prediction method based on large model instruction prompts. First, a domain-specific instruction dataset is constructed based on structured EHRs. Then, diverse question-and-answer examples are formed by using template extraction and semantic rewriting. This method is designed to address the technical problems in the prior art, such as the difficulty in effectively modeling the high-dimensional heterogeneity of structured electronic health records, the scarcity of labeled data in clinical disease scenarios, and the insufficient adaptation to the general LLM domain.
[0038] Then, a dual-modal alignment coding architecture is designed. Clinical event features are extracted through an event encoder and mapped to the LLM embedding space using linear projection to achieve semantic alignment between structured EHR and natural language instructions. Finally, a two-stage instruction fine-tuning mechanism is adopted to explore the correlation logic between clinical events and prediction targets. By combining the event encoder and the linear projection alignment module, the deep integration of structured EHR data and clinical natural language instructions is achieved, thereby accurately exploring disease-specific disease association patterns and improving the performance and robustness of clinical prediction.
[0039] Figure 1 This is a flowchart illustrating a clinical disease prediction method based on large model instruction prompts provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps:
[0040] In step S101, structured electronic health record data for clinical diseases is acquired.
[0041] In step S102, triplet event units are extracted from the structured electronic health record data for clinical diseases.
[0042] The triplet event unit includes a unique identifier, an event type, and an event value.
[0043] In step S103, the triplet event units are semantically encoded to obtain clinical event coding features.
[0044] In step S104, QA question pairs are constructed based on triplet event units, and natural language prompts are constructed using QA question pairs.
[0045] In step S105, the natural language prompt instructions are embedded and encoded to obtain the instruction embedding expression.
[0046] In step S106, a linear projector is used to project the clinical event coding features, so as to perform dimensional and semantic alignment between the projected clinical event coding features and the instruction embedding expression.
[0047] In step S107, the projected clinical event coding features and instruction embedding expressions are sequentially concatenated to obtain a bimodal hybrid embedding sequence.
[0048] In step S108, the first large model is used to make predictions based on bimodal hybrid embedding sequences to obtain clinical disease prediction results.
[0049] In some embodiments of this application, the method may be executed by a server or by a terminal device with certain processing capabilities.
[0050] In some embodiments of this application, clinical disease-specific EHR data can be acquired first. This clinical disease-specific EHR data can be the clinical disease-specific EHR data of patients requiring clinical disease prediction. For example, patient information (hosp / patients), hospital admission records (hosp / admissions), diagnoses (hosp / diagnoses_icd), laboratory tests (hosp / labevents), and medications (hosp / prescriptions) can be acquired.
[0051] In some embodiments of this application, triplet event units can be extracted from clinical disease-specific EHR data. Since a patient's clinical disease-specific EHR data may be scattered across different tables, triplet event units can be extracted from the patient's clinical disease-specific EHR data through multi-table association and integration, thereby forming an ordered patient-level event sequence that preserves clinical semantic integrity. This triplet event unit may include a unique identifier, an event type, and an event value.
[0052] In some embodiments of this application, the triplet event units can be semantically encoded to obtain clinical event coding features.
[0053] In some embodiments of this application, QA question pairs can be constructed based on triplet event units, and then natural language prompts can be constructed using these QA question pairs. These natural language prompts are used to guide the large model on how to obtain medical-grade feature information from the input information. Furthermore, the natural language prompts can be embedded and encoded to obtain an embedded expression.
[0054] In some embodiments of this application, a linear projector can be used to project clinical event coding features to align the projected clinical event coding features and instruction embeddings in terms of both dimension and semantics. Then, the projected clinical event coding features and instruction embeddings are sequentially concatenated to obtain a bimodal hybrid embedding sequence.
[0055] Finally, by using the first major model based on bimodal hybrid embedding sequences for prediction, the patient's clinical disease prediction results can be obtained. This first major model can be a trained LLM, the training method of which will be described in detail later and will not be repeated here.
[0056] According to the technical solution provided in the embodiments of this application, firstly, triple event units are extracted from clinical disease EHR data. On the one hand, the triple event units are semantically encoded to obtain clinical event encoding features. On the other hand, QA question pairs are constructed based on the triple event units. Natural language prompts are constructed using the QA question pairs, and the natural language prompts are embedded and encoded to obtain instruction embedding expressions. Then, the clinical event encoding features are projected to achieve dimensional and semantic alignment between the projected clinical event encoding features and the instruction embedding expressions. Finally, the projected clinical event encoding features and the instruction embedding expressions are concatenated to obtain a bimodal hybrid embedding sequence. The first large model derives the clinical disease prediction results based on this bimodal hybrid embedding sequence, which improves the accuracy of extracting clinical disease-related information from EHR data and can generate high-quality instruction samples adapted to clinical disease scenarios. Projecting and aligning the two to obtain a high-quality bimodal hybrid embedding sequence can improve classification prediction accuracy.
[0057] In some embodiments of this application, extracting triplet event units from clinical disease EHR data may include:
[0058] First, the clinical disease EHR data are preprocessed; the preprocessing includes at least time alignment and outlier removal.
[0059] Then, the clinical disease EHR data are integrated by multi-table association using unique identifiers, and the data in each table is split field by field to obtain triple event units; wherein, the unique identifier includes at least one of the patient unique identifier and the admission unique identifier.
[0060] In other words, multi-table heterogeneous EHRs can be converted into triple event units in the format of (hadm_id, event_type, event_value) according to the medical event data standard, where hadm_id is the unique admission identifier, event_type is the event type, and event_value is the event value.
[0061] In the data preprocessing process, strict adherence to clinical logic and quality requirements is essential: Starting with ICU inpatient records, ensure they are linked to valid hospital admission records to guarantee data hierarchy integrity; for cases of multiple ICU transfers during the same hospitalization period, retain only the initial record to simplify the trajectory; correct key time-series variables such as hospitalization duration, remove outliers, and obtain a high-quality data cohort. Based on this high-quality data cohort, further extraction of three types of events—diagnosis, laboratory testing, and medication prescription—results in time-aligned, structurally clear, multimodal clinical disease-specific EHR data, laying a reliable data foundation for subsequent model predictions.
[0062] In some embodiments of this application, constructing QA question pairs based on triplet event units may include:
[0063] First, obtain a preset QA question pair template. This preset QA question pair template can be generated based on a knowledge graph of a clinical disease domain, or generated using machine learning methods based on data from the clinical disease domain.
[0064] Then, using the event type and event value in the target triplet event unit as the question in the preset QA question pair template, an initial question-answer pair is obtained. The target triplet event unit can be any triplet event unit.
[0065] Next, the second major model is invoked to semantically rewrite the initial question-answer pair, resulting in a rewritten answer. The rewritten answer maintains medical semantic consistency with the initial question-answer pair, and the rewriting prompts include at least role settings, medical fact-locking constraints, sample examples, and rewriting instructions.
[0066] Among them, role setting refers to setting the semantic rewriting subject, such as an AI reasoning assistant with clinical disease-specific medical expertise; medical fact locking constraint is used to constrain the second major model to strictly follow medical facts during the reasoning process; sample examples can include a small number of relevant disease-specific prediction examples; rewriting instructions are used to limit the requirements of semantic rewriting so that the rewritten answer has a variety of expression methods.
[0067] Finally, by combining the target triplet event unit and the rewritten answer, we obtain the QA question pair for the target triplet event unit.
[0068] The second largest model can be an LLM, and it can be the same as or different from the first largest model.
[0069] In other words, initial question-and-answer pairs can be automatically generated based on a preset QA question template, targeting each clinical feature described in the triplet event unit. Then, a second large model (such as DeepSeek) can be used to semantically rewrite the initial QA pairs while preserving medical semantic consistency, thereby improving language diversity and reducing template bias.
[0070] In one example, the second large model can be a general-purpose large model with hundreds of billions of parameters, such as DeepSeek-V3. When using the second large model for semantic rewriting, to achieve precise medical semantic control on the general-purpose large model, this application embodiment designs a multi-level composite prompt word template, including four core parts: role setting, medical fact locking constraints, a small number of sample examples, and diverse rewriting instructions. This ensures that the model strictly follows medical facts during generation while achieving diverse expression methods. Through the combination of context learning and composite constraint prompt words, precise medical semantic rewriting is achieved. When medical knowledge is updated, only the sample examples in the prompt words need to be modified to update the knowledge of the second large model.
[0071] This method requires no expensive training resources or a long training period, and is low in cost and high in accuracy.
[0072] In some embodiments of this application, semantic encoding of triplet event units to obtain clinical event encoding features may include:
[0073] First, all the triplet event units of the patient are combined to obtain the clinical event flow.
[0074] Then, each triplet event unit in the clinical event stream is input into the pre-trained encoder one by one. The semantics within the event are modeled and integrated through a multi-layer attention mechanism to obtain high-dimensional continuous clinical event coding features.
[0075] In other words, a clinical event semantic coding module can be set up to extract and represent semantics from clinical disease-specific EHR data. The input of this module is fine-grained clinical event units in the standardized clinical event stream. Each event is constructed based on a (hadm_id, event_type, event_value) triple structure, covering key clinical elements such as disease-related demographic characteristics, laboratory test indicators, diagnosis and treatment records, and medication information.
[0076] In one example, this module can use a BioMedBERT model pre-trained in the medical field as the core encoder. Leveraging its domain advantages in medical terminology recognition and clinical semantic modeling, it effectively alleviates the semantic shift and information loss problems of general text encoding models in medical scenarios. Specifically, each triplet event unit in the clinical event stream is input into BioMedBERT one by one. Through a multi-layer self-attention mechanism, the internal semantics of the events are modeled and integrated, mapping discrete clinical events into high-dimensional continuous event embedding vectors. While preserving the semantic integrity of individual events, it implicitly characterizes the potential clinical relationships between events, ultimately outputting structured, domain-aware clinical event encoding features. This provides a high-quality structured data input foundation for subsequent multimodal feature fusion and downstream prediction tasks.
[0077] In some embodiments of this application, constructing natural language prompts using QA questions may include:
[0078] The role definition is determined based on the target QA question. This role definition includes the identity definition of an AI reasoning assistant with clinical disease-specific medical expertise.
[0079] Define the input guidance. This input guidance includes at least a prompt and placeholders, the content of which is determined based on the event type and event value of the target QA question pair.
[0080] Define the clinical objectives. These objectives include constraints on clinical feature extraction, which must include at least the type of clinical indicator and the time frame.
[0081] Define the output format constraints.
[0082] Then, by combining role definition, input information guidance, clinical goal constraints, and output format constraints, natural language prompts for the target QA question pair are obtained. The target QA question pair can be any QA question pair.
[0083] In other words, natural language prompts can be designed to guide large models on how to extract medical feature-level information. The core of this medical feature-level information extraction instruction design method lies in building a structured prompt instruction system around the clinical information extraction task. This prompt instruction can include role definition, input information guidance, clinical goal constraints, and output format constraints.
[0084] The role definition is used to define the reasoning identity of the AI assistant with medical expertise, thereby limiting the medical-related reasoning context of the model; the input information guidance can use "With the following information:" as a unified guiding phrase, combined with... Placeholders are used to introduce a set of structured features from the patient's clinical event flow, providing the data basis for the large model to perform the clinical information extraction task. Clinical goal constraints are used to explicitly constrain the fine-grained clinical feature extraction goals through formal natural language questions, which simultaneously limit the specific clinical indicator types and their corresponding time nodes, thereby avoiding semantic ambiguity. Output format constraints restrict the output format of the large model through preset response markers, enabling the large model to generate structured and parsable numerical results based on the real records in the clinical event flow.
[0085] Among them, this large model can be the first large model.
[0086] The available natural language prompts are used as input to the first main model. The embedding layer within this model embeds and encodes these prompts, automatically mapping words in the prompts to word embedding vectors. The first main model's fundamental semantic integration mechanism then generates instruction embeddings that carry task-oriented information. These instruction embeddings naturally align with the first main model's internal representation space, effectively adapting to subsequent inference processes and providing a high-quality natural language input foundation for cross-modal fusion of structured clinical data and natural language information.
[0087] In some embodiments of this application, projecting clinical event coding features using a linear projector may include:
[0088] Clinical event coding features are input into a linear projector.
[0089] The clinical event coding features are transformed using the weight matrix of a linear projector to obtain the projected values of the clinical event coding features.
[0090] In response to the determination that the projection values of clinical event coding features and instruction embedding expressions do not meet the alignment target, the weight matrix is updated.
[0091] The process iteratively executes the steps of transforming the clinical event coding features again with the updated weight matrix to obtain the updated clinical event coding feature projection values, and updating the weight matrix again in response to the determination that the updated clinical event coding feature projection values and instruction embedding expressions do not meet the alignment target, until the updated clinical event coding feature projection values and instruction embedding expressions meet the alignment target.
[0092] The alignment objectives include dimensional alignment and semantic alignment.
[0093] In other words, a cross-modal projection fusion module can be set up as the core bridge to achieve bimodal alignment and deep fusion between clinical event coding features and instruction embedding expressions. This cross-modal projection fusion module uses a trainable linear projector as its key component.
[0094] Specifically, clinical event coding features can be input into a linear projector, and a dual alignment goal can be achieved through learnable weight matrix transformation: on the one hand, dimensional alignment is achieved, mapping the clinical event coding features to a feature dimension space that is completely consistent with the instruction embedding expression; on the other hand, semantic alignment is achieved, continuously optimizing the projector parameters during model training so that the semantic distribution of the clinical event coding features and the inference space where the instruction embedding expression is located are precisely matched, thereby ensuring that the core semantics of the clinical event coding features are highly consistent with the task intent contained in the instruction embedding expression.
[0095] After completing the alignment process described above, the projected clinical event encoding features and instruction embeddings are sequentially concatenated according to a preset order to achieve unified modeling and deep fusion at the feature level, ultimately constructing a unified bimodal hybrid embedding sequence. This sequence fully preserves the core clinical information of the structured EHR while explicitly injecting task-oriented semantics of natural language instructions. It can be directly used as the input representation for the subsequent two-stage fine-tuning process of the large model, effectively breaking down the modal barriers between structured clinical data and the reasoning capabilities of large language models.
[0096] In some embodiments of this application, the bimodal hybrid embedding sequence can be input into a first large model to obtain clinical disease prediction results. This first large model includes at least a classification head, which determines the clinical disease prediction result based on the bimodal hybrid embedding sequence. In one example, a fully connected layer can be used as the classification head to distinguish between diseased and non-disease-related conditions. If the output is "non-disease-related," it indicates a normal state requiring no intervention; if the output is "disease-related," medical personnel need to be prepared.
[0097] Furthermore, the first large model is trained using a two-stage training mechanism; in the first training stage, natural language prompts from the clinical disease domain are used to perform domain-adaptive training on the general large model, and in the second training stage, EHR data from the clinical disease domain are used to perform task-oriented training on the classification head.
[0098] The EHR data for the clinical disease domain can be, for example, all or part of the EHR data related to this clinical disease in the MIMIC-IV database. The general large model can be a pre-trained first large model. By using natural language prompts in the clinical disease domain to perform domain-adaptive training on the pre-trained first large model, a trained first large model can be obtained, which is the first large model used for clinical disease analysis.
[0099] In some embodiments of this application, the first large model uses cross-entropy loss as the loss function during the training phase, and uses backpropagation to update the model parameters until convergence. The adjustment parameters when updating the model parameters include at least one of the following: learning rate, number of training rounds, feature dimension of the model, number of encoder layers, or number of attention heads.
[0100] The first master model after training can be evaluated using a test set. Evaluation can be performed using random sampling, for example, by dividing clinical disease domain EHR data into an 8:2 ratio, with 80% used to fine-tune the pre-trained master model and 20% used as the test set.
[0101] The evaluation metrics used include accuracy (Acc), precision (Pre), recall, F1 score, and area under the curve (AUC) as core indicators. Acc is the ratio of the number of samples correctly predicted by the model to the total number of samples in the test set; Pre is the proportion of samples predicted as positive by the model that were actually positive; Recall is the proportion of samples that were actually positive that were successfully predicted as positive by the model; the F1 score is the harmonic mean of precision and recall, used to balance the inherent contradiction between the two; and AUC reflects the classification performance during the interictal and preictal periods.
[0102] Figure 2 This is a network structure diagram of the clinical disease prediction method based on large model instruction prompts provided in the embodiments of this application. For example... Figure 2 As shown, this network structure can use all or part of the EHR data related to this clinical specialty in the MIMIC-IV database as input to extract triplet event units. Some exemplary triplet event units are shown in Table 1:
[0103] Table 1 Examples of Triplet Event Units
[0104]
[0105] On the one hand, the BioMedBERT model can be used to extract features from triplet event units to obtain clinical event coding features; on the other hand, triplet event units can be used to construct OA question pairs, and natural language prompts can be designed based on the OA question pairs through the instruction design module. Natural language prompts can be embedded and encoded using the embedding expression layer of an LLM to obtain instruction embedding expressions. Here, the LLM can be the first large model after training mentioned above.
[0106] Furthermore, a linear projector can be used to project the clinical event coding features to achieve dimensional and semantic alignment between the clinical event coding features and the instruction embedding expressions. The projected clinical event coding features and instruction embedding expressions are then sequentially concatenated to obtain a bimodal hybrid embedding sequence. Finally, LLM is used to perform predictions based on the bimodal hybrid embedding sequence to obtain clinical disease prediction results.
[0107] The technical solution provided in this application achieves high-precision prediction of clinical diseases through the preparation of a domain-specific instruction dataset, the construction of a bimodal alignment coding architecture, and a two-stage LLM instruction fine-tuning learning process. Specifically, the domain-specific instruction dataset preparation stage extracts core clinical information related to the disease from structured EHRs, providing a high-quality training foundation for bimodal alignment. The bimodal alignment coding architecture construction stage establishes the association between structured clinical data and natural language instructions through medical semantic encoding and cross-modal mapping. Finally, the two-stage LLM instruction fine-tuning learning stage mines the intrinsic relationship between clinical events and the prediction target and integrates it into accurate prediction logic.
[0108] Specifically, bimodal alignment is defined as the semantic adaptation process between structured EHR data and natural language prediction instructions. Since structured EHRs store multidimensional clinical events (such as laboratory indicators and medical records) in tabular form, and natural language instructions explicitly predict targets (such as disease diagnosis and treatment response) in query form, the core of bimodal alignment includes semantic encoding of clinical events, embedding space mapping, and hybrid sequence construction. Fine-grained event units decomposed from structured EHRs are considered core information carriers, while natural language prediction instructions are considered task-oriented guidance. The two form a unified input paradigm through an adaptation module. To build the bimodal alignment encoding architecture, the structured EHRs are first split and filtered to retain key content such as disease-related laboratory tests, treatment drugs, and diagnostic codes. An event encoder is used to encode the event units to capture medical-specific semantics, forming event feature representations. Simultaneously, a linear projector maps the event representations to the input embedding space of a basic LLM, concatenating them with the natural language prediction instruction text to construct a hybrid input sequence of "event embedding + instruction text," completing the deep alignment of bimodal information.
[0109] After completing the bimodal aligned encoding architecture, a two-stage LLM instruction fine-tuning learning mechanism is introduced to train the prediction model: the first stage is based on a domain-specific instruction dataset to carry out domain-adaptive pre-training, so that LLM can adapt to the distribution of disease clinical data and professional knowledge logic; the second stage is based on a specific disease prediction task dataset to carry out task-oriented fine-tuning, capture the influence weight of different clinical events on the prediction target, integrate the core value of bimodal information by minimizing the generative question answering loss function, and apply the trained model to pediatric clinical data.
[0110] The LLM instruction fine-tuning framework for clinical disease prediction proposed in this application can accurately capture the unique clinical associations of the disease domain and make full use of the high-dimensional heterogeneous information of structured EHRs. The domain-specific instruction dataset construction process designed in this application can generate high-quality instruction samples adapted to clinical disease scenarios through targeted extraction, template extraction, and semantic rewriting. The bimodal alignment and two-stage fine-tuning mechanism proposed in this application can mine the intrinsic association between structured clinical events and natural language instructions and integrate them into a unified predictive representation.
[0111] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.
[0112] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.
[0113] Figure 3 This is a schematic diagram of a clinical disease prediction device based on large model instruction prompts provided in an embodiment of this application. Figure 3 As shown, the device includes:
[0114] Module 301 is configured to acquire clinical disease EHR data.
[0115] Target 302 is configured to extract triplet event units from clinical disease EHR data; the triplet event unit includes a unique identifier, event type, and event value.
[0116] Semantic coding module 303 is configured to perform semantic coding on triplet event units to obtain clinical event coding features.
[0117] Module 304 is configured to construct QA question pairs based on triplet event units and use the QA question pairs to construct natural language prompts.
[0118] The embedding encoding module 305 is configured to embed and encode natural language prompts to obtain an embedded expression of the instructions.
[0119] The bimodal alignment module 306 is configured to project clinical event coding features using a linear projector to perform dimensional and semantic alignment between the projected clinical event coding features and the instruction embedding representation.
[0120] The bimodal alignment module 306 is also configured to sequentially concatenate the projected clinical event coding features and instruction embedding expressions to obtain a bimodal hybrid embedding sequence.
[0121] Prediction module 307 is configured to use the first major model to make predictions based on bimodal hybrid embedding sequences to obtain clinical disease prediction results.
[0122] According to the technical solution provided in the embodiments of this application, firstly, triple event units are extracted from clinical disease EHR data. On the one hand, the triple event units are semantically encoded to obtain clinical event encoding features. On the other hand, QA question pairs are constructed based on the triple event units. Natural language prompts are constructed using the QA question pairs, and the natural language prompts are embedded and encoded to obtain instruction embedding expressions. Then, the clinical event encoding features are projected to achieve dimensional and semantic alignment between the projected clinical event encoding features and the instruction embedding expressions. Finally, the projected clinical event encoding features and the instruction embedding expressions are concatenated to obtain a bimodal hybrid embedding sequence. The first large model derives the clinical disease prediction results based on this bimodal hybrid embedding sequence, which improves the accuracy of extracting clinical disease-related information from EHR data and can generate high-quality instruction samples adapted to clinical disease scenarios. Projecting and aligning the two to obtain a high-quality bimodal hybrid embedding sequence can improve classification prediction accuracy.
[0123] In some implementations, extracting triplet event units from clinical disease EHR data includes: preprocessing the clinical disease EHR data; the preprocessing includes at least time alignment and outlier removal; performing multi-table association and integration of the clinical disease EHR data through unique identifiers, and splitting the data in each table field by field to obtain triplet event units; wherein, the unique identifier includes at least one of patient unique identifier and admission unique identifier.
[0124] In some implementations, constructing QA question pairs based on triplet event units includes: obtaining a preset QA question pair template; using the event type and event value in the target triplet event unit as questions in the preset QA question pair template to obtain an initial question-answer pair; calling a second-large model to semantically rewrite the answers in the initial question-answer pair to obtain rewritten answers; wherein the rewritten answers have medical semantic consistency with the answers in the initial question-answer pair; the rewriting prompts at least include role settings, medical fact locking constraints, sample examples, and rewriting instructions; combining the target triplet event unit and the rewritten answers to obtain the QA question pair of the target triplet event unit; wherein the target triplet event unit is any triplet event unit.
[0125] In some implementations, semantic encoding of triplet event units is performed to obtain clinical event encoding features, including: combining all triplet event units of a patient to obtain a clinical event flow; inputting each triplet event unit in the clinical event flow into a pre-trained encoder, and modeling and integrating the internal semantics of the event through a multi-layer attention mechanism to obtain high-dimensional continuous clinical event encoding features.
[0126] In some implementations, natural language prompts are constructed using QA question pairs, including: determining a role definition based on the target QA question pair; the role definition includes the identity definition of an AI reasoning assistant with clinical disease-specific medical expertise; determining input information guidance; the input information guidance includes at least a prompt and placeholders, the content of which is determined based on the event type and event value of the target QA question pair; determining clinical goal constraints; the clinical goal constraints include clinical feature extraction goal constraints, which at least include clinical indicator types and time nodes; determining output format constraints; and combining the role definition, input information guidance, clinical goal constraints, and output format constraints to obtain the natural language prompts for the target QA question pair; wherein, the target QA question pair is any QA question pair.
[0127] In some implementations, a linear projector is used to project clinical event coding features, including: inputting the clinical event coding features into the linear projector; transforming the clinical event coding features using the weight matrix of the linear projector to obtain projected values of the clinical event coding features; updating the weight matrix in response to determining that the projected values of the clinical event coding features and the instruction embedding expression do not meet the alignment target; iteratively performing the steps of transforming the clinical event coding features again using the updated weight matrix to obtain updated projected values of the clinical event coding features, and updating the weight matrix again in response to determining that the updated projected values of the clinical event coding features and the instruction embedding expression do not meet the alignment target, until the updated projected values of the clinical event coding features and the instruction embedding expression meet the alignment target; wherein, the alignment target includes dimensional alignment and semantic alignment.
[0128] In some implementations, the first large model includes at least a classification head, which determines clinical disease prediction results based on a bimodal hybrid embedding sequence; the first large model is trained using a two-stage training mechanism; wherein the first training stage uses natural language prompts from the clinical disease domain to perform domain-adaptive training on the general large model, and the second training stage uses EHR data from the clinical disease domain to perform task-oriented training on the classification head; the first large model uses cross-entropy loss as the loss function during the training stage, and updates the model parameters using backpropagation until convergence, and the adjustment parameters when updating the model parameters include at least one of the following: learning rate, number of training epochs, feature dimension of the model, number of encoder layers, or number of attention heads.
[0129] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0130] Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of this application. Figure 4As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, it implements the steps in the various method embodiments described above. Alternatively, when the processor 401 executes the computer program 403, it implements the functions of each module / unit in the various device embodiments described above.
[0131] Electronic device 4 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 4 may include, but is not limited to, processor 401 and memory 402. Those skilled in the art will understand that... Figure 4 This is merely an example of electronic device 4 and does not constitute a limitation on electronic device 4. It may include more or fewer components than shown, or different components.
[0132] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0133] The memory 402 can be an internal storage unit of the electronic device 4, such as a hard disk or RAM of the electronic device 4. The memory 402 can also be an external storage device of the electronic device 4, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 4. The memory 402 can also include both internal and external storage units of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device.
[0134] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0135] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.
[0136] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A clinical disease prediction method based on large model instruction prompts, characterized in that, include: Acquire structured electronic health records (EHRs) for clinical diseases; Extracting triplet event units from clinical disease-specific EHR data; The triplet event unit includes a unique identifier, an event type, and an event value; Semantic encoding is performed on the triplet event units to obtain clinical event coding features; QA question pairs are constructed based on the triplet event units, and natural language prompts are constructed using the QA question pairs. The natural language prompts are embedded and encoded to obtain the embedded expression of the instructions; A linear projector is used to project the clinical event coding features to perform dimensional and semantic alignment between the projected clinical event coding features and the instruction embedding expression. Dimensional alignment means mapping the clinical event coding features to a feature dimension space that is completely consistent with the instruction embedding expression. Semantic alignment means matching the semantic distribution of the clinical event coding features with the inference space where the instruction embedding expression is located, so as to ensure that the core semantics of the clinical event coding features are consistent with the task intent contained in the instruction embedding expression. The projected clinical event coding features and instruction embedding expressions are sequentially concatenated to obtain a dual-modal hybrid embedding sequence. The first major model is used to perform predictions based on the bimodal hybrid embedding sequence to obtain clinical disease prediction results; The construction of QA question pairs based on the triplet event unit includes: Get the preset QA question template; Using the event type and event value in the target triplet event unit as the questions in the preset QA question pair template, the initial question-answer pairs are obtained; The second major model is invoked to semantically rewrite the answers in the initial question-answer pair, resulting in rewritten answers; wherein, the rewritten answers have medical semantic consistency with the answers in the initial question-answer pair; the rewriting prompts include at least role settings, medical fact locking constraints, sample examples, and rewriting instructions; The target triplet event unit and the rewritten answer are combined to obtain the QA question pair of the target triplet event unit; Wherein, the target triplet event unit is any triplet event unit; When medical knowledge is updated, the sample examples in the rewrite prompts are modified to complete the update of the second major model knowledge.
2. The clinical disease prediction method based on large model instruction prompts according to claim 1, characterized in that, Triple event units were extracted from clinical disease-specific EHR data, including: Preprocessing of clinical disease EHR data; the preprocessing includes at least time alignment and outlier removal; The clinical disease EHR data are integrated by multi-table association using a unique identifier, and the data in each table is split field by field to obtain the triplet event unit; The unique identifier includes at least one of the patient's unique identifier and the hospital admission unique identifier.
3. The clinical disease prediction method based on large model instruction prompts according to claim 1, characterized in that, Semantic encoding of the triplet event units yields clinical event coding features, including: Combining all triplet event units of a patient yields the clinical event flow; Each triplet event unit in the clinical event stream is input into a pre-trained encoder, and the internal semantics of the events are modeled and integrated through a multi-layer attention mechanism to obtain high-dimensional continuous clinical event coding features.
4. The clinical disease prediction method based on large model instruction prompts according to claim 1, characterized in that, Constructing natural language prompts using the aforementioned QA questions includes: The role definition is determined based on the target QA question; the role definition includes the identity definition of an AI reasoning assistant with clinical disease-specific medical expertise; The input information guidance is determined; the input information guidance includes at least a guiding statement and placeholders, the content of which is determined based on the event type and event value of the target QA question pair; Define clinical objectives; the clinical objectives include clinical feature extraction objective constraints, which at least include clinical indicator types and time points. Define output format constraints; By combining role definition, input information guidance, clinical goal limitation, and output format constraint, natural language prompts for target QA question pairs are obtained; The target QA question pair is any QA question pair.
5. The clinical disease prediction method based on large model instruction prompts according to claim 1, characterized in that, Projecting clinical event coding features using a linear projector includes: Input the coded features of clinical events into a linear projector; The clinical event coding features are transformed using the weight matrix of a linear projector to obtain the projected values of the clinical event coding features; In response to the determination that the projection values of clinical event coding features and instruction embedding expressions do not meet the alignment target, the weight matrix is updated; The process iteratively executes the steps of transforming the clinical event coding features again with the updated weight matrix to obtain the updated clinical event coding feature projection values, and updating the weight matrix again in response to the determination that the updated clinical event coding feature projection values and the instruction embedding expression do not meet the alignment target, until the updated clinical event coding feature projection values and the instruction embedding expression meet the alignment target. The alignment targets include dimensional alignment and semantic alignment.
6. The clinical disease prediction method based on large model instruction prompts according to claim 1, characterized in that, The first major model includes at least a classification head, which determines clinical disease prediction results based on a bimodal hybrid embedding sequence; The first large model was trained using a two-stage training mechanism; The first training phase uses natural language prompts from the clinical disease domain to perform domain-adaptive training on the general large model, while the second training phase uses EHR data from the clinical disease domain to perform task-oriented training on the classification head. The first large model uses cross-entropy loss as the loss function during the training phase and uses backpropagation to update the model parameters until convergence. The adjustment parameters when updating the model parameters include at least one of the following: learning rate, number of training rounds, feature dimension of the model, number of encoder layers, or number of attention heads.
7. A clinical disease prediction device based on large model instruction prompts, characterized in that, include: The acquisition module is configured to acquire structured electronic health records (EHRs) for clinical diseases. The extraction target is configured to extract triplet event units from clinical disease EHR data; the triplet event unit includes a unique identifier, an event type, and an event value. A semantic encoding module is configured to perform semantic encoding on the triplet event units to obtain clinical event encoding features; The construction module is configured to construct QA question pairs based on the triplet event units and to construct natural language prompts using the QA question pairs; An embedding encoding module is configured to embed the natural language prompt instructions to obtain an instruction embedding expression; The bimodal alignment module is configured to project clinical event coding features using a linear projector to perform dimensional and semantic alignment between the projected clinical event coding features and the instruction embedding expression. Dimensional alignment refers to mapping the clinical event coding features to a feature dimension space that is completely consistent with the instruction embedding expression, and semantic alignment refers to matching the semantic distribution of the clinical event coding features with the inference space where the instruction embedding expression is located, so as to ensure that the core semantics of the clinical event coding features are consistent with the task intent contained in the instruction embedding expression. The dual-modal alignment module is also configured to sequentially concatenate the projected clinical event coding features and instruction embedding expressions to obtain a dual-modal hybrid embedding sequence; The prediction module is configured to use the first major model to make predictions based on the bimodal hybrid embedding sequence to obtain clinical disease prediction results; The construction of QA question pairs based on the triplet event unit includes: Get the preset QA question template; Using the event type and event value in the target triplet event unit as the questions in the preset QA question pair template, the initial question-answer pairs are obtained; The second major model is invoked to semantically rewrite the answers in the initial question-answer pair, resulting in rewritten answers; wherein, the rewritten answers have medical semantic consistency with the answers in the initial question-answer pair; the rewriting prompts include at least role settings, medical fact locking constraints, sample examples, and rewriting instructions; The target triplet event unit and the rewritten answer are combined to obtain the QA question pair of the target triplet event unit; Wherein, the target triplet event unit is any triplet event unit; When medical knowledge is updated, the sample examples in the rewrite prompts are modified to complete the update of the second major model knowledge.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the clinical disease prediction method based on large model instruction prompts as described in any one of claims 1 to 6.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the clinical disease prediction method based on large model instruction prompts as described in any one of claims 1 to 6.