An individualized drug efficacy and safety evaluation method and system based on digital twinning and causal reinforcement learning

Abstract

The present application relates to the technical field of digital medicine and drug evaluation, and particularly relates to an individualized drug efficacy and safety evaluation method and system based on digital twinning and causal reinforcement learning. Through a patient digital twinning model, individualized counterfactual prediction is realized, which can evaluate the potential efficacy and adverse event risk of different drug regimens at the individual patient level. Meanwhile, off-policy causal reinforcement learning and double robust estimation methods are used to evaluate the individualized value of long-term sequential drug strategies from observational real-world data, improving the rigor of causal inference. In addition, a combined efficacy-safety value function is constructed, which unifies efficacy benefits, safety risks, and compliance costs into the same decision framework, enabling multi-objective joint optimization. Finally, an interpretable analysis report is provided, including key features, value comparisons, and risk analyses, which enhances clinicians' understanding and trust in the recommended results, facilitating the application in clinical practice.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital medical and drug evaluation, and particularly relates to an individualized drug efficacy and safety evaluation method and system based on digital twinning and causal reinforcement learning. BACKGROUND

[0002] Individualized precision medicine is an important direction of modern medicine development. Traditional drug evaluation is mainly based on randomized controlled trial (RCT), which evaluates the average treatment effect of the group. However, RCT has limitations such as limited sample size, strict inclusion and exclusion criteria, and short follow-up time. Moreover, the results reflect the average effect of the population, which is difficult to directly apply to individual patient treatment decisions. Due to differences in genotype, phenotype, medical history, and combined medication, different patients may have significant differences in response to the same drug.

[0003] Real-world data (RWD) is derived from electronic medical records, medical insurance databases, patient registration systems, and covers a wider range of patient populations and longer follow-up times, providing the possibility for individualized treatment decisions. However, RWD has problems such as selection bias, confounding factors, data missing, and measurement errors. How to make causal inferences from observational data and accurately evaluate the individualized efficacy and safety of a specific patient receiving a specific medication regimen is a major challenge currently faced.

[0004] Existing individualized treatment methods mainly include propensity score matching, instrumental variable method, etc., but these methods can usually only evaluate the average treatment effect at a certain point in time, and are difficult to capture the dynamics and long-term cumulative effects of medication decisions. In addition, most existing methods only focus on efficacy evaluation, and lack systematic prediction of drug safety, especially adverse event risk, leading to a separation between efficacy and safety evaluation, and the inability to achieve the optimal trade-off between efficacy and safety.

[0005] In recent years, artificial intelligence technology, especially deep learning and reinforcement learning, has made significant progress in the medical field. Digital twinning (Digital Twin) technology can simulate patient outcomes under different treatment regimens by building a virtual replica of the patient, providing support for individualized treatment decisions. Causal reinforcement learning (Causal Reinforcement Learning) combines the advantages of causal inference and reinforcement learning, and can learn the optimal treatment strategy from observational data and evaluate the long-term cumulative return.

[0006] However, there is currently a lack of systematic methods to organically combine digital twins with causal reinforcement learning to achieve personalized integrated evaluation of drug efficacy and safety. Furthermore, existing methods suffer from insufficient interpretability, making it difficult for clinicians to understand and trust the recommendations of AI systems, thus limiting their application in clinical practice.

[0007] To address the above issues, there is an urgent need in this field for an innovative personalized medicine evaluation method that can: (1) perform counterfactual prediction based on patient digital twins to comprehensively assess the potential outcomes of specific patients receiving different medication regimens; (2) use causal reinforcement learning for off-strategy evaluation to accurately estimate the long-term value of personalized treatment from observational data; (3) simultaneously consider efficacy benefits, safety risks, and adherence costs within a unified framework to achieve joint optimization of efficacy and safety; and (4) provide highly interpretable evaluation criteria to enhance clinical acceptability. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a personalized drug efficacy and safety evaluation method based on digital twins and causal reinforcement learning, comprising the following steps:

[0009] S1. By collecting data from multiple sources, structured clinical indicator data, medical order data, imaging and laboratory summary data, and free text data of the target patients are obtained respectively, and a multimodal real-world dataset of the target patients is constructed.

[0010] S2. Based on a multimodal real-world dataset, construct a digital twin model of the target patient using a temporal Transformer architecture;

[0011] S3. Based on a digital twin model, counterfactual outcome prediction is performed on multiple candidate drug regimens for the target patient to obtain the counterfactual outcome prediction results for each candidate drug regimen. The counterfactual outcome prediction results include efficacy index prediction data and adverse event prediction data.

[0012] S4. Construct a policy-dependent causal reinforcement learning model based on historical real-world data, evaluate the causal effects of historical medication strategies through a double robust estimation algorithm, and construct an individualized long-term value function for each candidate medication regimen.

[0013] S5. Based on the counterfactual outcome prediction results and the individualized long-term value function, construct a joint value evaluation model for drug efficacy and safety;

[0014] S6. Based on the drug efficacy and safety combined value evaluation model, with the highest drug efficacy and safety for the target patients as the optimization objective, the combined value of each candidate drug regimen is quantitatively calculated to obtain the drug efficacy and safety evaluation results. The drug regimen with the highest combined value is determined as the recommended regimen, and the corresponding drug results and its interpretability analysis report are output.

[0015] Based on the above methods, the present invention also provides a personalized drug efficacy and safety evaluation system based on digital twins and causal reinforcement learning. The system is implemented based on any one of the above-mentioned personalized drug efficacy and safety evaluation methods based on digital twins and causal reinforcement learning, including a data acquisition module, used to acquire structured clinical indicator data, medical order data, imaging test summary data and free text data of the target patient through multi-source data acquisition, and construct a multimodal real-world dataset of the target patient.

[0016] The digital twin model building module is used to build a digital twin model of the target patient based on a time-series Transformer architecture using a multimodal real-world dataset.

[0017] The counterfactual outcome prediction module is used to predict the counterfactual outcome of multiple candidate drug regimens for a target patient based on a digital twin model, and to obtain the counterfactual outcome prediction results for each candidate drug regimen. The counterfactual outcome prediction results include efficacy index prediction data and adverse event prediction data.

[0018] The causal evaluation module is used to build a policy-dependent causal reinforcement learning model based on historical real-world data. It uses a double robust estimation algorithm to evaluate the causality of historical medication strategies and constructs an individualized long-term value function for each candidate medication regimen.

[0019] The joint decision-making module is used to construct a joint value evaluation model for drug efficacy and safety based on the counterfactual outcome prediction results and the individualized long-term value function. Based on the joint value evaluation model for drug efficacy and safety, the module calculates the maximum joint value of drug efficacy and safety with the objective of maximizing the function value, and obtains the optimal medication regimen for the target patient.

[0020] The joint decision-making module also includes a value function integration unit, used to integrate the counterfactual outcome prediction of the digital twin module and the long-term value function of the causal assessment module; a multi-objective optimization unit, used to construct a joint efficacy-safety value function and search for the optimal solution; and an interpretability generation unit, used to generate an evaluation report containing key features, value comparisons, and risk analysis.

[0021] The beneficial effects of this invention are as follows: It achieves personalized counterfactual prediction through a patient digital twin model, enabling the assessment of the potential efficacy and adverse event risks of different medication regimens at the individual patient level; simultaneously, it employs off-policy causal reinforcement learning and bi-robust estimation methods to evaluate the personalized value of long-term sequential medication strategies from observational real-world data, improving the rigor of causal inference; furthermore, it constructs a joint efficacy-safety value function, unifying efficacy benefits, safety risks, and adherence costs into a single decision framework to achieve multi-objective joint optimization; finally, it provides an interpretable analysis report containing key features, value comparisons, and risk analysis, enhancing clinicians' understanding and trust in the recommendation results, and facilitating its widespread application in clinical practice. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of a personalized drug efficacy and safety evaluation method based on digital twins and causal reinforcement learning, according to an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of the structure of a personalized drug efficacy and safety evaluation system based on digital twins and causal reinforcement learning, according to an embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram of the terminal device structure of a personalized drug efficacy and safety evaluation method based on digital twins and causal reinforcement learning according to an embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram of a computer-readable storage medium structure for a personalized drug efficacy and safety evaluation method based on digital twins and causal reinforcement learning, according to an embodiment of the present invention.

[0026] In the diagram, 200 is the terminal device, 210 is the memory, 211 is the RAM, 212 is the cache memory, 213 is the ROM, 214 is the program / utility, 215 is the program module, 220 is the processor, 230 is the bus, 240 is the external device, 250 is the I / O interface, 260 is the network adapter, and 300 is the program product. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention; that is, the described embodiments are only a part of the embodiments of the invention, and not all of them. The components of the embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0028] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0029] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or machine that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or machine. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or machine that includes said element.

[0030] Example 1:

[0031] like Figure 1 As shown, Embodiment 1 of the present invention provides a method for evaluating the efficacy and safety of drugs based on digital twins and causal reinforcement learning, comprising the following steps:

[0032] S1. By collecting data from multiple sources, structured clinical indicator data, medical order data, imaging and laboratory summary data, and free text data of the target patients are obtained respectively, and a multimodal real-world dataset of the target patients is constructed.

[0033] S2. Based on a multimodal real-world dataset, construct a digital twin model of the target patient using a temporal Transformer architecture;

[0034] S3. Based on a digital twin model, counterfactual outcome prediction is performed on multiple candidate drug regimens for the target patient to obtain the counterfactual outcome prediction results for each candidate drug regimen. The counterfactual outcome prediction results include efficacy index prediction data and adverse event prediction data.

[0035] S4. Construct a policy-dependent causal reinforcement learning model based on historical real-world data, evaluate the causal effects of historical medication strategies through a double robust estimation algorithm, and construct an individualized long-term value function for each candidate medication regimen.

[0036] S5. Based on the counterfactual outcome prediction results and the individualized long-term value function, construct a joint value evaluation model for drug efficacy and safety;

[0037] S6. Based on the drug efficacy and safety combined value evaluation model, with the highest drug efficacy and safety for the target patients as the optimization objective, the combined value of each candidate drug regimen is quantitatively calculated to obtain the drug efficacy and safety evaluation results. The drug regimen with the highest combined value is determined as the recommended regimen, and the corresponding drug results and its interpretability analysis report are output.

[0038] Specifically, multimodal real-world data of the target patient is acquired. This multimodal real-world data includes structured clinical indicator data, medical order data, imaging and laboratory summary data, and free text data. The data is aligned and resampled along the treatment timeline to provide time-series input for digital twin construction and causal reinforcement learning. This embodiment demonstrates the application of this method in the oncology department of a tertiary hospital to evaluate the individualized efficacy and safety of an immune checkpoint inhibitor combined with chemotherapy regimen in patients with advanced non-small cell lung cancer (NSCLC). First, multimodal real-world data of the target patient is acquired. The target patient is a 65-year-old male NSCLC patient, EGFR / ALK negative, PD-L1 expression ≥50%, PS score of 1, and a history of mild COPD. Data from the past two years was extracted from the hospital's EMR system, including: Structured clinical indicators: age, sex, weight, height, BMI, complete blood count (white blood cells, neutrophils, hemoglobin, platelets), liver and kidney function (ALT, AST, creatinine, blood urea nitrogen), electrocardiogram (heart rate, QTc interval), pulmonary function (FEV1, FVC), etc., totaling more than 50 indicators, recorded once per visit to form time-series data. Medical order data: previously used medications (resistance to gefitinib after 12 months of first-line treatment, followed by pemetrexed + carboplatin chemotherapy), current medications (COPD-related medication: salbutamol inhaler), dosage, frequency, and route of administration. Imaging and laboratory summary: chest CT report text describing tumor size, location, and metastasis; pathology report describing histological type and differentiation degree; gene testing report listing driver gene mutations. Free text: Medical records documenting the patient's chief complaint, symptoms, signs, and the doctor's diagnostic and treatment decision-making process; discharge summary summarizing the treatment process and effects during hospitalization. After data collection, de-identification processing is performed, removing patient names and ID numbers, generalizing birth dates to age, and generalizing specific addresses to province and city levels, thus constructing a multimodal real-world dataset for the patient.

[0039] Furthermore, step S2 includes the following sub-steps:

[0040] S201. Obtain the multimodal real-world dataset obtained in step S1, and extract the structured clinical indicator data, medical order data, image examination summary data and free text data of the target patient respectively;

[0041] S202. Perform time-series feature encoding on the structured clinical indicator data of the target patient to obtain the first feature vector sequence;

[0042] S203. Perform entity recognition and measurement normalization processing on the medical order data of the target patient to obtain the second feature vector sequence;

[0043] S204. Using natural language processing algorithms, semantic feature recognition is performed on the image examination summary data and free text data of the target patient to obtain a third feature vector sequence;

[0044] S205. Based on the first feature vector sequence, the second feature vector sequence, and the third feature vector sequence, perform multimodal feature fusion to obtain the multimodal fused feature vector data of the target patient;

[0045] S206. Based on multimodal fusion feature data, and using a temporal Transformer architecture, learn the temporal state evolution pattern of the target patient's multimodal real-world data through a self-attention mechanism, and construct a digital twin model of the target patient.

[0046] Specifically, the implementation flow of each sub-step in the above embodiments is as follows:

[0047] First, the structured clinical indicator data is time-series encoded to obtain a first feature vector sequence. The structured clinical indicator data includes, but is not limited to, physiological indicators such as patient age, gender, weight, blood pressure, heart rate, blood glucose, blood lipids, liver and kidney function indicators, and electrocardiogram parameters, as well as diagnostic codes (such as ICD-10 codes), past medical history, and family history. For continuous indicators, standardization or normalization is used; for categorical indicators, one-hot encoding or embedded vector representation is used. Time-series encoding uses positional encoding (PE) to capture temporal sequence information. Specifically, more than 50 indicators are arranged in chronological order to form a time-series matrix. ,in This represents the number of time steps, i.e., the current number of visits. Continuous indicators are standardized using Z-scores, and categorical indicators, such as gender and PS score, are encoded using one-hot encoding. Location encoding is added to indicate the time sequence, represented as: , ,in Indicates the time step position. Indicates the feature dimension index. This represents the total dimension of the features.

[0048] Next, the medical order data undergoes drug entity recognition and dosage normalization processing to obtain a second feature vector sequence. The medical order data includes information such as prescription drug name, dosage, frequency, route of administration, and duration of administration. Named Entity Recognition (NER) technology is used to extract the generic drug name and map it to a standard drug coding system (such as ATC coding or RxNorm coding). Dosage units are standardized and unified, converting different routes of administration and frequencies into daily dose equivalents. Drug embedding vectors are used to represent the characteristics of each drug. Specifically, the generic drug names (gefitinib, pemetrexed, carboplatin, salbutamol) are extracted and mapped to ATC codes (L01XE02, L01BA04, L01XA02, R03AC02). Daily dose equivalents are calculated; for example, pemetrexed 500mg / m² q3w is converted to a daily dose of approximately 167mg / m² / day. Each drug is represented as a 128-dimensional vector using a pre-trained drug embedding model (such as Mol2Vec).

[0049] Next, natural language processing is performed on the image examination summary data and free text data to extract clinical semantic features, resulting in a third feature vector sequence. The image examination summary data includes textual descriptions and structured fields (such as tumor size and lesion location) from imaging examination reports (e.g., CT, MRI, ultrasound). The free text data includes unstructured clinical text such as medical records, discharge summaries, and consultation opinions. Pre-trained clinical domain language models (e.g., BioBERT, ClinicalBERT) are used for text encoding to extract semantic feature vectors. For key clinical entities (e.g., symptoms, signs, diseases, treatments), named entity recognition and relation extraction techniques are used for structuring. Specifically, the BioBERT model is used to encode text such as CT reports, pathology reports, and medical records, extracting a 768-dimensional semantic feature vector. The named entity recognition model is used to extract key clinical entities, such as the right upper lung lesion, about 4.5cm×3.2cm in size, with unclear boundaries and accompanied by mediastinal lymph node enlargement. The structured information such as location (right upper lung), size (4.5×3.2cm) and metastasis (mediastinal lymph nodes) are extracted.

[0050] Then, the first feature vector sequence, the second feature vector sequence, and the third feature vector sequence are fused using multimodal methods and input into the temporal Transformer model. Multimodal fusion employs an attention fusion mechanism to learn the interaction relationships between features of different modalities. Specifically, the multimodal feature vectors at each time step are concatenated or weighted and summed to obtain a fused feature vector. This fused feature vector is input into the encoder part of the temporal Transformer model. Specifically, at each time step... By concatenating structured indicator features, drug embedding features, and text semantic features, a fused feature vector is obtained. (50+128+768=946 dimensions).

[0051] Finally, the patient's temporal state evolution pattern is learned through the self-attention mechanism of the temporal Transformer model, and a patient state transition function is established to obtain the digital twin model. The temporal Transformer model includes multiple self-attention layers and feedforward neural network layers. The self-attention mechanism can capture long-distance dependencies between different time steps and learn the dynamic evolution pattern of the patient's health state. The training objective of the digital twin model is to minimize the difference between the predicted state and the actual observed state. Specifically, the Transformer encoder contains 6 layers, each with 8 attention heads, and a hidden layer dimension of 512. The input is a fused feature sequence. Output the hidden state sequence at each time step. The decoder employs an autoregressive structure, based on... and the predicted state at the previous moment Predict the current state The training objective is to minimize the mean squared error (MSE) between the predicted state and the actual observed state. The model is pre-trained on historical data of 5,000 NSCLC patients from the past 5 years at the hospital, and then fine-tuned on data from the target patients to learn the individualized state evolution patterns of the patients, thus obtaining a digital twin model.

[0052] Furthermore, step S3 includes the following sub-steps:

[0053] S301: Obtain multiple candidate medication regimens for the target patient. Based on the digital twin model obtained in step S2, use the Monte Carlo sampling algorithm to sample and predict the target patient's current state and candidate medication regimens, obtaining the target patient's state evolution trajectory data corresponding to each candidate medication regimen; specifically...

[0054] S302: Based on the target patient state evolution trajectory data, the expected value and confidence interval of the efficacy index are calculated through statistical analysis, and the occurrence time data and occurrence probability data of adverse events in each target patient state evolution trajectory data are recorded to generate an adverse event risk distribution.

[0055] S305: Based on the expected value and confidence interval of the efficacy indicators and the risk distribution of adverse events, the counterfactual outcome prediction results are obtained.

[0056] Specifically, the implementation process of each sub-step in the above embodiments is as follows: First, the current state of the target patient and the candidate drug regimen are input into the digital twin model; then, multiple possible patient state evolution trajectories are generated using the Monte Carlo sampling method through the generative decoder of the digital twin model; next, statistical analysis is performed on the multiple patient state evolution trajectories to calculate the expected value and confidence interval of the efficacy indicators; the efficacy indicators include progression-free survival, overall survival, and remission rate; then, the occurrence time and probability of potential adverse events are identified based on the patient state evolution trajectories to generate an adverse event risk distribution; finally, the expected value and confidence interval of the efficacy indicators and the adverse event risk distribution are used as the counterfactual outcome prediction output.

[0057] Specifically, in this embodiment, the candidate treatment regimens include: Regimen A: Pembrolizumab monotherapy (200 mg q3w); Regimen B: Nivolumab + Ipilimumab dual immunotherapy (Nivolumab 3 mg / kg + Ipilimumab 1 mg / kg q3w×4, followed by Nivolumab 240 mg q2w maintenance); Regimen C: Pembrolizumab + pemetrexed + carboplatin immunotherapy combined with chemotherapy (Pembrolizumab 200 mg + pemetrexed 500 mg / m² + carboplatin AUC 5 q3w×4, followed by pembrolizumab + pemetrexed maintenance); Regimen D: Atezolizumab + bevacizumab + chemotherapy (Atezolizumab 1200 mg + Bevacizumab) 15 mg / kg + pemetrexed 500 mg / m² + carboplatin AUC 6 q3w×4, then Atezolizumab + Bevacizumab + pemetrexed for maintenance. For each candidate regimen, the patient's current status should be recorded. and medication regimen Inputting a digital twin model, Monte Carlo sampling was used to generate 1000 state evolution trajectories to simulate the treatment process over the next 2 years (approximately 24 months). Efficacy endpoints: progression-free survival (PFS), overall survival (OS), objective response rate (ORR), and disease control rate (DCR). Adverse events: Grade 3-4 immune-related adverse events (irAEs) such as pneumonia, enteritis, hepatitis, rash, and thyroid dysfunction; chemotherapy-related hematologic toxicities such as neutropenia and thrombocytopenia; bevacizumab-related bleeding, hypertension, and proteinuria. Taking scheme C as an example, the predicted results are as follows: median PFS: 8.2 months (95% CI: 6.8-9.9); median OS: 18.5 months (95% CI: 15.2-22.1); ORR: 45.2% (95% CI: 40.1-50.5); cumulative incidence of grade 3-4 irAEs: 18.5% (pneumonia 5.2%, enteritis 4.1%, hepatitis 3.3%, rash 2.8%, endocrine disorders 3.1%); cumulative incidence of grade 3-4 hematologic toxicities: 35.7% (neutropenia 28.3%, thrombocytopenia 12.4%), where CI represents the confidence interval.

[0058] Finally, efficacy-safety prediction reports for each regimen are generated and presented in visual charts (Kaplan-Meier survival curves, cumulative incidence of adverse events curves).

[0059] Furthermore, step S4 includes the following sub-steps:

[0060] S401: Based on the multimodal real-world dataset of the target patient, extract medication trajectory data of multiple historical reference patients from the historical real-world database. The medication trajectory data of the historical reference patients includes a state sequence composed of patient clinical data, an action sequence composed of patient medication adjustments, and a reward sequence composed of patient efficacy feedback.

[0061] S402: Based on historical reference patients' medication trajectory data, a propensity score model is used to predict the probability value of the target patient choosing a specific medication adjustment action in each state, thus obtaining a propensity score;

[0062] S403: Based on the value function estimation model, predict the expected long-term return corresponding to each state of the target patient and the medication adjustment action to obtain the value function estimate;

[0063] S404: Based on the propensity score and value function estimation, the departure strategy assessment value of the target patient is calculated using a double robust estimation formula;

[0064] S405: Based on the off-strategy assessment value, through importance sampling correction and confidence lower bound calculation, the individualized long-term value function and strategy risk upper bound of the candidate drug regimen for the target patient are obtained.

[0065] Furthermore, the doubly robust estimation formula is expressed as:

[0066] ;

[0067] in, This indicates the strategic value of dual robust estimation. Indicates the number of samples. Indicates the target policy in state Select action The probability, The rating indicates the tendency towards certain behavioral strategies. This indicates the actual observation report. This represents the estimation of the value function. It represents the expected value under the target strategy.

[0068] Specifically, the implementation process of each sub-step in the above embodiment is as follows: First, extract medication trajectory data of multiple historical reference patients from a historical real-world database. The medication trajectory data of each historical reference patient includes a state sequence, an action sequence, and a reward sequence. Next, construct a propensity score model to estimate the probability of choosing a specific medication action in each state, and obtain a propensity score. Then, construct a value function estimation model to predict the expected long-term return of each state-action pair, and obtain a value function estimate. Then, use a double robust estimation formula to combine the propensity score and the value function estimate to calculate the policy evaluation value. Finally, through importance sampling correction and confidence lower bound calculation, output the individualized long-term value function and policy risk upper bound of the candidate medication plan.

[0069] Specifically, in this embodiment, historical medication history is first extracted. Data from 3000 NSCLC patients who previously received immunotherapy ± chemotherapy are extracted from the hospital database. Each patient is followed up for an average of 15 months, and their status (clinical indicators, symptoms, tumor measurements), actions (medication regimen adjustments), and rewards (efficacy improvement +1, disease progression -2, serious adverse events -1, death -5) are recorded at each time point. Next, a propensity score model is trained. Using a logistic regression model, with patient status (age, gender, PS score, PD-L1 expression, driver genes, number of previous lines of treatment, liver and kidney function, comorbidities, etc.) as independent variables, the probability of selecting each medication regimen is predicted. The model achieved an AUC of 0.78 on the validation set, indicating that it effectively captured the patterns of doctors' medication decisions. Next, a value function estimation model was trained. A deep Q-network (DQN) was used, with the input state... and actions Output Q value This represents the expected long-term survival benefit after taking this action from this state. The network structure is a 3-layer fully connected neural network (256-128-64), using Mean Squared Bellman Error as the loss function, and is stabilized using empirical replay and target network techniques. Then, a double robust estimation is performed. For candidate C (Pembrolizumab + chemotherapy), its policy evaluation value is calculated: ;

[0070] in, The policy probability of option C (if state) If the indication meets the criteria for scheme C, the score is 1; otherwise, it is 0. Assess the tendency to adopt historical behavioral strategies. For patients The actual cumulative survival benefit Let C represent the expected value. The long-run value function of C is calculated. (Survival benefit in months). Final confidence interval and upper bound of risk. Using bootstrap resampling 1000 times, the 95% confidence interval for the double robust estimate was calculated to be [10.8, 14.3]. The CVaR method was used to calculate the upper bound of risk at the 5th percentile, i.e., the average survival benefit in the worst-case 5% scenario is 6.2 months, suggesting that some patients may experience poor efficacy or adverse events leading to early discontinuation of treatment. Causal assessments were performed on the four candidate options to obtain the long-term value function: Option A: =10.2 (95% CI: 8.5-12.1); Scheme B: =11.8 (95% CI: 9.7-14.2); Scheme C: =12.5 (95% CI: 10.8-14.3); Scheme D: =13.1 (95% CI: 11.0-15.4), where CI represents the confidence interval.

[0071] Furthermore, the combined efficacy and safety value evaluation model for drugs described in step S5 is expressed as follows:

[0072] ;

[0073] in, This represents the combined efficacy-safety value function. This represents the therapeutic value function. Represents the safety value function (based on adverse event risk score). This represents the cost of patient compliance, with α, β, and γ being weighted coefficients set according to clinical priority.

[0074] Specifically, based on the counterfactual outcome prediction results and the individualized long-term value function, a joint value evaluation model for drug efficacy and safety is constructed. In this embodiment, clinical priority is given to survival benefit, therefore, weights are set accordingly. , Compliance costs Based on a comprehensive assessment of factors such as medication complexity, infusion time, and adverse reaction burden: Option A (single-drug): (Relatively simple, q3w infusion, with a lower risk of adverse reactions); Regimen B (dual immunization): (Requires two drugs in combination, with a higher initial dosing frequency); Regimen C (immunotherapy + chemotherapy): (Chemotherapy increases hematological toxicity and adverse reactions such as nausea and vomiting); Regimen D (immunotherapy + anti-angiogenic therapy + chemotherapy): (The most complex regimen, with the heaviest adverse reaction burden). Based on the above parameters, the combined value of each regimen was calculated using the drug efficacy and safety combined value evaluation model:

[0075] ;

[0076] ;

[0077] ;

[0078] .

[0079] Furthermore, the interpretability analysis report in step S6 includes: identifying key temporal features affecting the prediction results in the digital twin model through an attention mechanism; explaining the long-term value advantage of the recommended scheme compared to other schemes based on a causal reinforcement learning model; and providing comparative analysis results of efficacy-safety trade-offs for different candidate schemes, including expected efficacy indicators, major adverse event risks and their confidence intervals.

[0080] Specifically, based on the combined efficacy and safety value evaluation model for drugs, regimen D (Atezolizumab + Bevacizumab + chemotherapy) The maximum value is 12.79, therefore it is selected as the recommended regimen. An evaluation report is generated, including: Recommended regimen: Atezolizumab + Bevacizumab + Pemetrexed + Carboplatin; Expected efficacy: Median PFS approximately 9.5 months, median OS approximately 20.3 months, ORR approximately 50%; Major risks: Grade 3-4 irAEs approximately 20%, hematologic toxicity approximately 38%, bleeding and hypertension risk approximately 25%; Long-term value: Expected survival benefit approximately 13.1 months (considering efficacy) or 12.79 months (considering efficacy, safety, and compliance); Comparison with other regimens: Regimen D is more effective than regimens A, B, and C, but has a relatively higher adverse event burden; If patients cannot tolerate or refuse complex regimens, regimen C (immunotherapy + chemotherapy) can be considered as a second-best option. The report is presented in structured tables and visual charts for easy understanding and decision-making by clinicians.

[0081] Example 2

[0082] like Figure 2 As shown in Example 1, Example 2 of this invention proposes a personalized drug efficacy and safety evaluation system based on a digital twin and causal reinforcement learning method.

[0083] Specifically, the system includes a data acquisition module, which is used to acquire structured clinical indicator data, medical order data, imaging and laboratory summary data and free text data of the target patient through multi-source data acquisition, and construct a multimodal real-world dataset of the target patient;

[0084] The digital twin model building module is used to build a digital twin model of the target patient based on a time-series Transformer architecture using a multimodal real-world dataset.

[0085] The counterfactual outcome prediction module is used to predict the counterfactual outcome of multiple candidate drug regimens for a target patient based on a digital twin model, and to obtain the counterfactual outcome prediction results for each candidate drug regimen. The counterfactual outcome prediction results include efficacy index prediction data and adverse event prediction data.

[0086] The causal evaluation module is used to build a policy-dependent causal reinforcement learning model based on historical real-world data. It uses a double robust estimation algorithm to evaluate the causality of historical medication strategies and constructs an individualized long-term value function for each candidate medication regimen.

[0087] The joint decision-making module is used to construct a joint value evaluation model for drug efficacy and safety based on the counterfactual outcome prediction results and the individualized long-term value function. Based on the joint value evaluation model for drug efficacy and safety, the module calculates the maximum joint value of drug efficacy and safety with the objective of maximizing the function value, and obtains the optimal medication regimen for the target patient.

[0088] The joint decision-making module also includes a value function integration unit, used to integrate the counterfactual outcome prediction of the digital twin module and the long-term value function of the causal assessment module; a multi-objective optimization unit, used to construct a joint efficacy-safety value function and search for the optimal solution; and an interpretability generation unit, used to generate an evaluation report containing key features, value comparisons, and risk analysis.

[0089] Furthermore, the digital twin model construction module also includes: a multimodal preprocessing unit for feature extraction and encoding of structured clinical indicator data, medical order data, imaging test summary data, and free text data; a temporal modeling unit for constructing a temporal state evolution model based on the Transformer architecture; a generative decoding unit for generating patient state evolution trajectories under different medication regimens; and a counterfactual reasoning unit for calculating efficacy indicator predictions and adverse event risk distributions.

[0090] Furthermore, the causal assessment module also includes: a data playback unit for extracting medication trajectory data from a historical real-world database; a propensity score estimation unit for estimating the probability of choosing a specific medication action under different states; a value function estimation unit for predicting the expected long-term return of state-action pairs; a dual robust estimation unit for combining propensity scores and value function estimation to perform off-policy assessment; and a confidence interval calculation unit for calculating the lower confidence bound of the policy value and the upper bound of the policy risk.

[0091] Specifically, the workflow of this system is as follows:

[0092] First, the data acquisition module acquires structured clinical indicator data, medical order data, imaging and laboratory summary data, and free text data of the target patient through multi-source data acquisition, and then constructs a multimodal real-world dataset of the target patient. Next, this multimodal real-world dataset is input into the digital twin model construction module. The multimodal preprocessing unit of this module first extracts and encodes features from the structured clinical indicator data, medical order data, imaging and laboratory summary data, and free text data. Then, the temporal modeling unit constructs a temporal state evolution model based on the Transformer architecture. Next, the generative decoding unit generates the patient state evolution trajectory under different medication regimens. Finally, the counterfactual reasoning unit calculates the efficacy indicator prediction and adverse event risk distribution, thus completing the construction of the digital twin model of the target patient. Next, the counterfactual outcome prediction module, based on the digital twin model, performs counterfactual outcome predictions for multiple candidate medication regimens of the target patient, obtaining the counterfactual outcome prediction results for each candidate medication regimen. These results include efficacy indicator prediction data and adverse event prediction data. Meanwhile, the causal assessment module constructs a policy-dependent causal reinforcement learning model based on historical real-world data. Its data replay unit first extracts medication trajectory data from the historical real-world database. Then, the propensity score estimation unit estimates the probability of choosing a specific medication action under different states. Next, the value function estimation unit predicts the expected long-term return of the state-action pair. Subsequently, the dual-robust estimation unit combines propensity score and value function estimation for policy-dependent assessment. Finally, the confidence interval calculation unit calculates the lower confidence bound of the policy value and the upper bound of the policy risk, thus performing a causal assessment of historical medication strategies through the dual-robust estimation algorithm and constructing individualized long-term value functions for each candidate medication plan. Then, the joint decision-making module, based on the counterfactual reversal prediction results and individualized long-term value functions, first integrates the counterfactual reversal prediction from the digital twin module and the long-term value function from the causal assessment module. Then, the multi-objective optimization unit constructs a joint efficacy-safety value function and searches for the optimal solution, calculating the maximum joint value of drug efficacy and safety with the objective of maximizing the function value, and obtaining the optimal medication plan for the target patient. Finally, the interpretability generation unit generates an evaluation report containing key features, value comparisons, and risk analysis.

[0093] Example 3

[0094] like Figure 3 As shown in Example 1, Example 3 proposes a terminal device for a personalized drug efficacy and safety evaluation method based on digital twins and causal reinforcement learning. The terminal device 200 includes at least one memory 210, at least one processor 220, and a bus 230 connecting different platform systems.

[0095] The memory 210 may include a readable medium in the form of volatile memory, such as RAM 211 and / or cache memory 212, and may further include ROM 213.

[0096] The memory 210 also stores a computer program that can be executed by the processor 220, causing the processor 220 to execute any of the above-described methods for evaluating the efficacy and safety of individualized drugs based on digital twins and causal reinforcement learning in this application embodiment. The specific implementation method and the achieved technical effects are consistent with those described in the embodiments of the above methods, and some details will not be repeated here. The memory 210 may also include a program / utility 214 having a set (at least one) of program modules 215. Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment.

[0097] Accordingly, processor 220 can execute the aforementioned computer program, as well as executable program / utility 214.

[0098] Bus 230 can represent one or more of several types of bus structures, including a memory bus or memory controller, peripheral bus, graphics acceleration port, processor, or a local bus using any of the various bus structures.

[0099] Terminal device 200 can also communicate with one or more external devices 240, such as keyboards, pointing devices, Bluetooth devices, etc., and with one or more devices capable of interacting with it, and / or with any device that enables it to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed via I / O interface 250. Furthermore, terminal device 200 can communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 260. Network adapter 260 can communicate with other modules of terminal device 200 via bus 230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with terminal device 200, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0100] Example 4

[0101] like Figure 4As shown in Example 1, this example proposes a computer-readable storage medium for a personalized drug efficacy and safety evaluation method based on digital twins and causal reinforcement learning. The computer-readable storage medium stores instructions that, when executed by a processor, implement any one of the aforementioned personalized drug efficacy and safety evaluation methods based on digital twins and causal reinforcement learning. The specific implementation method and the achieved technical effects are consistent with those described in the examples above, and some details will not be repeated.

[0102] The program product 300 provided in this embodiment for implementing the above method can be a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. However, the program product 300 of the present invention is not limited thereto. In this embodiment, the readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. The program product 300 can employ any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0103] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on a user computing device, partially on a user device, as a standalone software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to user computing devices via any type of network, including local area networks (LANs) or wide area networks (WANs), or they can be connected to external computing devices (e.g., via the Internet using an Internet service provider).

[0104] This invention is described from the perspectives of its intended use, effectiveness, progress, and novelty. Its practical and progressive features meet the functional enhancement and use requirements emphasized by the Patent Law. The above description and drawings are merely preferred embodiments of this application and are not intended to limit this application. Therefore, all structures, devices, features, etc., that are similar to or identical to those of this application, i.e., all equivalent substitutions or modifications made in accordance with the scope of this patent application, shall fall within the scope of protection of this patent application.

[0105] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A personalized drug efficacy and safety evaluation method based on digital twins and causal reinforcement learning, characterized in that, Includes the following steps: S1. By collecting data from multiple sources, structured clinical indicator data, medical order data, imaging and laboratory summary data, and free text data of the target patients are obtained respectively, and a multimodal real-world dataset of the target patients is constructed. S2. Based on a multimodal real-world dataset, construct a digital twin model of the target patient using a temporal Transformer architecture; S3. Based on a digital twin model, counterfactual outcome prediction is performed on multiple candidate drug regimens for the target patient to obtain the counterfactual outcome prediction results for each candidate drug regimen. The counterfactual outcome prediction results include efficacy index prediction data and adverse event prediction data. S4. Construct a policy-dependent causal reinforcement learning model based on historical real-world data, evaluate the causal effects of historical medication strategies through a double robust estimation algorithm, and construct an individualized long-term value function for each candidate medication regimen. S5. Based on the counterfactual outcome prediction results and the individualized long-term value function, construct a joint value evaluation model for drug efficacy and safety; S6. Based on the drug efficacy and safety combined value evaluation model, with the highest drug efficacy and safety for the target patients as the optimization objective, the combined value of each candidate drug regimen is quantitatively calculated to obtain the drug efficacy and safety evaluation results. The drug regimen with the highest combined value is determined as the recommended regimen, and the corresponding drug results and its interpretability analysis report are output.

2. The method for evaluating the efficacy and safety of drugs based on digital twins and causal reinforcement learning according to claim 1, characterized in that, Step S2 includes the following sub-steps: S201. Obtain the multimodal real-world dataset obtained in step S1, and extract the structured clinical indicator data, medical order data, image examination summary data and free text data of the target patient respectively; S202. Perform time-series feature encoding on the structured clinical indicator data of the target patient to obtain the first feature vector sequence; S203. Perform entity recognition and measurement normalization processing on the medical order data of the target patient to obtain the second feature vector sequence; S204. Using natural language processing algorithms, semantic feature recognition is performed on the image examination summary data and free text data of the target patient to obtain a third feature vector sequence; S205. Based on the first feature vector sequence, the second feature vector sequence, and the third feature vector sequence, perform multimodal feature fusion to obtain the multimodal fused feature vector data of the target patient; S206. Based on multimodal fusion feature data, and using a temporal Transformer architecture, learn the temporal state evolution pattern of the target patient's multimodal real-world data through a self-attention mechanism, and construct a digital twin model of the target patient.

3. The method for evaluating the efficacy and safety of drugs based on digital twins and causal reinforcement learning according to claim 1, characterized in that, Step S3 includes the following sub-steps: S301: Obtain multiple candidate medication regimens for the target patient. Based on the digital twin model obtained in step S2, use the Monte Carlo sampling algorithm to sample and predict the current state of the target patient and the candidate medication regimens, and obtain the target patient state evolution trajectory data corresponding to each candidate medication regimen. S302: Based on the target patient state evolution trajectory data, the expected value and confidence interval of the efficacy index are calculated through statistical analysis, and the occurrence time data and occurrence probability data of adverse events in each target patient state evolution trajectory data are recorded to generate an adverse event risk distribution. S305: Based on the expected value and confidence interval of the efficacy indicators and the risk distribution of adverse events, the counterfactual outcome prediction results are obtained.

4. The method for evaluating the efficacy and safety of drugs based on digital twins and causal reinforcement learning according to claim 1, characterized in that, Step S4 includes the following sub-steps: S401: Based on the multimodal real-world dataset of the target patient, extract medication trajectory data of multiple historical reference patients from the historical real-world database. The medication trajectory data of the historical reference patients includes a state sequence composed of patient clinical data, an action sequence composed of patient medication adjustments, and a reward sequence composed of patient efficacy feedback. S402: Based on historical reference patients' medication trajectory data, a propensity score model is used to predict the probability value of the target patient choosing a specific medication adjustment action in each state, thus obtaining a propensity score; S403: Based on the value function estimation model, predict the expected long-term return corresponding to each state of the target patient and the medication adjustment action to obtain the value function estimate; S404: Based on the propensity score and value function estimation, the departure strategy assessment value of the target patient is calculated using a double robust estimation formula; S405: Based on the off-strategy assessment value, through importance sampling correction and confidence lower bound calculation, the individualized long-term value function and strategy risk upper bound of the candidate drug regimen for the target patient are obtained.

5. The method for evaluating the efficacy and safety of drugs based on digital twins and causal reinforcement learning according to claim 4, characterized in that, The aforementioned double-robust estimation formula is expressed as follows: ； in, This indicates the strategic value of dual robust estimation. Indicates the number of samples. Indicates the target policy in state Select action The probability, The rating indicates the tendency towards certain behavioral strategies. This indicates the actual observation report. This represents the estimation of the value function. It represents the expected value under the target strategy.

6. The method for evaluating the efficacy and safety of drugs based on digital twins and causal reinforcement learning according to claim 1, characterized in that, The combined efficacy and safety value evaluation model for drugs described in step S5 is expressed as follows: ； in, This represents the combined value function of efficacy and safety. This represents the therapeutic value function. Represents the security value function. This represents the cost of patient compliance, with α, β, and γ being weighted coefficients set according to clinical priority.

7. The method for evaluating the efficacy and safety of drugs based on digital twins and causal reinforcement learning according to claim 1, characterized in that, The interpretability analysis report in step S6 includes: identifying key temporal features affecting prediction results in the digital twin model through attention mechanisms; explaining the long-term value advantages of the recommended scheme compared to other schemes based on a causal reinforcement learning model; and providing comparative analysis results of efficacy-safety trade-offs for different candidate schemes, including expected efficacy indicators, major adverse event risks and their confidence intervals.

8. A personalized drug efficacy and safety evaluation system based on digital twins and causal reinforcement learning, wherein the system is implemented based on the personalized drug efficacy and safety evaluation method based on digital twins and causal reinforcement learning as described in any one of claims 1-7, characterized in that, include: The data acquisition module is used to acquire structured clinical indicator data, medical order data, imaging and laboratory summary data, and free text data of the target patient through multi-source data acquisition, and to construct a multimodal real-world dataset of the target patient; The digital twin model building module is used to build a digital twin model of the target patient based on a time-series Transformer architecture using a multimodal real-world dataset. The counterfactual outcome prediction module is used to predict the counterfactual outcome of multiple candidate drug regimens for a target patient based on a digital twin model, and to obtain the counterfactual outcome prediction results for each candidate drug regimen. The counterfactual outcome prediction results include efficacy index prediction data and adverse event prediction data. The causal evaluation module is used to build a policy-dependent causal reinforcement learning model based on historical real-world data. It uses a double robust estimation algorithm to evaluate the causality of historical medication strategies and constructs an individualized long-term value function for each candidate medication regimen. The joint decision-making module is used to construct a joint value evaluation model for drug efficacy and safety based on the counterfactual outcome prediction results and the individualized long-term value function. Based on the joint value evaluation model for drug efficacy and safety, the module calculates the maximum joint value of drug efficacy and safety with the objective of maximizing the function value, and obtains the optimal medication regimen for the target patient. The joint decision-making module also includes a value function integration unit, used to integrate the counterfactual outcome prediction of the digital twin module and the long-term value function of the causal assessment module; a multi-objective optimization unit, used to construct a joint efficacy-safety value function and search for the optimal solution; and an interpretability generation unit, used to generate an evaluation report containing key features, value comparisons, and risk analysis.

9. A personalized drug efficacy and safety evaluation system based on digital twins and causal reinforcement learning according to claim 8, characterized in that, The digital twin model construction module also includes: a multimodal preprocessing unit for feature extraction and encoding of structured clinical indicator data, medical order data, imaging test summary data and free text data; a temporal modeling unit for constructing a temporal state evolution model based on the Transformer architecture; a generative decoding unit for generating patient state evolution trajectories under different medication regimens; and a counterfactual reasoning unit for calculating efficacy indicator predictions and adverse event risk distributions.

10. A personalized drug efficacy and safety evaluation system based on digital twins and causal reinforcement learning according to claim 8, characterized in that, The causal assessment module further includes: a data playback unit for extracting medication trajectory data from a historical real-world database; a propensity score estimation unit for estimating the probability of choosing a specific medication action under different states; a value function estimation unit for predicting the expected long-term return of state-action pairs; a dual robust estimation unit for combining propensity scores and value function estimation to perform off-policy assessment; and a confidence interval calculation unit for calculating the lower confidence bound of the policy value and the upper bound of the policy risk.

Images