Individualized drug curative effect and safety evaluation method and system based on digital twinborn and causal reinforcement learning

Abstract

The invention relates to the technical field of digital medical treatment and drug evaluation, in particular to an individualized drug curative effect and safety evaluation method and system based on digital twinborn and causal reinforcement learning. Individualized anti-fact prediction is realized through a patient digital twinning model, and potential curative effects and adverse event risks of different medication schemes can be evaluated on a single patient level. Meanwhile, an off-strategy causal reinforcement learning and dual-robust estimation method is adopted, the individualized value of a long-term sequential medication strategy is evaluated from observed real world data, and the preciseness of causal inference is improved; in addition, a curative effect-safety joint value function is constructed, curative effect benefits, safety risks and compliance cost are uniformly incorporated into the same decision framework, and multi-objective joint optimization is achieved; finally, an interpretability analysis report containing key features, value comparison and risk analysis is provided, the understanding and trust of clinicians on recommendation results are enhanced, and application and popularization in clinical practice are facilitated.

Description

Technical Field

[0001] This invention relates to the field of digital healthcare and drug evaluation technology, and in particular to a personalized drug efficacy and safety evaluation method and system based on digital twins and causal reinforcement learning. Background Technology

[0002] Personalized precision medicine is an important direction in the development of modern medicine. Traditional drug evaluation is mainly based on randomized controlled trials (RCTs) to assess the average treatment effect in a population. However, RCTs have limitations such as limited sample size, strict inclusion and exclusion criteria, and short follow-up periods. Furthermore, their results reflect the average effect in a population and are difficult to directly apply to individual patient treatment decisions. Different patients may respond significantly differently to the same drug due to differences in genotype, phenotype, medical history, and concomitant medications.

[0003] Real-world data (RWD), derived from electronic medical records, health insurance databases, and patient registration systems, covers a broader patient population and longer follow-up periods, enabling personalized treatment decisions. However, RWD suffers from selection bias, confounding factors, missing data, and measurement errors. A major challenge remains: how to make causal inferences from observational data and accurately assess the individualized efficacy and safety of specific medication regimens for particular patients.

[0004] Existing personalized treatment methods mainly include propensity score matching and instrumental variable methods, but these methods can usually only assess the average treatment effect at a certain point in time, making it difficult to capture the dynamics of medication decisions and the long-term cumulative effects. In addition, most existing methods only focus on efficacy evaluation and lack systematic prediction of drug safety, especially the risk of adverse events, resulting in a disconnect between efficacy and safety evaluation and failing to achieve the optimal trade-off between efficacy and safety.

[0005] In recent years, artificial intelligence technologies, especially deep learning and reinforcement learning, have made significant progress in the medical field. Digital twin technology, by constructing virtual replicas of patients, can simulate patient outcomes under different treatment plans, providing support for personalized treatment decisions. Causal reinforcement learning combines the advantages of causal inference and reinforcement learning, enabling it to learn optimal treatment strategies from observational data and evaluate long-term cumulative returns.

[0006] However, there is currently a lack of systematic methods to organically combine digital twins with causal reinforcement learning to achieve personalized integrated evaluation of drug efficacy and safety. Furthermore, existing methods suffer from insufficient interpretability, making it difficult for clinicians to understand and trust the recommendations of AI systems, thus limiting their application in clinical practice.

[0007] To address the above issues, there is an urgent need in this field for an innovative personalized medicine evaluation method that can: (1) perform counterfactual prediction based on patient digital twins to comprehensively assess the potential outcomes of specific patients receiving different medication regimens; (2) use causal reinforcement learning for off-strategy evaluation to accurately estimate the long-term value of personalized treatment from observational data; (3) simultaneously consider efficacy benefits, safety risks, and adherence costs within a unified framework to achieve joint optimization of efficacy and safety; and (4) provide highly interpretable evaluation criteria to enhance clinical acceptability. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a personalized drug efficacy and safety evaluation method based on digital twins and causal reinforcement learning, comprising the following steps: S1. By collecting data from multiple sources, structured clinical indicator data, medical order data, imaging and laboratory summary data, and free text data of the target patients are obtained respectively, and a multimodal real-world dataset of the target patients is constructed. S2. Based on a multimodal real-world dataset, construct a digital twin model of the target patient using a temporal Transformer architecture; S3. Based on a digital twin model, counterfactual outcome prediction is performed on multiple candidate drug regimens for the target patient to obtain the counterfactual outcome prediction results for each candidate drug regimen. The counterfactual outcome prediction results include efficacy index prediction data and adverse event prediction data. S4. Construct a policy-dependent causal reinforcement learning model based on historical real-world data, evaluate the causal effects of historical medication strategies through a double robust estimation algorithm, and construct an individualized long-term value function for each candidate medication regimen. S5. Based on the counterfactual outcome prediction results and the individualized long-term value function, construct a joint value evaluation model for drug efficacy and safety; S6. Based on the drug efficacy and safety combined value evaluation model, with the highest drug efficacy and safety for the target patients as the optimization objective, the combined value of each candidate drug regimen is quantitatively calculated to obtain the drug efficacy and safety evaluation results. The drug regimen with the highest combined value is determined as the recommended regimen, and the corresponding drug results and its interpretability analysis report are output.

[0009] Based on the above methods, the present invention also provides a personalized drug efficacy and safety evaluation system based on digital twins and causal reinforcement learning. The system is implemented based on any one of the above-mentioned personalized drug efficacy and safety evaluation methods based on digital twins and causal reinforcement learning, including a data acquisition module, used to acquire structured clinical indicator data, medical order data, imaging test summary data and free text data of the target patient through multi-source data acquisition, and construct a multimodal real-world dataset of the target patient. The digital twin model building module is used to build a digital twin model of the target patient based on a time-series Transformer architecture using a multimodal real-world dataset. The counterfactual outcome prediction module is used to predict the counterfactual outcome of multiple candidate drug regimens for a target patient based on a digital twin model, and to obtain the counterfactual outcome prediction results for each candidate drug regimen. The counterfactual outcome prediction results include efficacy index prediction data and adverse event prediction data. The causal evaluation module is used to build a policy-dependent causal reinforcement learning model based on historical real-world data. It uses a double robust estimation algorithm to evaluate the causality of historical medication strategies and constructs an individualized long-term value function for each candidate medication regimen. The joint decision-making module is used to construct a joint value evaluation model for drug efficacy and safety based on the counterfactual outcome prediction results and the individualized long-term value function. Based on the joint value evaluation model for drug efficacy and safety, the module calculates the maximum joint value of drug efficacy and safety with the objective of maximizing the function value, and obtains the optimal medication regimen for the target patient. The joint decision-making module also includes a value function integration unit, used to integrate the counterfactual outcome prediction of the digital twin module and the long-term value function of the causal assessment module; a multi-objective optimization unit, used to construct a joint efficacy-safety value function and search for the optimal solution; and an interpretability generation unit, used to generate an evaluation report containing key features, value comparisons, and risk analysis.

[0010] The beneficial effects of this invention are as follows: It achieves personalized counterfactual prediction through a patient digital twin model, enabling the assessment of the potential efficacy and adverse event risks of different medication regimens at the individual patient level; simultaneously, it employs off-policy causal reinforcement learning and bi-robust estimation methods to evaluate the personalized value of long-term sequential medication strategies from observational real-world data, improving the rigor of causal inference; furthermore, it constructs a joint efficacy-safety value function, unifying efficacy benefits, safety risks, and adherence costs into a single decision framework to achieve multi-objective joint optimization; finally, it provides an interpretable analysis report containing key features, value comparisons, and risk analysis, enhancing clinicians' understanding and trust in the recommendation results, and facilitating its widespread application in clinical practice. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of a personalized drug efficacy and safety evaluation method based on digital twins and causal reinforcement learning, according to an embodiment of the present invention.

[0012] Figure 2 This is a schematic diagram of the structure of a personalized drug efficacy and safety evaluation system based on digital twins and causal reinforcement learning, according to an embodiment of the present invention.

[0013] Figure 3 This is a schematic diagram of the terminal device structure of a personalized drug efficacy and safety evaluation method based on digital twins and causal reinforcement learning according to an embodiment of the present invention.

[0014] Figure 4 This is a schematic diagram of a computer-readable storage medium structure for a personalized drug efficacy and safety evaluation method based on digital twins and causal reinforcement learning, according to an embodiment of the present invention.

[0015] In the diagram, 200 is the terminal device, 210 is the memory, 211 is the RAM, 212 is the cache memory, 213 is the ROM, 214 is the program / utility, 215 is the program module, 220 is the processor, 230 is the bus, 240 is the external device, 250 is the I / O interface, 260 is the network adapter, and 300 is the program product. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention; that is, the described embodiments are only a part of the embodiments of the invention, and not all of them. The components of the embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0017] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0018] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or machine that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or machine. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or machine that includes said element.

[0019] Example 1: like Figure 1 As shown, Embodiment 1 of the present invention provides a method for evaluating the efficacy and safety of drugs based on digital twins and causal reinforcement learning, comprising the following steps: S1. By collecting data from multiple sources, structured clinical indicator data, medical order data, imaging and laboratory summary data, and free text data of the target patients are obtained respectively, and a multimodal real-world dataset of the target patients is constructed. S2. Based on a multimodal real-world dataset, construct a digital twin model of the target patient using a temporal Transformer architecture; S3. Based on a digital twin model, counterfactual outcome prediction is performed on multiple candidate drug regimens for the target patient to obtain the counterfactual outcome prediction results for each candidate drug regimen. The counterfactual outcome prediction results include efficacy index prediction data and adverse event prediction data. S4. Construct a policy-dependent causal reinforcement learning model based on historical real-world data, evaluate the causal effects of historical medication strategies through a double robust estimation algorithm, and construct an individualized long-term value function for each candidate medication regimen. S5. Based on the counterfactual outcome prediction results and the individualized long-term value function, construct a joint value evaluation model for drug efficacy and safety; S6. Based on the drug efficacy and safety combined value evaluation model, with the highest drug efficacy and safety for the target patients as the optimization objective, the combined value of each candidate drug regimen is quantitatively calculated to obtain the drug efficacy and safety evaluation results. The drug regimen with the highest combined value is determined as the recommended regimen, and the corresponding drug results and its interpretability analysis report are output.

[0020] Specifically, multimodal real-world data of the target patient is acquired. This multimodal real-world data includes structured clinical indicator data, medical order data, imaging and laboratory summary data, and free text data. The data is aligned and resampled along the treatment timeline to provide time-series input for digital twin construction and causal reinforcement learning. This embodiment demonstrates the application of this method in the oncology department of a tertiary hospital to evaluate the individualized efficacy and safety of an immune checkpoint inhibitor combined with chemotherapy regimen in patients with advanced non-small cell lung cancer (NSCLC). First, multimodal real-world data of the target patient is acquired. The target patient is a 65-year-old male NSCLC patient, EGFR / ALK negative, PD-L1 expression ≥50%, PS score of 1, and a history of mild COPD. Data from the past two years was extracted from the hospital's EMR system, including: Structured clinical indicators: age, sex, weight, height, BMI, complete blood count (white blood cells, neutrophils, hemoglobin, platelets), liver and kidney function (ALT, AST, creatinine, blood urea nitrogen), electrocardiogram (heart rate, QTc interval), pulmonary function (FEV1, FVC), etc., totaling more than 50 indicators, recorded once per visit to form time-series data. Medical order data: previously used medications (resistance to gefitinib after 12 months of first-line treatment, followed by pemetrexed + carboplatin chemotherapy), current medications (COPD-related medication: salbutamol inhaler), dosage, frequency, and route of administration. Imaging and laboratory summary: chest CT report text describing tumor size, location, and metastasis; pathology report describing histological type and differentiation degree; gene testing report listing driver gene mutations. Free text: Medical records documenting the patient's chief complaint, symptoms, signs, and the doctor's diagnostic and treatment decision-making process; discharge summary summarizing the treatment process and effects during hospitalization. After data collection, de-identification processing is performed, removing patient names and ID numbers, generalizing birth dates to age, and generalizing specific addresses to province and city levels, thus constructing a multimodal real-world dataset for the patient.

[0021] Furthermore, step S2 includes the following sub-steps: S201. Obtain the multimodal real-world dataset obtained in step S1, and extract the structured clinical indicator data, medical order data, image examination summary data and free text data of the target patient respectively; S202. Perform time-series feature encoding on the structured clinical indicator data of the target patient to obtain the first feature vector sequence; S203. Perform entity recognition and measurement normalization processing on the medical order data of the target patient to obtain the second feature vector sequence; S204. Using natural language processing algorithms, semantic feature recognition is performed on the image examination summary data and free text data of the target patient to obtain a third feature vector sequence; S205. Based on the first feature vector sequence, the second feature vector sequence, and the third feature vector sequence, perform multimodal feature fusion to obtain the multimodal fused feature vector data of the target patient; S206. Based on multimodal fusion feature data, and using a temporal Transformer architecture, learn the temporal state evolution pattern of the target patient's multimodal real-world data through a self-attention mechanism, and construct a digital twin model of the target patient.

[0022] Specifically, the implementation flow of each sub-step in the above embodiments is as follows: First, the structured clinical indicator data is time-series encoded to obtain a first feature vector sequence. The structured clinical indicator data includes, but is not limited to, physiological indicators such as patient age, gender, weight, blood pressure, heart rate, blood glucose, blood lipids, liver and kidney function indicators, and electrocardiogram parameters, as well as diagnostic codes (such as ICD-10 codes), past medical history, and family history. For continuous indicators, standardization or normalization is used; for categorical indicators, one-hot encoding or embedded vector representation is used. Time-series encoding uses positional encoding (PE) to capture temporal sequence information. Specifically, more than 50 indicators are arranged in chronological order to form a time-series matrix. ,in This represents the number of time steps, i.e., the current number of visits. Continuous indicators are standardized using Z-scores, and categorical indicators, such as gender and PS score, are encoded using one-hot encoding. Location encoding is added to indicate the time sequence, represented as: , ,in Indicates the time step position. Indicates the feature dimension index. This represents the total dimension of the features.

[0023] Next, the medical order data undergoes drug entity recognition and dosage normalization processing to obtain a second feature vector sequence. The medical order data includes information such as prescription drug name, dosage, frequency, route of administration, and duration of administration. Named Entity Recognition (NER) technology is used to extract the generic drug name and map it to a standard drug coding system (such as ATC coding or RxNorm coding). Dosage units are standardized and unified, converting different routes of administration and frequencies into daily dose equivalents. Drug embedding vectors are used to represent the characteristics of each drug. Specifically, the generic drug names (gefitinib, pemetrexed, carboplatin, salbutamol) are extracted and mapped to ATC codes (L01XE02, L01BA04, L01XA02, R03AC02). Daily dose equivalents are calculated; for example, pemetrexed 500mg / m² q3w is converted to a daily dose of approximately 167mg / m² / day. Each drug is represented as a 128-dimensional vector using a pre-trained drug embedding model (such as Mol2Vec).

[0024] Next, natural language processing is performed on the image examination summary data and free text data to extract clinical semantic features, resulting in a third feature vector sequence. The image examination summary data includes textual descriptions and structured fields (such as tumor size and lesion location) from imaging examination reports (e.g., CT, MRI, ultrasound). The free text data includes unstructured clinical text such as medical records, discharge summaries, and consultation opinions. Pre-trained clinical domain language models (e.g., BioBERT, ClinicalBERT) are used for text encoding to extract semantic feature vectors. For key clinical entities (e.g., symptoms, signs, diseases, treatments), named entity recognition and relation extraction techniques are used for structuring. Specifically, the BioBERT model is used to encode text such as CT reports, pathology reports, and medical records, extracting a 768-dimensional semantic feature vector. The named entity recognition model is used to extract key clinical entities, such as the right upper lung lesion, about 4.5cm×3.2cm in size, with unclear boundaries and accompanied by mediastinal lymph node enlargement. The structured information such as location (right upper lung), size (4.5×3.2cm) and metastasis (mediastinal lymph nodes) are extracted.

[0025] Then, the first feature vector sequence, the second feature vector sequence, and the third feature vector sequence are fused using multimodal methods and input into the temporal Transformer model. Multimodal fusion employs an attention fusion mechanism to learn the interaction relationships between features of different modalities. Specifically, the multimodal feature vectors at each time step are concatenated or weighted and summed to obtain a fused feature vector. This fused feature vector is input into the encoder part of the temporal Transformer model. Specifically, at each time step... By concatenating structured indicator features, drug embedding features, and text semantic features, a fused feature vector is obtained. (50+128+768=946 dimensions).

[0026] Finally, the patient's temporal state evolution pattern is learned through the self-attention mechanism of the temporal Transformer model, and a patient state transition function is established to obtain the digital twin model. The temporal Transformer model includes multiple self-attention layers and feedforward neural network layers. The self-attention mechanism can capture long-distance dependencies between different time steps and learn the dynamic evolution pattern of the patient's health state. The training objective of the digital twin model is to minimize the difference between the predicted state and the actual observed state. Specifically, the Transformer encoder contains 6 layers, each with 8 attention heads, and a hidden layer dimension of 512. The input is a fused feature sequence. Output the hidden state sequence at each time step. The decoder employs an autoregressive structure, based on... and the predicted state at the previous moment Predict the current state The training objective is to minimize the mean squared error (MSE) between the predicted state and the actual observed state. The model is pre-trained on historical data of 5,000 NSCLC patients from the past 5 years at the hospital, and then fine-tuned on data from the target patients to learn the individualized state evolution patterns of the patients, thus obtaining a digital twin model.

[0027] Furthermore, step S3 includes the following sub-steps: S301: Obtain multiple candidate medication regimens for the target patient. Based on the digital twin model obtained in step S2, use the Monte Carlo sampling algorithm to sample and predict the target patient's current state and candidate medication regimens, obtaining the target patient's state evolution trajectory data corresponding to each candidate medication regimen; specifically... S302: Based on the target patient state evolution trajectory data, the expected value and confidence interval of the efficacy index are calculated through statistical analysis, and the occurrence time data and occurrence probability data of adverse events in each target patient state evolution trajectory data are recorded to generate an adverse event risk distribution. S305: Based on the expected value and confidence interval of the efficacy indicators and the risk distribution of adverse events, the counterfactual outcome prediction results are obtained.

[0028] Specifically, the implementation process of each sub-step in the above embodiments is as follows: First, the current state of the target patient and the candidate drug regimen are input into the digital twin model; then, multiple possible patient state evolution trajectories are generated using the Monte Carlo sampling method through the generative decoder of the digital twin model; next, statistical analysis is performed on the multiple patient state evolution trajectories to calculate the expected value and confidence interval of the efficacy indicators; the efficacy indicators include progression-free survival, overall survival, and remission rate; then, the occurrence time and probability of potential adverse events are identified based on the patient state evolution trajectories to generate an adverse event risk distribution; finally, the expected value and confidence interval of the efficacy indicators and the adverse event risk distribution are used as the counterfactual outcome prediction output.

[0029] Specifically, in this embodiment, the candidate treatment regimens include: Regimen A: Pembrolizumab monotherapy (200 mg q3w); Regimen B: Nivolumab + Ipilimumab dual immunotherapy (Nivolumab 3 mg / kg + Ipilimumab 1 mg / kg q3w×4, followed by Nivolumab 240 mg q2w maintenance); Regimen C: Pembrolizumab + pemetrexed + carboplatin immunotherapy combined with chemotherapy (Pembrolizumab 200 mg + pemetrexed 500 mg / m² + carboplatin AUC 5 q3w×4, followed by pembrolizumab + pemetrexed maintenance); Regimen D: Atezolizumab + bevacizumab + chemotherapy (Atezolizumab 1200 mg + Bevacizumab) 15 mg / kg + pemetrexed 500 mg / m² + carboplatin AUC 6 q3w×4, then Atezolizumab + Bevacizumab + pemetrexed for maintenance. For each candidate regimen, the patient's current status should be recorded. and medication regimen Inputting a digital twin model, Monte Carlo sampling was used to generate 1000 state evolution trajectories to simulate the treatment process over the next 2 years (approximately 24 months). Efficacy endpoints: progression-free survival (PFS), overall survival (OS), objective response rate (ORR), and disease control rate (DCR). Adverse events: Grade 3-4 immune-related adverse events (irAEs) such as pneumonia, enteritis, hepatitis, rash, and thyroid dysfunction; chemotherapy-related hematologic toxicities such as neutropenia and thrombocytopenia; bevacizumab-related bleeding, hypertension, and proteinuria. Taking scheme C as an example, the predicted results are as follows: median PFS: 8.2 months (95% CI: 6.8-9.9); median OS: 18.5 months (95% CI: 15.2-22.1); ORR: 45.2% (95% CI: 40.1-50.5); cumulative incidence of grade 3-4 irAEs: 18.5% (pneumonia 5.2%, enteritis 4.1%, hepatitis 3.3%, rash 2.8%, endocrine disorders 3.1%); cumulative incidence of grade 3-4 hematologic toxicities: 35.7% (neutropenia 28.3%, thrombocytopenia 12.4%), where CI represents the confidence interval.

[0030] Finally, efficacy-safety prediction reports for each regimen are generated and presented in visual charts (Kaplan-Meier survival curves and cumulative incidence curves of adverse events).

[0031] Furthermore, step S4 includes the following sub-steps: S401: Based on the multimodal real-world dataset of the target patient, extract medication trajectory data of multiple historical reference patients from the historical real-world database. The medication trajectory data of the historical reference patients includes a state sequence composed of patient clinical data, an action sequence composed of patient medication adjustments, and a reward sequence composed of patient efficacy feedback. S402: Based on historical reference patients' medication trajectory data, a propensity score model is used to predict the probability value of the target patient choosing a specific medication adjustment action in each state, thus obtaining a propensity score; S403: Based on the value function estimation model, predict the expected long-term return corresponding to each state of the target patient and the medication adjustment action to obtain the value function estimate; S404: Based on the propensity score and value function estimation, the departure strategy assessment value of the target patient is calculated using a double robust estimation formula; S405: Based on the off-strategy assessment value, through importance sampling correction and confidence lower bound calculation, the individualized long-term value function and strategy risk upper bound of the candidate drug regimen for the target patient are obtained.

[0032] Furthermore, the bi-robust estimation formula is expressed as: ; in, This indicates the strategic value of dual robust estimation. Indicates the number of samples. Indicates the target policy in state Select action The probability, The rating indicates the tendency towards certain behavioral strategies. This indicates the actual observation report. This represents the estimation of the value function. It represents the expected value under the target strategy.

[0033] Specifically, the implementation process of each sub-step in the above embodiment is as follows: First, extract medication trajectory data of multiple historical reference patients from a historical real-world database. The medication trajectory data of each historical reference patient includes a state sequence, an action sequence, and a reward sequence. Next, construct a propensity score model to estimate the probability of choosing a specific medication action in each state, and obtain a propensity score. Then, construct a value function estimation model to predict the expected long-term return of each state-action pair, and obtain a value function estimate. Then, use a double robust estimation formula to combine the propensity score and the value function estimate to calculate the policy evaluation value. Finally, through importance sampling correction and confidence lower bound calculation, output the individualized long-term value function and policy risk upper bound of the candidate medication plan.

[0034] Specifically, in this embodiment, historical medication history is first extracted. Data from 3000 NSCLC patients who previously received immunotherapy ± chemotherapy are extracted from the hospital database. Each patient is followed up for an average of 15 months, and their status (clinical indicators, symptoms, tumor measurements), actions (medication regimen adjustments), and rewards (efficacy improvement +1, disease progression -2, serious adverse events -1, death -5) are recorded at each time point. Next, a propensity score model is trained. Using a logistic regression model, with patient status (age, gender, PS score, PD-L1 expression, driver genes, number of previous lines of treatment, liver and kidney function, comorbidities, etc.) as independent variables, the probability of selecting each medication regimen is predicted. The model achieved an AUC of 0.78 on the validation set, indicating that it effectively captured the patterns of doctors' medication decisions. Next, a value function estimation model was trained. A deep Q-network (DQN) was used, with the input state... and actions Output Q value This represents the expected long-term survival benefit after taking this action from this state. The network structure is a 3-layer fully connected neural network (256-128-64), using Mean Squared Bellman Error as the loss function, and is stabilized using empirical replay and target network techniques. Then, a double robust estimation is performed. For candidate C (Pembrolizumab + chemotherapy), its policy evaluation value is calculated: ; in, The policy probability of option C (if state) If the indication meets the criteria for scheme C, the score is 1; otherwise, it is 0. Assess the tendency to adopt historical behavioral strategies. For patients The actual cumulative survival benefit Let C represent the expected value. The long-run value function of C is calculated. (Survival benefit in months). Final confidence interval and upper bound of risk. Using bootstrap resampling 1000 times, the 95% confidence interval for the double robust estimate was calculated to be [10.8, 14.3]. The CVaR method was used to calculate the upper bound of risk at the 5th percentile, i.e., the average survival benefit in the worst-case 5% scenario is 6.2 months, suggesting that some patients may experience poor efficacy or adverse events leading to early discontinuation of treatment. Causal assessments were performed on the four candidate options to obtain the long-term value function: Option A: =10.2 (95% CI: 8.5-12.1); Scheme B: =11.8 (95% CI: 9.7-14.2); Scheme C: =12.5 (95% CI: 10.8-14.3); Scheme D: =13.1 (95% CI: 11.0-15.4), where CI represents the confidence interval.

[0035] Furthermore, the combined efficacy and safety value evaluation model for drugs described in step S5 is expressed as follows: ; in, This represents the combined efficacy-safety value function. This represents the therapeutic value function. Represents the safety value function (based on adverse event risk score). This represents the cost of patient compliance, with α, β, and γ being weighted coefficients set according to clinical priority.

[0036] Specifically, based on the counterfactual outcome prediction results and the individualized long-term value function, a joint value evaluation model for drug efficacy and safety is constructed. In this embodiment, clinical priority is given to survival benefit, therefore, weights are set accordingly. , Compliance costs Based on a comprehensive assessment of factors such as medication complexity, infusion time, and adverse reaction burden: Option A (single-drug): (Relatively simple, q3w infusion, with a lower risk of adverse reactions); Regimen B (dual immunization): (Requires two drugs in combination, with a higher initial dosing frequency); Regimen C (immunotherapy + chemotherapy): (Chemotherapy increases hematological toxicity and adverse reactions such as nausea and vomiting); Regimen D (immunotherapy + anti-angiogenic therapy + chemotherapy): (The most complex regimen, with the heaviest adverse reaction burden). Based on the above parameters, the combined value of each regimen was calculated using the drug efficacy and safety combined value evaluation model: ; ; ; .

[0037] Furthermore, the interpretability analysis report in step S6 includes: identifying key temporal features affecting the prediction results in the digital twin model through an attention mechanism; explaining the long-term value advantage of the recommended scheme compared to other schemes based on a causal reinforcement learning model; and providing comparative analysis results of efficacy-safety trade-offs for different candidate schemes, including expected efficacy indicators, major adverse event risks and their confidence intervals.

[0038] Specifically, based on the combined efficacy and safety value evaluation model for drugs, regimen D (Atezolizumab + Bevacizumab + chemotherapy) The maximum value is 12.79, therefore it is selected as the recommended regimen. An evaluation report is generated, including: Recommended regimen: Atezolizumab + Bevacizumab + Pemetrexed + Carboplatin; Expected efficacy: Median PFS approximately 9.5 months, median OS approximately 20.3 months, ORR approximately 50%; Major risks: Grade 3-4 irAEs approximately 20%, hematologic toxicity approximately 38%, bleeding and hypertension risk approximately 25%; Long-term value: Expected survival benefit approximately 13.1 months (considering efficacy) or 12.79 months (considering efficacy, safety, and compliance); Comparison with other regimens: Regimen D is more effective than regimens A, B, and C, but has a relatively higher adverse event burden; If patients cannot tolerate or refuse complex regimens, regimen C (immunotherapy + chemotherapy) can be considered as a second-best option. The report is presented in structured tables and visual charts for easy understanding and decision-making by clinicians.

[0039] Example 2

[0040] like Figure 2 As shown in Example 1, Example 2 of this invention proposes a personalized drug efficacy and safety evaluation system based on a digital twin and causal reinforcement learning method.

[0041] Specifically, the system includes a data acquisition module, which is used to acquire structured clinical indicator data, medical order data, imaging and laboratory summary data and free text data of the target patient through multi-source data acquisition, and construct a multimodal real-world dataset of the target patient; The digital twin model building module is used to build a digital twin model of the target patient based on a time-series Transformer architecture using a multimodal real-world dataset. The counterfactual outcome prediction module is used to predict the counterfactual outcome of multiple candidate drug regimens for a target patient based on a digital twin model, and to obtain the counterfactual outcome prediction results for each candidate drug regimen. The counterfactual outcome prediction results include efficacy index prediction data and adverse event prediction data. The causal evaluation module is used to build a policy-dependent causal reinforcement learning model based on historical real-world data. It uses a double robust estimation algorithm to evaluate the causality of historical medication strategies and constructs an individualized long-term value function for each candidate medication regimen. The joint decision-making module is used to construct a joint value evaluation model for drug efficacy and safety based on the counterfactual outcome prediction results and the individualized long-term value function. Based on the joint value evaluation model for drug efficacy and safety, the module calculates the maximum joint value of drug efficacy and safety with the objective of maximizing the function value, and obtains the optimal medication regimen for the target patient. The joint decision-making module also includes a value function integration unit, used to integrate the counterfactual outcome prediction of the digital twin module and the long-term value function of the causal assessment module; a multi-objective optimization unit, used to construct a joint efficacy-safety value function and search for the optimal solution; and an interpretability generation unit, used to generate an evaluation report containing key features, value comparisons, and risk analysis.

[0042] Furthermore, the digital twin model construction module also includes: a multimodal preprocessing unit for feature extraction and encoding of structured clinical indicator data, medical order data, imaging test summary data, and free text data; a temporal modeling unit for constructing a temporal state evolution model based on the Transformer architecture; a generative decoding unit for generating patient state evolution trajectories under different medication regimens; and a counterfactual reasoning unit for calculating efficacy indicator predictions and adverse event risk distributions.

[0043] Furthermore, the causal assessment module also includes: a data playback unit for extracting medication trajectory data from a historical real-world database; a propensity score estimation unit for estimating the probability of choosing a specific medication action under different states; a value function estimation unit for predicting the expected long-term return of state-action pairs; a dual robust estimation unit for combining propensity scores and value function estimation to perform off-policy assessment; and a confidence interval calculation unit for calculating the lower confidence bound of the policy value and the upper bound of the policy risk.

[0044] Specifically, the workflow of this system is as follows: First, the data acquisition module acquires structured clinical indicator data, medical order data, imaging and laboratory summary data, and free text data of the target patient through multi-source data acquisition, and then constructs a multimodal real-world dataset of the target patient. Next, this multimodal real-world dataset is input into the digital twin model construction module. The multimodal preprocessing unit of this module first extracts and encodes features from the structured clinical indicator data, medical order data, imaging and laboratory summary data, and free text data. Then, the temporal modeling unit constructs a temporal state evolution model based on the Transformer architecture. Next, the generative decoding unit generates the patient state evolution trajectory under different medication regimens. Finally, the counterfactual reasoning unit calculates the efficacy indicator prediction and adverse event risk distribution, thus completing the construction of the digital twin model of the target patient. Next, the counterfactual outcome prediction module, based on the digital twin model, performs counterfactual outcome predictions for multiple candidate medication regimens of the target patient, obtaining the counterfactual outcome prediction results for each candidate medication regimen. These results include efficacy indicator prediction data and adverse event prediction data. Meanwhile, the causal assessment module constructs a policy-dependent causal reinforcement learning model based on historical real-world data. Its data replay unit first extracts medication trajectory data from the historical real-world database. Then, the propensity score estimation unit estimates the probability of choosing a specific medication action under different states. Next, the value function estimation unit predicts the expected long-term return of the state-action pair. Subsequently, the dual-robust estimation unit combines propensity score and value function estimation for policy-dependent assessment. Finally, the confidence interval calculation unit calculates the lower confidence bound of the policy value and the upper bound of the policy risk, thus performing a causal assessment of historical medication strategies through the dual-robust estimation algorithm and constructing individualized long-term value functions for each candidate medication plan. Then, the joint decision-making module, based on the counterfactual reversal prediction results and individualized long-term value functions, first integrates the counterfactual reversal prediction from the digital twin module and the long-term value function from the causal assessment module. Then, the multi-objective optimization unit constructs a joint efficacy-safety value function and searches for the optimal solution, calculating the maximum joint value of drug efficacy and safety with the objective of maximizing the function value, and obtaining the optimal medication plan for the target patient. Finally, the interpretability generation unit generates an evaluation report containing key features, value comparisons, and risk analysis.

[0045] Example 3

[0046] like Figure 3 As shown in Example 1, Example 3 proposes a terminal device for a personalized drug efficacy and safety evaluation method based on digital twins and causal reinforcement learning. The terminal device 200 includes at least one memory 210, at least one processor 220, and a bus 230 connecting different platform systems.

[0047] The memory 210 may include a readable medium in the form of volatile memory, such as RAM 211 and / or cache memory 212, and may further include ROM 213.

[0048] The memory 210 also stores a computer program that can be executed by the processor 220, causing the processor 220 to execute any of the above-described methods for evaluating the efficacy and safety of individualized drugs based on digital twins and causal reinforcement learning in this application embodiment. The specific implementation method and the achieved technical effects are consistent with those described in the embodiments of the above methods, and some details will not be repeated here. The memory 210 may also include a program / utility 214 having a set (at least one) of program modules 215. Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment.

[0049] Accordingly, processor 220 can execute the aforementioned computer program, as well as executable program / utility 214.

[0050] Bus 230 can represent one or more of several types of bus structures, including a memory bus or memory controller, peripheral bus, graphics acceleration port, processor, or a local bus using any of the various bus structures.

[0051] Terminal device 200 can also communicate with one or more external devices 240, such as keyboards, pointing devices, Bluetooth devices, etc., and with one or more devices capable of interacting with it, and / or with any device that enables it to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed via I / O interface 250. Furthermore, terminal device 200 can communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 260. Network adapter 260 can communicate with other modules of terminal device 200 via bus 230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with terminal device 200, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0052] Example 4

[0053] like Figure 4As shown in Example 1, this example proposes a computer-readable storage medium for a personalized drug efficacy and safety evaluation method based on digital twins and causal reinforcement learning. The computer-readable storage medium stores instructions that, when executed by a processor, implement any one of the aforementioned personalized drug efficacy and safety evaluation methods based on digital twins and causal reinforcement learning. The specific implementation method and the achieved technical effects are consistent with those described in the examples above, and some details will not be repeated.

[0054] The program product 300 provided in this embodiment for implementing the above method can be a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. However, the program product 300 of the present invention is not limited thereto. In this embodiment, the readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. The program product 300 can employ any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0055] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on a user computing device, partially on a user device, as a standalone software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to user computing devices via any type of network, including local area networks (LANs) or wide area networks (WANs), or they can be connected to external computing devices (e.g., via the Internet using an Internet service provider).

[0056] This invention is described from the perspectives of its intended use, effectiveness, progress, and novelty. Its practical and progressive features meet the functional enhancement and use requirements emphasized by the Patent Law. The above description and drawings are merely preferred embodiments of this application and are not intended to limit this application. Therefore, all structures, devices, features, etc., that are similar to or identical to those of this application, i.e., all equivalent substitutions or modifications made in accordance with the scope of this patent application, shall fall within the scope of protection of this patent application.

[0057] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An individualized drug efficacy and safety evaluation method based on digital twin and causal reinforcement learning, characterized in that, The method comprises the following steps: S1. Obtain structured clinical indicator data, medical order data, image test summary data and free text data of a target patient through multi-source data collection, and construct a multi-modal real world data set of the target patient; S2. Construct a digital twin model of the target patient based on the time series Transformer architecture according to the multi-modal real world data set; S3. Based on the digital twin model, the counterfactual outcome prediction of the multiple candidate drug regimens of the target patient is carried out, and the counterfactual outcome prediction result of each candidate drug regimen is obtained, which includes efficacy indicator prediction data and adverse event prediction data; S4. Construct an off-policy causal reinforcement learning model based on historical real world data, and evaluate the causal relationship of historical drug strategies through a double-robust estimation algorithm to construct an individualized long-term value function of each candidate drug regimen; S5. According to the counterfactual outcome prediction result and the individualized long-term value function, a drug efficacy and safety joint value evaluation model is constructed; S6. Based on the drug efficacy and safety joint value evaluation model, the joint value of each candidate drug regimen is quantitatively calculated to obtain the drug efficacy and safety evaluation result, and the drug regimen with the highest joint value is determined as the recommended regimen, and the corresponding drug result and its explainability analysis report are output.

2. The method according to claim 1, wherein, Step S2 comprises the following sub-steps: S201. Obtain the multi-modal real world data set obtained in step S1, and extract the structured clinical indicator data, medical order data, image test summary data and free text data of the target patient; S202. Time series feature coding is performed on the structured clinical indicator data of the target patient to obtain a first feature vector sequence; S203. Entity recognition and measurement standardization processing are performed on the medical order data of the target patient to obtain a second feature vector sequence; S204. Through a natural language processing algorithm, semantic feature recognition is performed on the image test summary data and free text data of the target patient to obtain a third feature vector sequence; S205. According to the first feature vector sequence, the second feature vector sequence and the third feature vector sequence, multi-modal feature fusion is performed to obtain multi-modal fusion feature vector data of the target patient; S206. According to the multi-modal fusion feature data, based on the time series Transformer architecture, the time series state evolution law of the multi-modal real world data of the target patient is learned through the self-attention mechanism, and the digital twin model of the target patient is constructed.

3. The method of claim 1, wherein the method is characterized by, Step S3 comprises the following sub-steps: S301: Obtain multiple candidate drug regimens of the target patient, and perform target patient state sampling prediction on the current state and candidate drug regimen of the target patient through a Monte Carlo sampling algorithm based on the digital twin model obtained in step S2 to obtain target patient state evolution trajectory data corresponding to each candidate drug regimen; S302: According to the target patient state evolution trajectory data, the expected value and confidence interval of the therapeutic effect index are calculated by statistical analysis, and the occurrence time data and occurrence probability data of adverse events in each target patient state evolution trajectory data are recorded respectively to generate the adverse event risk distribution; S305: According to the expected value and confidence interval of the therapeutic effect index and the adverse event risk distribution, the counterfactual outcome prediction result is obtained.

4. The method of claim 1, wherein the method is characterized by, Step S4 includes the following sub-steps: S401: According to the multi-modal real world data set of the target patient, the medication trajectory data of a plurality of historical reference patients is extracted from the historical real world database, wherein the medication trajectory data of the historical reference patients includes a state sequence composed of patient clinical data, an action sequence composed of patient medication adjustment, and a reward sequence composed of patient therapeutic effect feedback; S402: According to the medication trajectory data of the historical reference patients, the probability value of the target patient selecting a specific medication adjustment action in each state is predicted by the propensity score model to obtain the propensity score; S403: Based on the value function estimation model, the expected long-term return corresponding to each state and medication adjustment action of the target patient is predicted to obtain the value function estimation; S404: According to the propensity score and the value function estimation, the off-policy evaluation value of the target patient is calculated by the double robust estimation formula; S405: According to the off-policy evaluation value, the individualized long-term value function and the strategy risk upper bound of the candidate medication scheme of the target patient are obtained by importance sampling correction and confidence lower bound calculation.

5. The method of claim 4, wherein the method is characterized by, The double robust estimation formula is represented as: ； where, represents the policy value of the double robust estimate, represents the number of samples, represents the probability of the target policy selecting action in state , represents the propensity score of the behavior policy, represents the actual observed return, represents the value function estimate, represents the expected value under the target policy.

6. The method of claim 1, wherein the method is based on digital twin and causal reinforcement learning for individualized drug efficacy and safety evaluation. The drug efficacy and safety combined value evaluation model in step S5 is represented as: ； wherein, represents a combined value function of efficacy and safety, represents a value function of efficacy, represents a value function of safety, represents a cost of patient compliance, and a, b, g are weight coefficients, which are set according to clinical priorities.

7. The method of claim 1, wherein the method is based on digital twin and causal reinforcement learning for individualized drug efficacy and safety evaluation. The explainability analysis report in step S6 includes: identifying key time sequence features affecting the prediction result in the digital twin model through attention mechanism; based on the causal reinforcement learning model, the long-term value advantage of the recommended scheme compared with other schemes is explained; the efficacy-safety trade-off comparative analysis results of different candidate schemes are provided, including expected efficacy indicators, major adverse event risks and their confidence intervals.

8. An individualized drug efficacy and safety evaluation system based on digital twin and causal reinforcement learning, the system is realized based on the individualized drug efficacy and safety evaluation method based on digital twin and causal reinforcement learning according to any one of claims 1-7, characterized in that, It includes: A data acquisition module for acquiring structured clinical indicator data, medical order data, image test summary data and free text data of a target patient through multi-source data acquisition to construct a multi-modal real world data set of the target patient; A digital twin model construction module for constructing a digital twin model of the target patient based on a time series Transformer architecture according to the multi-modal real world data set; A counterfactual outcome prediction module for predicting the counterfactual outcome of a plurality of candidate medication schemes of the target patient based on the digital twin model to obtain counterfactual outcome prediction results of each candidate medication scheme, wherein the counterfactual outcome prediction results include therapeutic effect index prediction data and adverse event prediction data; A causal evaluation module for constructing an off-policy causal reinforcement learning model based on historical real world data, and performing causal evaluation on historical medication strategies by a double robust estimation algorithm to construct individualized long-term value functions of each candidate medication scheme. The joint decision module is used for constructing a drug efficacy and safety joint value evaluation model according to the counterfactual outcome prediction result and the individualized long-term value function, calculating the maximum joint value of drug efficacy and safety based on the drug efficacy and safety joint value evaluation model with the maximum function value as the target, and obtaining the optimal target patient medication scheme; The joint decision module further includes a value function integration unit for integrating the counterfactual outcome prediction of the digital twin module and the long-term value function of the causal assessment module; a multi-objective optimization unit for constructing an efficacy-safety joint value function and searching for an optimal scheme; and an explainability generation unit for generating an evaluation report containing key features, value comparisons and risk analysis.

9. The individualized drug efficacy and safety evaluation system based on digital twin and causal reinforcement learning according to claim 8, characterized in that, The digital twin model construction module further includes a multi-modal preprocessing unit for feature extraction and coding of structured clinical indicator data, medical order data, image test summary data and free text data; a time series modeling unit for constructing a time series state evolution model based on a Transformer architecture; a generative decoding unit for generating patient state evolution trajectories under different medication schemes; and a counterfactual reasoning unit for calculating efficacy indicator prediction and adverse event risk distribution.

10. The individualized drug efficacy and safety evaluation system based on digital twin and causal reinforcement learning according to claim 8, characterized in that, The causal assessment module further includes a data playback unit for extracting medication trajectory data from a historical real-world database; a propensity score estimation unit for estimating the probability of selecting a specific medication action under different states; a value function estimation unit for predicting the expected long-term return of state-action pairs; a double-robust estimation unit for off-policy evaluation combining propensity score and value function estimation; and a confidence interval calculation unit for calculating the lower confidence bound of strategy value and the upper confidence bound of strategy risk.

Images