Medical follow-up visit dialogue strategy dynamic optimization method and system based on closed-loop feedback

Abstract

The invention relates to the technical field of artificial intelligence and intelligent medical treatment, and discloses a medical follow-up visit dialogue strategy dynamic optimization method and system based on closed-loop feedback. The method comprises the steps of collecting multi-source heterogeneous patient data, performing preprocessing operation on the patient data, and storing the preprocessed patient data to a patient data center; based on the data of the patient data center, periodically executing follow-up effect quantitative evaluation to obtain net effect values of different dialogue strategies on the long-term clinical outcome of the patient; taking the net effect value as a long-term delay reward, and optimizing the dialogue strategy based on a combined reward function fusing the short-term instant reward and the long-term delay reward; and interacting with the patient by using the optimized dialogue strategy, and feeding back the generated new interaction data to a patient data center for follow-up effect quantitative evaluation and strategy optimization in the next period to form closed-loop feedback. According to the invention, continuous self-evolution of the follow-up visit strategy is realized, and an intelligent and precise solution is provided for a medical scene needing long-term intervention.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and smart healthcare technology, and in particular to a method and system for dynamic optimization of medical follow-up dialogue strategies based on closed-loop feedback. Background Technology

[0002] With the accelerating aging of the global population and changes in lifestyle, chronic disease management has become a major challenge in the field of public health. Effective outpatient follow-up is crucial for controlling disease progression, improving patients' quality of life, and reducing readmission rates. However, current follow-up management models suffer from significant bottlenecks in terms of efficiency, scalability, and effectiveness. Currently, mainstream outpatient follow-up still heavily relies on labor-intensive methods such as telephone, text message, or face-to-face interviews conducted by healthcare professionals. This approach suffers from inherent drawbacks such as low efficiency, high labor costs, and difficulty in standardizing follow-up content and criteria. More importantly, the evaluation of follow-up effectiveness largely depends on qualitative observations by healthcare professionals and subjective feedback from patients, lacking objective and quantitative evaluation standards. This subjective and lagging evaluation method, limited by incomplete and non-real-time data collection, makes it difficult to form a standardized, data-driven management loop, resulting in an inability to adjust intervention strategies in a timely and precise manner, thus restricting the level of precision in chronic disease management.

[0003] To address the problems associated with traditional follow-up models, patient follow-up systems have been developed, and attempts have been made to introduce intelligent dialogue technology. However, these existing technologies also have some shortcomings, specifically: In terms of follow-up systems, most existing systems focus on recording patient data, reminding patients of follow-up schedules, and electronically distributing standardized questionnaires. Although some systems integrate preliminary intelligent functions, their core essence remains information management tools, lacking the ability to conduct in-depth, multi-dimensional quantitative evaluation of the "real effects" of follow-up interventions, and even less able to perform reverse, automated optimization of the intervention strategy itself based on the effect evaluation results. Regarding intelligent dialogue systems, task-oriented dialogue systems are beginning to be applied in medical follow-up scenarios. Their dialogue policy learning (DPL) or optimization (DPO) typically employs reinforcement learning methods. However, research shows that the optimization objective (i.e., reward function) of such systems is usually set as short-term, process-oriented task indicators, such as the success rate of a single dialogue, the average number of dialogue rounds, or immediate user satisfaction ratings. This short-sighted optimization orientation leads the system to focus more on the fluency of immediate interaction, rather than ensuring that its dialogue strategies have a substantial positive impact on patients' long-term clinical outcomes (such as reduced relapse rates and improved quality of life). Therefore, there is currently a lack of a closed-loop mechanism that can accurately quantify and assess patients' long-term, multi-dimensional clinical effects, and use these assessment results as key feedback signals to guide and optimize dialogue strategies in real time and dynamically. Summary of the Invention

[0004] The purpose of this invention is to address at least one deficiency in the existing technology and provide a method and system for dynamic optimization of medical follow-up dialogue strategies based on closed-loop feedback. This invention is based on causal inference and dynamically transforms the quantification results of long-term effects into reinforcement learning reward signals, realizing the transformation of follow-up dialogue strategies from optimizing short-term interactive experience to improving patients' long-term clinical outcomes. It effectively solves the core problem of the separation between effect evaluation and strategy optimization in the existing technology.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a method for dynamic optimization of medical follow-up dialogue strategies based on closed-loop feedback, the method comprising the following steps: Collect multi-source heterogeneous patient data, perform preprocessing operations on the patient data, and store the preprocessed patient data in the patient data center; Based on the data from the patient data center, a quantitative assessment of follow-up effectiveness is performed periodically. The quantitative assessment of follow-up effectiveness includes: calculating the net effect value of different dialogue strategies on the long-term clinical outcomes of patients based on a multi-dimensional assessment index system and using a causal inference method. The net effect value is used as the long-term delayed reward, and the dialogue strategy is optimized based on a combined reward function that integrates the short-term immediate reward and the long-term delayed reward; wherein the weight of the long-term delayed reward in the combined reward function is dynamically adjusted according to the data size and statistical significance used in its evaluation. The optimized dialogue strategy is used to conduct follow-up interactions with patients, and the new dialogue interaction data generated is fed back to the patient data center for quantitative evaluation of follow-up effectiveness and strategy optimization in the next cycle, thus forming a closed-loop feedback.

[0006] Furthermore, the collection of multi-source heterogeneous patient data and the preprocessing of the patient data specifically include: Collect patient baseline data, dialogue interaction data, and follow-up outcome data; The collected data is subjected to standardization processing, which includes format validation, outlier removal, and missing value handling. Natural language processing techniques are used to extract medical entities from unstructured text and align them with structured data by patient ID and timestamp to obtain a dynamic, patient-centric health record timeline.

[0007] Furthermore, the multi-dimensional assessment indicator system includes indicators from at least two of the following dimensions: clinical outcome indicators, process quality indicators, patient-reported outcome indicators, and satisfaction indicators. Initial weights are assigned to each dimension of the indicators using the analytic hierarchy process or the Delphi method, and these initial weights can be dynamically adjusted according to the disease type.

[0008] Furthermore, the causal inference method is a propensity score matching method, specifically including: Based on patients' baseline data, propensity scores were calculated for patients in different dialogue strategy groups; Patients are matched based on the propensity score to construct a balanced sample group; In the matched sample group, the average treatment effect of different dialogue strategies on the preset long-term clinical outcome indicators is calculated as the net effect value.

[0009] Furthermore, before performing the propensity score matching method, the method further includes: calculating the median follow-up time of the patient cohort using the reverse Kaplan-Meier method to verify the adequacy and comparability of the follow-up data used for causal effect assessment in the time dimension.

[0010] Furthermore, the formula for calculating the combined reward function is as follows:

[0011] in, For short-term, immediate rewards, For long-term delayed rewards, , These are the weights for short-term immediate rewards and long-term delayed rewards, respectively, with the weight of long-term delayed rewards being... The adjustment method is as follows:

[0012] in, for The maximum value, Let N(t) be the Sigmoid function, and N(t) be the effective sample size used for propensity score matching analysis in the t-th evaluation period. The minimum sample size threshold is set, and p(t) is the p-value corresponding to the net effect value. and This is a hyperparameter.

[0013] Furthermore, the dialogue strategy optimization process is modeled based on Markov decision process, and the construction of its state vector includes: concatenating and fusing the dialogue context vector, patient static information vector, patient dynamic information vector, and knowledge graph entity vector.

[0014] Furthermore, the dialogue strategy includes a collaborative reinforcement learning model and a large language model. The reinforcement learning model outputs abstract dialogue actions, and the large language model generates the final natural language response based on the abstract dialogue actions. In the process of dialogue strategy optimization, the long-term delayed reward is optimized simultaneously through gradient backpropagation, thereby optimizing both the reinforcement learning model and the large language model.

[0015] Furthermore, low-rank adaptive techniques are employed to update the parameters of the large language model, so as to adapt the model-generated content to the goal of optimizing long-term clinical outcomes while maintaining the model's universality.

[0016] Secondly, the present invention also provides a dynamic optimization system for medical follow-up dialogue strategies based on closed-loop feedback. The system, based on the method described in the first aspect, includes the following modules: Data acquisition and processing module: configured to acquire multi-source heterogeneous patient data, perform preprocessing operations on the patient data, and store the preprocessed patient data in the patient data center; Follow-up effect quantitative evaluation module: configured to periodically perform follow-up effect quantitative evaluation based on the data in the patient data center. The follow-up effect quantitative evaluation includes: calculating the net effect value of different dialogue strategies on the long-term clinical outcomes of patients based on a multi-dimensional evaluation index system and using causal inference methods. The first dialogue strategy optimization module is configured to use the net effect value as a long-term delayed reward and optimize the dialogue strategy based on a combined reward function that integrates the short-term immediate reward and the long-term delayed reward; wherein the weight of the long-term delayed reward in the combined reward function is dynamically adjusted according to the data scale and statistical significance on which it is evaluated. The second dialogue strategy optimization module is configured to use the optimized dialogue strategy to conduct follow-up interactions with patients and feed the resulting new dialogue interaction data back to the patient data center for quantitative evaluation of follow-up effectiveness and strategy optimization in the next cycle, thereby forming a closed-loop feedback.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention first collects and integrates heterogeneous patient data from multiple sources, forming a unified patient data center after standardized preprocessing, providing a reliable data foundation for subsequent analysis. Then, a quantitative assessment method based on causal inference is introduced to accurately calculate the net effect value of different dialogue strategies on patients' long-term clinical outcomes. This quantitative assessment method transforms long-term clinical effects into quantifiable long-term delayed rewards, dynamically integrating them with short-term immediate rewards to form a scientific combined reward function to guide dialogue strategy optimization. Secondly, by establishing an adaptive correlation mechanism between long-term reward weights and data scale and statistical significance, it ensures a smooth transition in strategy optimization from an initial focus on interactive experience to a later focus on long-term clinical value. Finally, the new interactive data generated by the optimized strategy is fed back to the data center, forming a complete closed loop of "data collection - effect evaluation - strategy optimization - re-interaction." This effectively solves the core problem of the disconnect between effect evaluation and strategy optimization in existing medical follow-up. Simultaneously, it enables continuous self-evolution of follow-up strategies, providing intelligent and precise solutions for medical scenarios requiring long-term intervention. Attached Figure Description

[0018] Figure 1 This is a flowchart of a dynamic optimization method for medical follow-up dialogue strategy based on closed-loop feedback, according to Embodiment 1 of the present invention. Figure 2 This is a block diagram of a dynamic optimization system for medical follow-up dialogue strategy based on closed-loop feedback, as described in Embodiment 3 of this aspect. Detailed Implementation To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] Example 1 Please see Figure 1A preferred embodiment of the present invention provides a method for dynamically optimizing a medical follow-up dialogue strategy based on closed-loop feedback, comprising the following steps: S1: Collect multi-source heterogeneous patient data, perform preprocessing operations on the patient data, and store the preprocessed patient data in the patient data center; S2: Based on the data from the patient data center, periodically perform quantitative assessment of follow-up effectiveness. The quantitative assessment of follow-up effectiveness includes: calculating the net effect value of different dialogue strategies on the long-term clinical outcomes of patients based on a multi-dimensional assessment index system and using causal inference methods. S3: The net effect value is used as the long-term delayed reward, and the dialogue strategy is optimized based on the combined reward function that integrates the short-term immediate reward and the long-term delayed reward; wherein, the weight of the long-term delayed reward in the combined reward function is dynamically adjusted according to the data scale and statistical significance on which it is evaluated. S4: Use the optimized dialogue strategy to conduct follow-up interactions with patients and feed the new dialogue interaction data generated therefrom back to the patient data center for quantitative evaluation of follow-up effectiveness and strategy optimization in the next cycle, thereby forming a closed-loop feedback.

[0020] This embodiment first collects and integrates heterogeneous patient data from multiple sources. After standardized preprocessing, a unified patient data center is formed, providing a reliable data foundation for subsequent analysis. Then, a quantitative assessment method based on causal inference is introduced to accurately calculate the net effect value of different dialogue strategies on patients' long-term clinical outcomes. This quantitative assessment method transforms long-term clinical effects into quantifiable long-term delayed rewards and dynamically integrates them with short-term immediate rewards to form a scientific combined reward function to guide dialogue strategy optimization. Secondly, by establishing an adaptive correlation mechanism between long-term reward weights and data scale and statistical significance, it ensures a smooth transition from focusing on interactive experience in the initial stage to focusing on long-term clinical value in the later stage of strategy optimization. Finally, the new interactive data generated by the optimized strategy is fed back to the data center, forming a complete closed loop of "data collection - effect evaluation - strategy optimization - re-interaction." This effectively solves the core problem of the disconnect between effect evaluation and strategy optimization in existing medical follow-up. Simultaneously, it enables continuous self-evolution of follow-up strategies, providing intelligent and precise solutions for medical scenarios requiring long-term intervention.

[0021] In an optional embodiment, step S1 specifically includes: S1.1: Collect patient baseline data, dialogue interaction data, and follow-up outcome data. In this embodiment, patient baseline data includes, but is not limited to, structured data such as demographic information (age, gender), clinical diagnosis (ICD code), staging, treatment plan (surgery, medication), and comorbidities. This structured data can be obtained from the hospital's internal business systems, such as the Hospital Information System (HIS), Electronic Medical Record System (EMR), and Laboratory Information System (LIS). Patient dialogue interaction data is the dialogue log between the dialogue strategy module and the patient, including the original text, timestamps, dialogue actions executed by the system, user intent parsed by the natural language understanding module, and slot filling results. Follow-up outcome data includes various health data generated by the patient during the follow-up period, such as vital signs (blood pressure, blood glucose), laboratory test results (complete blood count, tumor markers), key information from imaging reports, standard scale scores (such as the Quality of Life (QoL) scale and the National Rating Scale (NRS)), adverse event (AE) records, and patient satisfaction survey results regarding the follow-up service.

[0022] S1.2: Perform standardization processing on the collected data. The standardization processing includes format validation, outlier removal, and missing value handling. Specific operations are shown in Table 1 below: Table 1: Rules or Technical Means for Standardized Processing

[0023] (Continued from the table above)

[0024] S1.3: Use natural language processing technology to extract medical entities from unstructured text and align them with structured data by patient ID and timestamp to obtain a dynamic health record timeline centered on the patient.

[0025] In an optional embodiment, in step S2, the multi-dimensional assessment indicator system includes indicators from at least two of the following dimensions: clinical outcome indicators, process quality indicators, patient-reported outcome indicators, and satisfaction indicators. In this embodiment, clinical outcome indicators include survival rate (e.g., 1-year and 3-year survival rates), progression-free survival (PFS), relapse rate, and complication control rate; process quality indicators include follow-up task completion rate, patient compliance (completion of medication and follow-up behavior), and achievement rate of key physiological indicators (e.g., blood pressure and blood glucose); patient-reported outcomes (PROs) use internationally recognized scales, such as the EORTC QLQ-C30 for assessing quality of life and the PHQ-9 for assessing depressive status; satisfaction indicators include patient satisfaction scores for a single conversation experience and overall follow-up services. Subsequently, initial weights are assigned to each dimension of indicators using the analytic hierarchy process (AHP) or the Delphi method, and these initial weights can be dynamically adjusted according to the disease type. In this embodiment, the customized assessment indicator system for hypertension is shown in Table 2 below: Table 2: Customized Assessment Index System for Hypertension

[0026] The customized assessment indicator system for diabetes is shown in Table 3 below: Table 3: Customized Assessment Index System for Diabetes

[0027] Furthermore, in medical dialogue systems, different dialogue strategies (such as "strong intervention," "encouragement," and "information-oriented") may be applied to patient groups with varying degrees of illness severity, leading to confounding bias and inaccurate evaluation of strategy effectiveness. To address this issue, this embodiment employs propensity score matching (PSM) for causal inference to estimate the net effect of the dialogue strategy, specifically including the following steps: (1) Based on patients' baseline data, propensity scores were calculated for patients grouped under different dialogue strategies. Specifically, baseline data of patients was first collected, including quantitative or categorical variables such as age, gender, and disease stage. For example, assuming a dataset containing 1000 patients, each patient had age (continuous variable), gender (binary variable, 0 for female, 1 for male), and disease stage (categorical variable, such as stage I, II, III); then, based on the type of dialogue strategy mainly received by patients over a period of time, patients were divided into treatment and control groups. For example, T=1 was defined as the "strong intervention" strategy group (treatment group), and T=0 as the "information-oriented" strategy group (control group). Finally, propensity scores were calculated using a multivariate logistic regression model. The patient's baseline covariates (baseline data) were X=(X1,X2,...,X...). k The independent variable is , and the dependent variable is the dialogue strategy type T (T=1 or 0). For each patient i, their propensity score P(S) is... i ) indicates that, given baseline features The probability that the patient will receive a "strong intervention" strategy (T=1) is calculated using the following formula:

[0028] in, It is a linear combination, and the calculation formula is as follows:

[0029] (2) Patients are matched according to the propensity score to construct a balanced sample group. In this embodiment, the matching method is nearest neighbor matching (1:1 matching), that is, in the control group, the individual with the closest propensity score for each patient in the treatment group is matched. A caliper limit can be set during matching, for example, the score difference should not exceed 0.05, to ensure the quality of matching. After matching, the baseline characteristics of the treatment group and the control group should be balanced. Statistical tests (such as t-test or chi-square test) are used to verify that there are no significant differences between the two groups in terms of age, gender, disease stage, etc. (p value > 0.05), thereby simulating a "pseudo-randomized controlled trial" scenario.

[0030] (3) In the matched sample group, the average treatment effect of different dialogue strategies on the preset long-term clinical outcome indicators is calculated as the net effect value (ATT). The formula for calculating the net effect value ATT is as follows:

[0031] in, The number of samples in the treatment group. The sum of outcomes for all patients in the treatment group. Let be the set of control patients matched with patient i in the treatment group. In a 1:1 match, this set contains only one patient j. The sum of outcomes for all control group patients matched with the treatment group is calculated. This ATT value is used as the net effect value of the dialogue strategy and directly input into the "Dialogue Strategy Optimization Module" of the reinforcement learning system as the long-term reward signal Rlong. For example, if the ATT is +5 points, Rlong is set to 5; if the ATT represents a 15% blood pressure target achievement rate, Rlong is set to 0.15. This signal is used to guide model updates and optimize the dialogue strategy.

[0032] Furthermore, in clinical efficacy assessments based on real-world data, accurately estimating the follow-up time of the patient cohort is crucial. Directly calculating the average time from enrollment to the outcome event (e.g., death) or last contact for all patients can lead to a systematic underestimation of the overall follow-up time due to the short follow-up time of patients who experience early outcomes. To verify the adequacy and comparability of follow-up data for causal effect assessments (e.g., propensity score matching) across the time dimension, this embodiment first uses the inverse Kaplan-Meier method to calculate the median follow-up time of the entire patient cohort before performing PSM. This provides an unbiased estimate of the follow-up time, ensuring that the observation window on which subsequent causal inference is based is sufficiently long and comparable across different strategy groups, thereby enhancing the reliability of the research conclusions. Specifically, in traditional survival analysis, loss to follow-up is considered censored data. However, when calculating follow-up time, direct averaging can lower the overall average due to the short follow-up time of patients who experience early outcomes (e.g., death), leading to an underestimation of the follow-up time. This invention employs the reverse KM method, defining "occurrence of an outcome event" as censoring and "loss to follow-up or study termination" as outcome events. This approach allows for a more accurate estimation of the median follow-up time of the cohort, providing a reliable time baseline for subsequent analysis.

[0033] In an optional embodiment, in step S3, in a medical dialogue system, policy optimization needs to simultaneously consider both short-term interactive experience and long-term clinical effects. Traditional methods often only focus on the quality of immediate dialogue, making it difficult to have a sustained positive impact on patient health outcomes. This embodiment proposes an optimization framework based on the synergy of reinforcement learning (RL) and large language model (LLM), which guides the system policy to evolve in a direction that improves long-term clinical value through a dynamically weighted combined reward function.

[0034] First, the dialogue management process is modeled as a Markov Decision Process (MDP), and its core elements are defined as follows: State (S): A high-dimensional vector constructed by fusing information from multiple sources, providing the model with a comprehensive decision context.

[0035] Information source and encoding method: Dialogue context vector ( ): The Transformer encoder is used to encode the current dialogue history to obtain a fixed-dimensional semantic vector.

[0036] Patient static information vector ( ): For categorical variables such as disease type and stage, the embedding layer is used to convert them into dense vectors.

[0037] Patient dynamic information vector ( ): After normalizing recent clinical indicators (such as blood pressure and blood sugar), they are encoded through a small fully connected network.

[0038] Knowledge graph entity vectors ( When a dialogue mentions a medical entity (such as the drug "metformin"), a graph embedding algorithm (such as TransE) is used to extract the vector representation of that entity from the knowledge graph; if there is no relevant entity, the zero vector is used.

[0039] Concatenate all the above vectors along the feature dimension to form the final state representation: S = Concatenate( ) Action (A): An abstract "dialogue action" output by the system in each dialogue turn, rather than directly generated text. For example, [ , , ].

[0040] Reward (R): A dynamically weighted composite reward function is used, calculated as follows:

[0041] in, This is a short-term, immediate reward, awarded after each conversational interaction to ensure basic fluency and task completion. For example, successfully answering a patient's question (+1), the patient expressing positive emotions (+2), and clarification or repetition in the conversation (-1). This is a long-term delayed reward, derived from the causal effect assessment results periodically output by the follow-up effect quantification module. It is a sparse and delayed reward signal. For example, the assessment module might find that, in matched samples, an "encouraging" dialogue strategy, compared to an "information-oriented" strategy, resulted in an average increase of 5 points in patients' QoL scores over one year. This "+5" effect value would then be used as R_long and assigned to dialogue trajectories that implemented the "encouraging" strategy in past assessment periods. , These are the weights for short-term immediate rewards and long-term delayed rewards, respectively, with the weight of long-term delayed rewards being... The adjustment method is as follows:

[0042] in, for The maximum value, The sigmoid function smoothly maps the input to the (0, 1) interval, achieving non-linear asymptotic growth. N(t) is the effective sample size for propensity score matching analysis in the t-th evaluation period. The minimum sample size threshold is set, and p(t) is the p-value corresponding to the net effect value. The smaller the p-value, the more significant the effect and the more reliable the result. and This is a hyperparameter. It should be noted that in the initial stage of system operation, follow-up data is sparse. The confidence level is low, at this time With a smaller weight (e.g., 0.1), the system focuses more on optimizing the short-term interactive experience. As data accumulates... The assessment results are becoming increasingly robust. The weight of the factor will gradually increase (e.g., tending towards 0.5 or higher), thereby guiding the dialogue strategy to focus more on the positive impact on long-term clinical outcomes.

[0043] Policy learning model and update: Basic RL algorithm: The Proximal Policy Optimization (PPO) algorithm is adopted, which is suitable for the discrete action space of this scenario and has good training stability. RL and LLM collaboration: Separation of decision and generation: The RL model acts as the "decision maker," outputting the probability distribution of an abstract dialogue action A (such as provide_encouragement) based on the state S. This action is then fed into a pre-trained large language model (such as GPT-4) through a carefully designed prompt template (Prompt Engineering), and the LLM generates specific, natural, and empathetic response text. Collaborative update mechanism: Gradient signals from the combined rewards (especially R_long) are backpropagated through the policy gradient algorithm, simultaneously optimizing the parameters of both the RL model and the LLM.

[0044] Specifically, updating the RL model, As a delayed reward, it is allocated to the entire dialogue trajectory that produces this long-term effect. The goal of RL models (such as PPO or DQN) is to maximize the sum of these components. The cumulative reward, including the policy gradient. This directly affects the calculation of the advantage function. A positive... This increases the probability of all state-action pairs occurring in the trajectory, making the model more inclined to make decisions that bring long-term benefits (i.e., outputting more advantageous abstract dialogue actions). In the Q-Learning method, It will be used to update the Q-value. It backpropagates the long-term value to each state in the trajectory through the Bellman equation, ensuring that states leading to high long-term rewards have higher Q-values. The update for LLM is an indirect, gradient-guided update, not a direct use of... Instead of fine-tuning as a reward, the gradients of the RL model are used to "guide" the generative style of the LLM. This can be achieved through a variant framework called RLHF (Reinforcement Learning from Human Feedback), in which "Human Feedback" is replaced with "Long-term Outcome Feedback" in this embodiment. The technical implementation path is as follows: (1). The RL model outputs an abstract dialogue action a (such as provide_encouragement).

[0045] (2). LLM generates specific response text y based on this action a and the current dialogue context.

[0046] (3). During the training process of the RL model, when the calculation is... The resulting policy gradient At that time, this gradient is used not only to update the parameters of the RL model. .

[0047] (4) This embodiment establishes a gradient path from the RL model to the LLM. The most direct way is to treat the output of the RL model (the probability distribution of actions) as a differentiable input to the LLM generation process. When the RL model is generated by a good... When adjusting its strategy, the gradient signal for this adjustment can be backpropagated to the parameters of the LLM. superior.

[0048] (5) In order to avoid breaking the general language capability of LLM, only some of its parameters are usually updated, for example, by using the LoRA (Low-Rank Adaptation) technique. This means that the gradient will only update the low-rank adapter matrix inserted into the LLM, while the main part of the LLM remains frozen.

[0049] Example 2 This embodiment fully demonstrates the actual workflow of the dialogue strategy optimization system through two typical cases (patient A with hypertension and patient B with diabetes).

[0050] The patient's baseline data is shown in Table 4: Table 4: Patient baseline data

[0051] I. Initial Interaction and Data Accumulation (Months 1-3) 1. System input: (1). Baseline data of patients A and B; (2). Initial dialogue strategy (strategy 1: information-oriented).

[0052] 2. Dialogue interaction process: (1) The system to patient A: "Hello Mr. Zhang, please remember to measure your blood pressure today. Your current blood pressure control target is below 140 / 90 mmHg." (2) The system to patient B: "Hello Ms. Li, please remember to monitor your 2-hour postprandial blood glucose today. According to the guidelines, your target is below 10.0 mmol / L." 3. Data acquisition and output: (1) Dialogue interaction data: Both patients completed part of the dialogue, but their participation was not high, and they sometimes ignored system prompts. (2). Follow-up results: Patient A's blood pressure fluctuated greatly, and the "blood pressure target achievement rate" was 40% during the period; Patient B's blood glucose target achievement rate was about 60%, indicating confusion about dietary control, and the "Diabetes Distress Scale (DDS)" score was 3.5.

[0053] 4. Reward Calculation: (1) Only short-term rewards Received +1 reward for completing the dialogue; (2) Long-term rewards The weight is 0. Approaching 0; (3) The total reward value is low, reflecting the limited effectiveness of the initial strategy.

[0054] II. Periodic Assessment and Long-Term Incentive Generation (End of Month 3) Assessment inputs: baseline data, conversational data, and follow-up results data of all patients over the past 3 months, including quantitative indicators such as "blood pressure target achievement rate" and "DDS score".

[0055] 1. Quantitative evaluation process: (1). Application of indicator system: For patients with hypertension, the "blood pressure target achievement rate" is the core indicator. For patients with diabetes, the "DDS score" is the core indicator.

[0056] (2). PSM analysis: Using age, disease duration, comorbidities, etc. as covariates, patients receiving strategy 1 (information-oriented) and strategy 2 (encouragement + personalized education) were matched to form a comparison group with balanced baseline characteristics.

[0057] (3). Effect estimation (ATT): Hypertension group: The average blood pressure target achievement rate was 65% in Strategy 2 group and 50% in Strategy 1 group, with ATT=+15% (p=0.04). In the diabetes group: the average DDS score of strategy 2 was 2.0, that of strategy 1 was 3.2, and the ATT score was -1.2 (p=0.03).

[0058] 2. Evaluation Output: Assign long-term rewards to dialogue tracks that have implemented "Strategy 2" in the past 3 months: Hypertension-related tracks: = +0.15; Diabetes-related trajectory: = +1.2 III. Dialogue Strategy Updates and Iterations (Early April) 1. System input: Newly generated long-term reward signal (+0.15 or +1.2); 2. Strategy update process: (1) Dynamic weight adjustment: Due to sufficient data accumulation and significant evaluation results; The weight was increased from 0.1 to 0.3; The total reward function becomes: R = 0.7 × + 0.3× .

[0059] (2). Offline training of the model: The RL model is retrained using historical data containing higher R_long values; the model learns that executing "Policy 2" will yield higher cumulative rewards.

[0060] (3). Strategy evolution: Update the policy network parameters of the RL model; for similar states of patients A and B, the probability of outputting "strategy 2" related actions is significantly increased; including abstract actions such as provide_encouragement, ask_difficulty_and_educate.

[0061] 3. System output: The optimized new dialogue strategy significantly favors the use of an "encouraging + personalized education" interaction approach.

[0062] IV. Optimized follow-up interaction (4th month and beyond) 1. Optimized dialogue interaction: The system addressed Patient A: "Good morning, Mr. Zhang! It's a new day, let's work together to control our blood pressure. I noticed your blood pressure was a little high yesterday. Did you do any activity before the measurement? Or is there anything that's bothering you? Remember to relax during the measurement." The system's message to Patient B: "Ms. Li, good afternoon! I saw you recorded your lunch, great job! Regarding your dietary concerns, I've found that many people in similar situations have achieved good results using the 'plate method' (half vegetables, 1 / 4 staple food, 1 / 4 protein). Would you like to learn more?" 2. Expected results: Patient A will feel cared for, have better compliance, be more willing to interact, and the blood pressure target achievement rate is expected to increase.

[0063] Patient B received specific and actionable advice, resolved their confusion, improved their self-management efficacy, and their expected DDS score decreased.

[0064] This embodiment demonstrates a complete closed-loop optimization process of "interaction → accumulation → evaluation → update → re-interaction". Over time, the system can not only discover the overall advantages of "encouraging" strategies, but also further identify the optimal interaction methods for different patient subgroups (such as different age groups and different complications), achieving truly personalized follow-up and ultimately achieving the fundamental goal of improving patients' long-term clinical outcomes. Through this data-driven continuous optimization mechanism, the medical dialogue system can continuously adapt to patient needs, ensuring natural and fluent dialogue while effectively improving health management results.

[0065] Example 3 Please see Figure 2 An embodiment of the present invention provides a dynamic optimization system for medical follow-up dialogue strategies based on closed-loop feedback, comprising the following modules: Data acquisition and processing module: configured to acquire multi-source heterogeneous patient data, perform preprocessing operations on the patient data, and store the preprocessed patient data in the patient data center; Follow-up effect quantitative evaluation module: configured to periodically perform follow-up effect quantitative evaluation based on the data in the patient data center. The follow-up effect quantitative evaluation includes: calculating the net effect value of different dialogue strategies on the long-term clinical outcomes of patients based on a multi-dimensional evaluation index system and using causal inference methods. The first dialogue strategy optimization module is configured to use the net effect value as a long-term delayed reward and optimize the dialogue strategy based on a combined reward function that integrates the short-term immediate reward and the long-term delayed reward; wherein the weight of the long-term delayed reward in the combined reward function is dynamically adjusted according to the data scale and statistical significance on which it is evaluated. The second dialogue strategy optimization module is configured to use the optimized dialogue strategy to conduct follow-up interactions with patients and feed the resulting new dialogue interaction data back to the patient data center for quantitative evaluation of follow-up effectiveness and strategy optimization in the next cycle, thereby forming a closed-loop feedback.

[0066] The system proposed in this embodiment is based on the method described in Embodiment 1. Therefore, the options proposed in Embodiment 1 are also applicable to this embodiment. To avoid repetition, they will not be described again here.

[0067] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A dynamic optimization method for medical follow-up dialogue strategies based on closed-loop feedback, characterized in that, The method includes the following steps: Collect multi-source heterogeneous patient data, perform preprocessing operations on the patient data, and store the preprocessed patient data in the patient data center; Based on the data from the patient data center, a quantitative assessment of follow-up effectiveness is performed periodically. The quantitative assessment of follow-up effectiveness includes: calculating the net effect value of different dialogue strategies on the long-term clinical outcomes of patients based on a multi-dimensional assessment index system and using a causal inference method. The net effect value is used as the long-term delayed reward, and the dialogue strategy is optimized based on a combined reward function that integrates the short-term immediate reward and the long-term delayed reward; wherein the weight of the long-term delayed reward in the combined reward function is dynamically adjusted according to the data size and statistical significance used in its evaluation. The optimized dialogue strategy is used to conduct follow-up interactions with patients, and the new dialogue interaction data generated is fed back to the patient data center for quantitative evaluation of follow-up effectiveness and strategy optimization in the next cycle, thus forming a closed-loop feedback.

2. The method according to claim 1, characterized in that, The process of collecting multi-source heterogeneous patient data and performing preprocessing operations on the patient data specifically includes: Collect patient baseline data, dialogue interaction data, and follow-up outcome data; The collected data is subjected to standardization processing, which includes format validation, outlier removal, and missing value handling. Natural language processing techniques are used to extract medical entities from unstructured text and align them with structured data by patient ID and timestamp to obtain a dynamic, patient-centric health record timeline.

3. The method according to claim 2, characterized in that, The multi-dimensional assessment indicator system includes indicators from at least two of the following dimensions: clinical outcome indicators, process quality indicators, patient-reported outcome indicators, and satisfaction indicators. Initial weights are assigned to each dimension of the indicators using the analytic hierarchy process or the Delphi method, and these initial weights can be dynamically adjusted according to the disease type.

4. The method according to claim 3, characterized in that, The causal inference method is a propensity score matching method, specifically including: Based on patients' baseline data, propensity scores were calculated for patients in different dialogue strategy groups; Patients are matched based on the propensity score to construct a balanced sample group; In the matched sample group, the average treatment effect of different dialogue strategies on the preset long-term clinical outcome indicators is calculated as the net effect value.

5. The method according to claim 4, characterized in that, Prior to performing the propensity score matching method, the method further includes: calculating the median follow-up time of the patient cohort using the reverse Kaplan-Meier method to verify the adequacy and comparability of the follow-up data used for causal effect assessment in the time dimension.

6. The method according to any one of claims 1-5, characterized in that, The formula for calculating the combined reward function is as follows: in, For short-term, immediate rewards, For long-term delayed rewards, , These are the weights for short-term immediate rewards and long-term delayed rewards, respectively, with the weight of long-term delayed rewards being... The adjustment method is as follows: in, for The maximum value, Let N(t) be the Sigmoid function, and N(t) be the effective sample size used for propensity score matching analysis in the t-th evaluation period. The minimum sample size threshold is set, and p(t) is the p-value corresponding to the net effect value. and This is a hyperparameter.

7. The method according to claim 6, characterized in that, The dialogue strategy optimization process is based on Markov decision process modeling. The construction of its state vector includes: concatenating and fusing the dialogue context vector, patient static information vector, patient dynamic information vector and knowledge graph entity vector.

8. The method according to claim 7, characterized in that, The dialogue strategy comprises a collaborative reinforcement learning model and a large language model. The reinforcement learning model outputs abstract dialogue actions, and the large language model generates a final natural language response based on the abstract dialogue actions. Furthermore, during the dialogue strategy optimization process, the long-term delayed reward is optimized simultaneously through gradient backpropagation, thereby optimizing both the reinforcement learning model and the large language model.

9. The method according to claim 8, characterized in that, The parameters of the large language model are updated using a low-rank adaptive technique to ensure that the generated content adapts to the goal of optimizing long-term clinical outcomes while maintaining the model's generality.

10. A dynamic optimization system for medical follow-up dialogue strategies based on closed-loop feedback, characterized in that, The system, based on the method according to any one of claims 1-9, includes the following modules: Data acquisition and processing module: configured to acquire multi-source heterogeneous patient data, perform preprocessing operations on the patient data, and store the preprocessed patient data in the patient data center; Follow-up effect quantitative evaluation module: configured to periodically perform follow-up effect quantitative evaluation based on the data in the patient data center. The follow-up effect quantitative evaluation includes: calculating the net effect value of different dialogue strategies on the long-term clinical outcomes of patients based on a multi-dimensional evaluation index system and using causal inference methods. The first dialogue strategy optimization module is configured to use the net effect value as a long-term delayed reward and optimize the dialogue strategy based on a combined reward function that integrates the short-term immediate reward and the long-term delayed reward; wherein the weight of the long-term delayed reward in the combined reward function is dynamically adjusted according to the data scale and statistical significance on which it is evaluated. The second dialogue strategy optimization module is configured to use the optimized dialogue strategy to conduct follow-up interactions with patients and feed the resulting new dialogue interaction data back to the patient data center for quantitative evaluation of follow-up effectiveness and strategy optimization in the next cycle, thereby forming a closed-loop feedback.

Images