A dynamic follow-up strategy generation method for post-diagnosis management

Abstract

The application discloses a kind of dynamic follow-up strategy generation methods for post-diagnosis management, it is related to medical information processing technical field.Processing method includes: obtaining the original data of patient in current time step, using pre-training language model coding generation first patient state vector and preset candidate action set, the dynamic strategy score of combining reinforcement learning action value and clinical guideline compliance score, the candidate action with maximum dynamic strategy score is regarded as optimal follow-up action;The weight of dynamic strategy score is adjusted by a dynamic balance factor based on patient state uncertainty, realizes the individualization, dynamic self-adapting optimization of follow-up strategy, through clinical guideline constraint and uncertainty perception mechanism, ensure the clinical compliance and safety of decision-making process, improve post-diagnosis management efficiency and patient health outcome.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more particularly to the fields of artificial intelligence and medical information processing. Specifically, this invention relates to a method, electronic device, and storage medium for post-diagnosis dynamic follow-up decision-making using reinforcement learning and large language models. Background Technology

[0002] Post-diagnosis management is a crucial component of the modern healthcare system. Its goal is to consolidate treatment outcomes, prevent disease recurrence, improve patients' quality of life, and reduce readmission rates by continuously tracking, intervening in, and guiding patients' health status after discharge or outpatient care. Traditional post-diagnosis management relies heavily on standardized, static follow-up plans, such as fixed telephone follow-up times and universally applicable health education materials. This "one-size-fits-all" approach fails to meet the individualized needs of different patients and struggles to adapt to dynamic changes in patient condition, psychological state, and compliance, resulting in lower follow-up efficiency and effectiveness. With the development of artificial intelligence, the application of dialogue systems to assist healthcare services is increasing. However, existing technologies primarily target real-time, diagnostic medical inquiry scenarios. When applied to long-term, non-real-time post-diagnosis management, they suffer from the following technical limitations: First, the short-term and static nature of strategies. Existing strategy learning technologies primarily aim to generate appropriate responses for the current dialogue turn, with their state and action spaces designed around a single conversation. It lacks an optimization goal oriented towards long-term health outcomes, failing to form a dynamic and coherent follow-up "strategy" spanning weeks or even months, and merely generates responses at the "tactical" level. Second, clinical compliance and safety assurance are insufficient. Although existing technologies may utilize knowledge graphs to assist in response generation, their decision-making process is mainly dominated by data-driven models, lacking a clear and dynamically adjustable mechanism to ensure that all recommendations strictly adhere to clinical guidelines. In exploring personalized interactions, there is a risk of generating recommendations that deviate from evidence-based medicine, especially when the model faces patient states with high uncertainty, where safety risks are particularly prominent. Third, the interactive experience is limited. Existing technologies mostly employ generative models based on GRU or similar architectures, which lag behind current advanced generative large language models in terms of the naturalness, personalization, and empathy of the generated language, potentially leading to stiff interactions and affecting patient engagement and adherence. Therefore, existing technologies urgently need a method and system that can dynamically generate personalized follow-up strategies for long-term post-diagnosis management scenarios, and can self-optimize through continuous interaction with patients while ensuring clinical compliance. Summary of the Invention

[0003] The purpose of this invention is to provide a method, electronic device, and storage medium for generating dynamic follow-up strategies for post-diagnosis management, to solve one or more technical problems existing in the prior art, such as static follow-up strategies, lack of personalization, insufficient guarantee of clinical compliance, and poor user experience. To achieve the above objective, this invention provides the following technical solution:

[0004] In a first aspect, the present invention provides a method for generating dynamic follow-up strategies for post-diagnosis management, comprising the following steps: obtaining the current time step. The patient's original data was used, and a first patient state vector was generated by encoding it using a pre-trained language model. Based on the first patient state vector and the preset set of candidate actions By calculating each candidate action Dynamic strategy score The candidate action with the highest dynamic policy score is selected as the optimal follow-up action. .

[0005] As a preferred option, the dynamic strategy score The calculation formula is as follows: ,in, The candidate action with the highest dynamic policy score, To enhance learning strategy networks The output is in state Next action Action value function, In order to be in the clinical guideline knowledge base Actions under constraints Compliance score, As a dynamic equilibrium factor; execute the optimal follow-up action. And receive patient feedback to obtain the next time step. reward signal Second patient state vector ; Using state transition quadruples For the reinforcement learning policy network Update the first patient state vector. As a preferred embodiment, the first patient state vector... The generation steps include: acquiring the patient's electronic health record (EHR) data and historical dialogue data respectively; encoding the EHR data using a first Transformer encoder to obtain a first feature embedding; encoding the historical dialogue data using a second Transformer encoder to obtain a second feature embedding; and concatenating or weighting the first and second feature embeddings to generate the first patient state vector. As a preferred embodiment, the dynamic balance factor... The calculation method is as follows: ,in, Based on the first patient state vector The entropy value of the predicted future health status distribution is used to quantify state uncertainty; and These are preset, learnable parameters. As a preferred option, the compliance score... The calculation steps include: [converting the clinical guideline knowledge base...] Constructed as a knowledge graph; based on the first patient state vector Retrieve the corresponding set of recommended operations from the knowledge graph; calculate the candidate actions. The cosine similarity between the action embedding and the embeddings of each operation in the recommended operation set is used, and the highest similarity value is taken as the compliance score. As a preferred embodiment, the reward signal... The reward is calculated using a multi-objective reward function, which includes a weighted sum of rewards based on changes in patient physiological indicators, satisfaction rewards based on patient feedback and emotional analysis, and compliance rewards based on patient adherence to medical orders. As a preferred embodiment, the reinforcement learning policy network... The update employs the Deep Q-Network (DQN) algorithm, by minimizing the following loss function. To update network parameters : ,in, For mathematical expectation operators, For experience replay pool, As a discount factor, For the next action, These are the parameters of the target network. As a preferred embodiment, the optimal follow-up action is performed. The steps include: implementing the optimal follow-up action. The process involves mapping the text to one or more natural language commands; inputting these commands into a pre-trained generative large language model to generate context-aware, personalized follow-up dialogue text; and sending the follow-up dialogue text to the patient via instant messaging, in-app notifications, or SMS. As a preferred embodiment, the candidate action set... This includes at least the following types of actions: symptom inquiry, medication reminders, follow-up appointment scheduling, health education, lifestyle recommendations, and psychological counseling. As a preferred option, this method is initiated after the patient completes their initial consultation and is implemented according to the optimal follow-up actions. The type and content of the follow-up visits are used to dynamically adjust the execution interval of the next follow-up visit. Secondly, this application provides an electronic device, including:

[0006] A processor; and a memory storing computer instructions that, when executed by the processor, cause the electronic device to perform the method described in the first aspect above.

[0007] Thirdly, this application provides a computer-readable storage medium containing program instructions that, when executed by a processor, cause the method described in the first aspect to be implemented. Compared with the prior art, the beneficial effects of the present invention are:

[0008] This invention achieves personalized and dynamically adaptive follow-up strategies. Employing a reinforcement learning framework with the goal of maximizing long-term cumulative rewards, the system continuously and automatically adjusts and optimizes subsequent follow-up action sequences based on each patient's real-time feedback and status changes. This forms a truly personalized and dynamic strategy, overcoming the shortcomings of existing technologies where strategies are static, highly universal, and lack personalization.

[0009] This enhances the safety and clinical compliance of decision-making. This invention uniquely proposes a dynamic equilibrium factor... The system employs a strategy scoring function that integrates the value of data-driven reinforcement learning with compliance scores based on clinical guidelines. In particular, a balancing factor is linked to the uncertainty of the patient's condition, enabling the system to automatically favor more conservative and safer clinical guideline recommendations when faced with uncertainty. This constructs an intelligent "safety barrier" that significantly reduces the potential risks of AI decision-making.

[0010] This invention enhances patient engagement and adherence. It utilizes an advanced generative large language model to transform abstract follow-up actions into personalized, empathetic, and context-aware conversational text. Compared to templated, rigid messages, this high-quality interaction significantly improves patients' willingness and experience of communication, thereby increasing their adherence to medication, follow-up appointments, and other medical orders. This adherence is then fed back to the model through a multi-objective reward function, creating a positive feedback loop.

[0011] A more comprehensive patient state representation has been constructed. This invention constructs a multimodal and more comprehensive patient state vector by fusing structured electronic health record (EHR) data and unstructured historical dialogue data. This rich state representation provides reinforcement learning agents with more accurate decision-making basis, enabling them to perceive subtle changes that cannot be detected by a single data source, thereby making better follow-up decisions. Attached Figure Description To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is an overall flowchart of a dynamic follow-up strategy generation method for post-diagnosis management provided by an embodiment of the present invention. Figure 2 This is a detailed flowchart of the dynamic strategy score calculation and action selection steps in a dynamic follow-up strategy generation method for post-diagnosis management provided in an embodiment of the present invention. Detailed Implementation The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that the terminology used herein is only for describing specific implementations and is not intended to limit the exemplary implementations of the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof. Refer to Figure 1This invention provides a method for generating dynamic follow-up strategies for post-diagnosis management. This method starts after a patient completes their initial treatment and runs continuously throughout a long post-diagnosis management process. The method can be executed by a system deployed on a cloud server or a dedicated medical server and interacts with the patient through terminals such as mobile applications, mini-programs, or SMS services. The steps of this method are described in detail below. Step 101: Acquire and encode patient data to generate a state vector. At each decision-making time step (For example, daily, every three days, or at time points dynamically determined by the system), the system first acquires the latest raw data related to the patient. This data is multimodal and mainly includes two parts: one part is structured electronic health record (EHR) data, such as the latest diagnostic records, laboratory test results (e.g., blood glucose, blood pressure values), prescription information, allergy history, etc. The other part is unstructured historical dialogue data, i.e., all interactive texts between the system and the patient since the last follow-up, including the patient's active description of their symptoms, answers to questions, emotional expressions, etc. To transform this heterogeneous data into a unified representation that the model can understand, this embodiment employs a two-stream Transformer encoder structure. Specifically: the EHR data is encoded using a first Transformer encoder. First, discrete EHR entries (e.g., diagnostic codes, drug names) and continuous numerical values ​​(e.g., test results) are mapped into vectors through an embedding layer, and then input into the first Transformer encoder. Through its self-attention mechanism, the inherent correlation between various medical records is captured, and the first feature embedding is output. A second Transformer encoder (e.g., BERT or its medical domain variant) is used to encode the historical dialogue text, capturing the semantic information and contextual dependencies of the dialogue, and outputting a second feature embedding. Finally, the first and second feature embeddings are concatenated or weighted and fused to generate the current time step. Comprehensive first patient state vector By integrating structured EHR data with unstructured dialogue history, a comprehensive and dynamic patient profile can be constructed, providing more accurate input for subsequent decision-making. Step 102: Calculate the dynamic strategy score to determine the optimal follow-up action. This step is the core of the decision-making process in this invention, as referred to... Figure 2 The detailed process is shown below. The system is based on the current state vector. and a preset set of candidate actions To select the optimal next action. The candidate action set. It is a discrete set encompassing all possible follow-up actions, which may include, for example: symptom inquiry (e.g., "asking if the headache has subsided"), medication reminders (e.g., "reminding you to take your blood pressure medication"), follow-up appointment scheduling (e.g., "suggesting a follow-up appointment next week"), health education (e.g., "sending an article about a low-sodium diet"), lifestyle recommendations (e.g., "suggesting 30 minutes of aerobic exercise daily"), and psychological support (e.g., "identifying negative emotions and sending messages of care and encouragement"). For the set... Each candidate action in The system calculates its dynamic strategy score. The calculation formula is as follows: The formula contains three key components:

[0012] Reinforcement learning action value function This item is composed of a deep neural network (i.e., a policy network). The output represents the current state. Next action This is an estimate of the long-term cumulative rewards that the model is expected to generate. It represents the "experienced" or "exploratory" part of the model, designed to discover personalized strategies that maximize long-term health benefits for patients.

[0013] Clinical guideline compliance score This step ensures the safety and professionalism of the procedure. First, a clinical guideline knowledge base is constructed by integrating authoritative clinical guidelines and expert consensus. This is further represented as a knowledge graph. Given a patient's state... (For example, information including "type 2 diabetes" and "high fasting blood glucose"), the system uses the knowledge graph. The system retrieves a set of recommended actions related to the current state (e.g., recommending "monitor blood glucose" or "dietary guidance"). Then, candidate actions are calculated. The compliance score is calculated by taking the cosine similarity between the action embedding and the embeddings of each operation in the recommended operation set, with the maximum similarity value being the compliance score. The higher the score, the better the performance. The more it conforms to the requirements of evidence-based medicine, the better. By introducing clinical guideline knowledge base constraints, it is ensured that every candidate action generated by the system has been tested by evidence-based medicine, greatly improving the system's safety and reliability.

[0014] Dynamic equilibrium factor This factor is used to intelligently weigh the two scores mentioned above. Its calculation method is as follows: ,in, Based on state vector The entropy value represents the predicted distribution of patients' future health conditions. The higher the entropy value, the more uncertain the model's prediction of the patients' future trajectory. and These are learnable parameters. When uncertainty is high, Increase Approaching 1 makes the dynamic policy score... Mainly composed of compliance score When uncertainty is high, the system behaves more conservatively. Conversely, when uncertainty is low, As the value approaches zero, the system trusts its experience learned through reinforcement learning more. This dynamic balancing mechanism enables the system to intelligently weigh the exploration of optimal personalized strategies against adherence to established medical guidelines, automatically switching to a more conservative and safer mode under conditions of high uncertainty. After calculating the dynamic strategy scores of all candidate actions, the system selects the action with the highest score as the optimal follow-up action. Step 103: Execute the action, obtain feedback and new state. Determine the optimal action. Next, the system needs to convert it into natural language that the patient can understand and send it. This step is accomplished using a pre-trained generative large language model (such as the GPT series models). Specifically, the abstract action instructions are... (For example, `action:symptom_inquiry, topic:headache, tone:empathetic`) is input as a prompt to the large language model, which generates a context-appropriate, tone-appropriate, and personalized follow-up dialogue text. For example, it might generate: "Hello, I noticed you mentioned headaches yesterday. Are you feeling better today?" The generated text is sent to the patient via instant messaging, in-app notifications, or SMS. The system then waits for and receives feedback from the patient. Feedback includes the patient's text reply and physiological data uploaded through wearable devices (such as steps and heart rate). The system processes this feedback to obtain two key outputs: a reward signal. This embodiment uses a multi-objective reward function to calculate the reward value. This function is a weighted sum of multiple sub-rewards, for example:

[0015] Health improvement reward: calculated based on the change in the patient's physiological indicators (such as blood sugar and blood pressure) towards the healthy range.

[0016] Satisfaction reward: Sentiment analysis is performed on patient responses; positive sentiment is rewarded with positive rewards, and negative sentiment is rewarded with negative rewards.

[0017] Adherence reward: If the action is medication or a follow-up appointment reminder, a reward is given based on whether the patient confirms completion. Using a multi-objective reward function guides the reinforcement learning process towards optimizing for overall improvement in patient health, satisfaction, and adherence, rather than solely pursuing a single, one-sided indicator. Secondly, the patient's state vector at the next time step. The system integrates the new feedback information with the previous state information, and then generates an updated second patient state vector through the encoding process described in step 101. Step 104: Update the reinforcement learning policy network. The system will generate the state transition quadruple from this interaction. Stored in a pool called the "experience replay pool". Within a large buffer. During training, the system will draw from the experience replay pool. A mini-batch of past experiences is randomly sampled. For each sampled experience, a Deep Q-Network (DQN) algorithm is used to update the policy network. parameters Specifically, this is achieved by minimizing the following temporal difference loss function. To optimize: ,in, For mathematical expectation operators, For experience replay pool, For the next action, It is a discount factor between 0 and 1, representing the degree of importance attached to future rewards. It is a parameter of an independent "target network," and it is a parameter of the main network. A periodic copy is used to stabilize the training process. This loss function is continuously minimized using gradient descent. This method will gradually learn to more accurately predict the long-term value of performing different actions under different states. Furthermore, it includes a dynamic time adjustment mechanism based on the currently selected optimal follow-up action. The system can dynamically adjust the execution interval of the next follow-up based on the type and content of the follow-up. For example, if the current action is an urgent symptom inquiry and the patient's feedback is poor, the system may shorten the next follow-up interval to a few hours; if it is just routine health education, the interval may be set to several days. This invention also provides a corresponding system embodiment, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements all the steps of the above method. For example, the memory stores a clinical guideline knowledge base, a pre-trained language model, a reinforcement learning policy network model, and experience replay pool data. The processor is responsible for performing all computational tasks such as data acquisition, state encoding, policy score calculation, action selection, interaction with the large language model, reward calculation, and network parameter updates. In summary, this invention constructs a closed-loop, self-optimizing dynamic follow-up strategy generation framework by combining reinforcement learning, large language models, and knowledge graph constraints. It can not only provide highly personalized post-diagnosis management services for each patient, but also ensure the safety and compliance of the decision-making process through an innovative dynamic balancing mechanism, thus demonstrating great application potential in improving the quality and efficiency of medical services. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The above descriptions are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating dynamic follow-up strategies for post-diagnosis management, characterized in that, Includes the following steps: Obtain the current time step The patient's original data was used, and a first patient state vector was generated by encoding it using a pre-trained language model. ; Based on the first patient state vector and the preset set of candidate actions By calculating each candidate action Dynamic strategy score The candidate action with the highest dynamic policy score is selected as the optimal follow-up action. .

2. The method for generating a dynamic follow-up strategy for post-diagnosis management according to claim 1, characterized in that: The dynamic strategy score The calculation formula is as follows: ,in, To enhance learning strategy networks The output is in state Next action Action value function, In order to be in the clinical guideline knowledge base Actions under constraints Compliance score, As a dynamic equilibrium factor; execute the optimal follow-up action. And receive patient feedback to obtain the next time step. reward signal Second patient state vector ; Using state transition quadruples For the reinforcement learning policy network Update.

3. The method for generating a dynamic follow-up strategy for post-diagnosis management according to claim 1, characterized in that, First patient state vector The generation steps include: acquiring the patient's electronic health record (EHR) data and historical dialogue data respectively; encoding the EHR data using a first Transformer encoder to obtain a first feature embedding; encoding the historical dialogue data using a second Transformer encoder to obtain a second feature embedding; and concatenating or weighting the first and second feature embeddings to generate the first patient state vector. .

4. The method for generating a dynamic follow-up strategy for post-diagnosis management according to claim 2, characterized in that, The dynamic balance factor The calculation method is as follows: ,in, Based on the first patient state vector The entropy value of the predicted future health status distribution is used to quantify state uncertainty; and These are preset learnable parameters.

5. The method for generating a dynamic follow-up strategy for post-diagnosis management according to claim 2, characterized in that, The compliance score The calculation steps include: [converting the clinical guideline knowledge base...] Constructed as a knowledge graph; based on the first patient state vector Retrieve the corresponding set of recommended operations from the knowledge graph; calculate the candidate actions. The cosine similarity between the action embedding and the embeddings of each operation in the recommended operation set is used, and the highest similarity value is used as the compliance score.

6. The method for generating a dynamic follow-up strategy for post-diagnosis management according to claim 2, characterized in that, The reward signal The reward is calculated by a multi-objective reward function, which includes a weighted sum of health improvement rewards based on changes in patients' physiological indicators, satisfaction rewards based on patient feedback and emotion analysis, and compliance rewards based on patients' adherence to medical orders.

7. The method for generating a dynamic follow-up strategy for post-diagnosis management according to claim 2, characterized in that, The reinforcement learning policy network The update employs the Deep Q-Network (DQN) algorithm, by minimizing the following loss function. To update network parameters : ,in, For mathematical expectation operators, For experience replay pool, As a discount factor, For the next action, These are the parameters of the target network.

8. The method for generating a dynamic follow-up strategy for post-diagnosis management according to claim 1, characterized in that, The optimal follow-up action is performed. The steps include: implementing the optimal follow-up action. The text is mapped to one or more natural language instructions; the natural language instructions are input into a pre-trained generative large language model to generate context-aware, personalized follow-up dialogue text; the follow-up dialogue text is sent to the patient via instant messaging, in-app notifications, or SMS.

9. The method for generating a dynamic follow-up strategy for post-diagnosis management according to claim 1, characterized in that, The candidate action set This includes at least the following types of actions: symptom inquiry, medication reminders, follow-up appointment scheduling, health education, lifestyle advice, and psychological counseling.

10. A method for generating dynamic follow-up strategies for post-diagnosis management according to any one of claims 1-8, characterized in that, The method is initiated after the patient completes their initial medical examination and is based on the optimal follow-up actions. The type and content of the follow-up visits are used to dynamically adjust the execution interval for the next follow-up visit.

Images