Scheme determination method and device based on reinforcement learning

Abstract

The invention relates to a scheme determination method and device based on reinforcement learning, and aims to solve the problem that an existing medical scheme providing system depends on a predefined rule and a static data model and cannot adapt to the dynamic change of a patient in real time. According to the method, historical medical data and real-time feedback data of a patient are acquired, a preset deep reinforcement learning model is input, a plurality of preliminary treatment schemes are generated, and multi-objective optimization processing is performed on the schemes, so that an optimal comprehensive treatment scheme is obtained. The multi-objective optimization takes into account a balance of multiple therapeutic objectives, such as maximizing toxin clearance, minimizing side effects, and maintaining blood balance. In a word, by introducing the deep reinforcement learning technology, the treatment scheme can be dynamically adjusted, personalized and real-time adaptive treatment suggestions are provided, and the accuracy of clinical decisions and the effectiveness of treatment are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and device for determining a solution based on reinforcement learning. Background Art

[0002] Among the current existing technologies, personalized treatment decision support systems have attempted to address the problem of individualized treatment needs to a certain extent. These systems use patients' electronic medical records (EHRs), physiological data, and medical literature, combined with certain rule engines or data mining technologies, to provide patients with personalized treatment recommendations. Such systems are usually based on fixed medical guidelines and combine patients' historical data to generate recommendations.

[0003] For example, IBM Watson for Oncology is a typical application example. It uses a large amount of medical literature and patient data to analyze the patient's condition and generate personalized treatment recommendations. The system can extract information from a large amount of medical research and clinical trial data and match it with the patient's specific situation to provide treatment plans. However, these systems often rely on predefined rules and static data models, and lack the ability to adapt to the dynamic changes of patients in real time. Summary of the invention

[0004] The main purpose of the present invention is to provide a method and device for determining a plan based on reinforcement learning to solve the technical problem that the medical plan providing system in the prior art relies on predefined rules and static data models and lacks the ability to adapt to the dynamic changes of patients in real time.

[0005] To achieve the above-mentioned objectives, the present invention provides a method for determining a plan based on reinforcement learning, comprising the following steps: obtaining the patient's historical medical data and real-time feedback data, wherein the historical medical data includes at least any of the following: electronic medical records and diagnostic records, and the real-time feedback data includes at least: physiological detection data: inputting the patient's historical medical data and real-time feedback data into a preset deep reinforcement learning model to generate multiple preliminary treatment plans, and performing multi-objective optimization processing on the multiple preliminary treatment plans to obtain the optimal comprehensive treatment plan.

[0006] The present invention also provides a reinforcement learning-based scheme determination device, comprising: an acquisition unit, used to acquire the patient's historical medical data and real-time feedback data, wherein the historical medical data includes at least any of the following: electronic medical records and diagnostic records, and the real-time feedback data includes at least: physiological detection data; a generation unit, used to input the patient's historical medical data and real-time feedback data into a preset deep reinforcement learning model, generate multiple preliminary treatment plans, and perform multi-objective optimization processing on the multiple preliminary treatment plans to obtain the optimal comprehensive treatment plan.

[0007] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0008] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods are implemented.

[0009] The reinforcement learning-based scheme determination method and device provided by the present invention can dynamically adapt to the patient's real-time health status changes by utilizing a deep reinforcement learning model, breaking through the limitations of the prior art that relies on predefined rules and static data models. Traditional medical scheme provision systems are often based on fixed medical guidelines and historical data, and it is difficult to flexibly respond to the dynamic changes of patients during treatment. The scheme of the present invention introduces reinforcement learning technology, combines the patient's historical medical data with real-time feedback data, generates multiple preliminary treatment plans, and performs multi-objective optimization processing on these plans, and finally obtains an optimal comprehensive plan that can balance various treatment goals. This optimization process can not only consider how to maximize the treatment effect, such as improving the toxin clearance rate, but also balance other key factors, such as minimizing side effects and maintaining blood balance.

[0010] In summary, the present invention overcomes the shortcomings of existing medical solution provision systems in terms of dynamic adaptability, provides a more flexible and efficient tool for clinical decision-making, and greatly improves the accuracy and adaptability of personalized treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a schematic diagram of the steps of a solution determination method based on reinforcement learning in one embodiment of the present invention; Figure 2 is a structural block diagram of a solution determination device based on reinforcement learning in one embodiment of the present invention; Figure 3 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0012] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0014] Reference Figure 1The embodiment of the present invention provides a solution determination method based on reinforcement learning, comprising the following steps: S1, obtaining the patient's historical medical data and real-time feedback data, wherein the historical medical data includes at least any one of the following: electronic medical records and diagnostic records, and the real-time feedback data includes at least: physiological detection data.

[0015] S2, inputting the patient's historical medical data and real-time feedback data into a preset deep reinforcement learning model, generating multiple preliminary treatment plans, and performing multi-objective optimization processing on the multiple preliminary treatment plans to obtain the optimal comprehensive treatment plan.

[0016] That is, by acquiring the patient's historical medical data and real-time feedback data, this technical solution can fully understand the patient's condition and physiological state. Historical medical data includes electronic medical records and diagnostic records, which provide the model with the patient's past health status and treatment history, helping the system understand the patient's basic health background and disease evolution. Real-time feedback data includes physiological test data, such as blood pressure, blood oxygen level, etc., which reflect the patient's current health status and immediate response to treatment in real time. By integrating these two data types, this technical solution can create a comprehensive and dynamic patient health profile.

[0017] Next, the patient's historical medical data and real-time feedback data are input into the preset deep reinforcement learning model to generate multiple preliminary treatment plans. The deep reinforcement learning model, with its unique learning mechanism, can learn how to make decisions in a complex and dynamic environment through continuous trial and error and optimization. The model can not only process a large amount of historical data, but also analyze feedback data in real time to generate highly adaptable preliminary treatment plans. These plans are not simply generated based on fixed rules, but are continuously explored and adjusted by the intelligent agent in the simulated environment, reflecting the deep understanding of the patient's condition and the ability to dynamically adjust.

[0018] Furthermore, these preliminary treatment plans will undergo multi-objective optimization to obtain the best comprehensive treatment plan. During the multi-objective optimization process, the system will consider multiple treatment goals at the same time, such as maximizing toxin clearance, minimizing side effects, and maintaining blood balance. This process requires quantifying the effect of each goal and finding the best comprehensive plan through the optimization algorithm. For example, if the toxin clearance rate is very high but the side effects are large in some plans, the system will adjust these plans to balance the relationship between toxin clearance and side effects, so as to find a treatment plan with the best comprehensive effect. This optimization process ensures that the treatment is not only optimized for a single goal, but also comprehensively considers the trade-offs between multiple goals to provide a more comprehensive treatment strategy.

[0019] This reinforcement learning-based regimen determination method significantly improves the accuracy and flexibility of personalized treatment. Unlike traditional systems that rely on static rules, the method of the present invention can adapt to changes in patients in real time, and provide more accurate treatment recommendations through the dynamic adjustment function of the deep reinforcement learning model. For example, during the treatment process, if the patient's physiological state suddenly changes, the traditional system may not be able to adjust the treatment plan in time, while the method of the present invention can re-optimize the treatment plan based on real-time data, quickly respond to changes in patients, and ensure treatment effectiveness and safety. In addition, through multi-objective optimization, this method can comprehensively balance multiple treatment goals, avoid the side effects that may be caused by previous single-target optimization, and improve the overall treatment effect.

[0020] In summary, the technical solution of the present invention effectively solves the problems of insufficient real-time adaptability and static rule dependence in the prior art through the combination of deep reinforcement learning and multi-objective optimization, and provides a dynamic and flexible solution for personalized treatment. This method not only improves the comprehensive performance of the treatment plan, but also enhances the system's ability to adapt to the dynamic changes of patients, thereby providing more accurate and effective treatment support in practical applications.

[0021] In one example, after obtaining the optimal comprehensive treatment plan, the plan determination method also includes: adjusting the treatment equipment used for the patient based on the optimal comprehensive treatment plan, wherein the treatment equipment includes at least: a blood purification system; and cyclically executing the steps of "obtaining real-time feedback data from the patient, inputting the real-time feedback data from the patient into a preset deep reinforcement learning model, generating multiple preliminary treatment plans, and performing multi-objective optimization processing on the multiple preliminary treatment plans to obtain the optimal comprehensive treatment plan, and adjusting the treatment equipment used for the patient based on the obtained optimal comprehensive treatment plan."

[0022] That is, after obtaining the optimal comprehensive treatment plan, the system will adjust the treatment equipment used by the patient according to the plan. In this example, the treatment equipment mainly includes a blood purification system. This adjustment is based on the specific treatment parameters in the optimal plan, such as blood flow rate, ultrafiltration rate, dialysate composition, etc., to ensure that the settings of the treatment equipment can fully meet the requirements of the optimal plan. For example, if the optimal plan recommends increasing the blood flow rate to improve the toxin removal efficiency, the system will adjust the relevant parameters in the blood purification equipment to achieve this goal.

[0023] Next, the system enters a cycle to continuously optimize the treatment plan. This cycle includes the following steps: 1. Obtain real-time feedback data from patients: During the treatment process, the system continuously monitors the patient's physiological state and collects real-time data through physiological detection equipment. These data include blood pressure, blood oxygen level, blood flow, etc., which can reflect the patient's immediate response to treatment.

[0024] 2. Input real-time feedback data into the deep reinforcement learning model: The collected real-time feedback data is input into the preset deep reinforcement learning model. This model uses real-time data to update its understanding of the patient's condition and generates multiple new preliminary treatment plans through the learning process. These plans are based on the current real-time feedback to adapt to the patient's immediate needs.

[0025] 3. Generate multiple preliminary treatment plans and perform multi-objective optimization: The deep reinforcement learning model generates multiple preliminary treatment plans and performs multi-objective optimization on these plans. Multi-objective optimization aims to balance multiple treatment goals, such as improving toxin clearance, reducing side effects, and maintaining blood balance. Through the optimization algorithm, the system can find a treatment plan with the best overall effect.

[0026] 4. Adjust the treatment equipment based on the optimal comprehensive treatment plan: According to the optimized optimal comprehensive treatment plan, the system will adjust the settings of the treatment equipment again. These adjustments ensure that the operating parameters of the equipment are consistent with the latest treatment plan to achieve the best treatment effect.

[0027] The core of this cycle is its dynamic adaptability. By continuously acquiring real-time data and updating the treatment plan, the system can flexibly respond to changes in the patient's physiological state and ensure continuous optimization of treatment. This dynamic adjustment mechanism enables the treatment process to be continuously improved according to the patient's actual response, thereby providing more personalized and effective treatment.

[0028] For example, in a practical application, if a patient experiences blood pressure fluctuations during blood purification treatment, the system will update the treatment plan based on these real-time data, and may adjust the blood flow rate or ultrafiltration rate to stabilize blood pressure and improve treatment effect. The adjusted plan will reconfigure the device to meet the new treatment requirements and continue to optimize after the next data acquisition.

[0029] The advantage of this cyclic optimization process is that it can quickly respond to the patient's dynamic changes, avoiding the delays and discomfort that may occur in traditional treatment methods, thereby improving the overall treatment effect and patient comfort. This method can not only adjust the device settings in real time, but also continuously improve the accuracy and effectiveness of the treatment plan.

[0030] In one example, the multiple preliminary treatment plans are subjected to multi-objective optimization processing to obtain an optimal comprehensive treatment plan, including: defining multiple optimization goals, converting the optimization goals into objective functions, and constructing an optimization algorithm framework based on the objective functions of the multiple optimization goals to obtain a multi-objective optimization algorithm, wherein the multiple optimization goals include at least: maximizing toxin clearance rate, minimizing side effects, and maintaining blood balance; using the multi-objective optimization algorithm to iteratively calculate the multiple preliminary treatment plans to obtain the value of each objective function under each of the preliminary treatment plans, and based on the value of each objective function under each of the preliminary treatment plans, outputting a set of preliminary treatment plans that balance multiple optimization goals as the comprehensive treatment plan.

[0031] In this example, first, several key goals that need to be balanced in the optimization of the treatment plan are identified. These goals usually include: maximizing toxin clearance, minimizing side effects, and maintaining blood balance. Each goal represents a different aspect of the treatment, such as toxin clearance represents a direct indicator of the effectiveness of the treatment, while side effects and blood balance are related to the patient's comfort and overall health. In order to use these optimization goals in calculations, they need to be converted into mathematical objective functions. The objective function is a mathematical expression that quantifies the degree to which each goal is achieved. For example, toxin clearance can be defined as a function that calculates the amount of toxin removed per unit time based on different settings of the treatment plan; side effects can be modeled by their severity; and blood balance can be quantified by maintaining the normal chemical composition of the blood.

[0032] At this point, based on the defined objective function, an optimization algorithm framework is constructed. This framework needs to comprehensively consider all objective functions and determine how to balance these objectives. The core task of the optimization algorithm framework is to solve how to find an overall optimal solution while ensuring that each objective is optimized as much as possible. This usually involves selecting appropriate algorithms (such as genetic algorithms, particle swarm optimization, etc.) to deal with multi-objective optimization problems. Then, through the constructed algorithm framework, a specific multi-objective optimization algorithm is obtained. This algorithm can handle multiple objective functions and balance between these functions. For example, the algorithm can find a solution that optimizes all objective functions as much as possible through iterative calculations and continuous adjustment of the parameters of the treatment plan.

[0033] During the optimization process, the above-mentioned multi-objective optimization algorithm is used to iteratively calculate the preliminary treatment plan. Each preliminary treatment plan will be input into the algorithm to calculate the value of each objective function. For example, the algorithm will calculate the toxin clearance rate, the side effect score, the blood balance, etc. under a certain treatment plan. Finally, based on the calculated value of each objective function, a preliminary treatment plan that comprehensively balances multiple optimization goals is output. This plan combines the requirements of all optimization goals to ensure that the toxin clearance rate is maximized, while minimizing side effects and maintaining blood balance. Ultimately, the system will output a set of treatment plans that achieve a balance between the various goals, so that they can be provided as the optimal comprehensive treatment plan for clinical application.

[0034] For example, when optimizing a blood purification program, the algorithm may find that increasing blood flow rate can significantly increase toxin clearance, but it may also increase side effects. Through multi-objective optimization, the system will find an optimal balance between increasing toxin clearance and reducing side effects, and maintain blood balance. The final treatment plan is the best comprehensive effect achieved between these goals.

[0035] The core of this optimization treatment method is that it can comprehensively consider multiple treatment goals and find an overall optimal treatment plan through iterative calculation, avoiding the unbalanced effects that may be caused by focusing on a single goal, and improving the comprehensiveness of the treatment and the patient's overall health level.

[0036] In one example, after the patient's real-time feedback data is input into the deep reinforcement learning model, the scheme determination method further includes: detecting whether there is any abnormality in the real-time feedback data through the preset deep reinforcement learning model, and if there is any abnormality, sending an alarm prompt.

[0037] When patients' real-time feedback data, such as blood pressure, blood oxygen level, blood flow rate, etc., are input into the preset deep reinforcement learning (DRL) model, the model begins to conduct in-depth analysis of the data. The deep reinforcement learning model is trained on a large amount of historical data and can identify complex patterns and relationships, thereby accurately analyzing real-time data. At this stage, the model not only focuses on the immediate value of the data, but also comprehensively considers the trends and historical changes of the data to establish an accurate understanding of normal and abnormal situations.

[0038] Next, the DRL model processes the input data to detect whether there are any anomalies. Anomalies are usually defined as data points that deviate from the normal range or show significant fluctuations that are inconsistent with historical data. For example, if a patient experiences a sharp drop in blood pressure during dialysis, or blood oxygen levels suddenly drop to dangerous levels, these situations may be identified as anomalies by the model. Deep reinforcement learning models judge these deviations through set thresholds or anomaly detection algorithms. Thresholds can be standards calculated based on historical data and normal patterns, and the model will mark any abnormal data that exceeds these standards.

[0039] When the model detects an abnormal situation, the system automatically generates and sends an alarm prompt. These alarm prompts can be conveyed to medical staff in a variety of ways, including visual alarms, sound prompts, SMS notifications, or system pop-ups. The core purpose of the alarm prompt is to quickly convey abnormal information to the medical team so that they can take timely measures. For example, during dialysis, the system detects that the patient's blood pressure suddenly drops below the preset safety threshold. The model immediately generates an alarm and notifies the medical staff. After receiving the alarm, the doctor or caregiver can respond quickly, such as adjusting the settings of the dialysis machine, or giving the patient additional medical intervention to avoid further health risks.

[0040] The advantage of this process is that it can monitor the patient's physiological state in real time and respond immediately when abnormal conditions occur, greatly improving the safety of treatment. For example, if the system detects that the patient's blood oxygen level is continuously below the normal range, the alarm prompt can remind medical staff to check the patient's respiratory status or adjust the composition of the dialysis fluid to prevent possible respiratory failure or other complications. Compared with the limitations of traditional methods that rely on manual monitoring, this automated abnormality detection and alarm mechanism can reduce response time and improve the timeliness and accuracy of intervention.

[0041] By applying deep reinforcement learning models to anomaly detection in real-time data, the embodiments of the present invention demonstrate an intelligent solution that can provide efficient monitoring and emergency response in a dynamically changing medical environment. This approach not only improves the safety of the treatment process, but also enhances the medical system's ability to respond to emergencies, thereby better protecting the health of patients.

[0042] In one example, the steps of "obtaining real-time feedback data from a patient, inputting the real-time feedback data from the patient into a preset deep reinforcement learning model, generating multiple preliminary treatment plans, performing multi-objective optimization processing on the multiple preliminary treatment plans to obtain an optimal comprehensive treatment plan, and adjusting the treatment equipment used by the patient based on the obtained optimal comprehensive treatment plan" are cyclically executed, including: step 1, obtaining real-time feedback data from the patient, and inputting the real-time feedback data from the patient into a preset deep reinforcement learning model to generate multiple preliminary treatment plans; step 2, performing multi-objective optimization processing on the multiple preliminary treatment plans to obtain an optimal comprehensive treatment plan, and adjusting the treatment equipment used by the patient based on the obtained optimal comprehensive treatment plan; cyclically executing steps 1 and 2, and associating the comprehensive treatment plan with the real-time feedback data detected by the patient after adjusting the treatment equipment based on the comprehensive treatment plan to form a sample data; after obtaining multiple sample data, selecting N sample data closest to the current moment and inputting them into the preset deep reinforcement learning model, and retraining the preset deep reinforcement learning model based on the N sample data to improve the performance of the deep reinforcement learning model.

[0043] That is, the steps performed in this loop aim to improve the personalization and adaptability of treatment and ultimately improve the patient's treatment effect by dynamically adjusting and optimizing the treatment plan. Specifically, the method includes obtaining real-time feedback data from patients and inputting this data into a deep reinforcement learning model to generate preliminary treatment plans. These plans are then processed through multi-objective optimization to obtain the optimal comprehensive treatment plan, and the treatment equipment is adjusted based on this plan. This process is repeated continuously to form a closed-loop system that can continuously improve and optimize the treatment effect.

[0044] First, in step 1, the system continuously obtains real-time feedback data from patients, including key physiological indicators such as blood pressure, blood oxygen level, and blood flow rate. These data are input into the preset deep reinforcement learning model. The deep reinforcement learning model is trained with a large amount of data and can generate multiple preliminary treatment plans based on real-time feedback data. These plans take into account the patient's current physiological state and historical treatment data, and try to find the most suitable treatment method. In step 2, the system performs multi-objective optimization on the generated multiple preliminary treatment plans. Multi-objective optimization aims to balance and optimize multiple treatment goals, such as maximizing toxin clearance, minimizing side effects, and maintaining blood balance. Through the optimization algorithm, the system can find an optimal comprehensive treatment plan that provides the best comprehensive effect while considering all goals. Based on this optimal comprehensive treatment plan, the system will adjust the patient's treatment equipment, such as the settings of the blood purification system, such as blood flow rate and dialysate composition, to ensure the optimal and safe treatment process.

[0045] The loop execution of steps 1 and 2 means that the system continuously collects data, generates solutions, and optimizes the treatment throughout the treatment process. This iterative process helps to respond to changes in the patient's physiological state in a timely manner and continuously optimize the treatment plan. For example, if there are new changes or abnormalities in the patient's physiological data, the system will automatically adjust the treatment plan to adapt to the new situation, thereby improving the accuracy of the treatment. After each cycle, the system associates the generated comprehensive treatment plan with the patient's real-time feedback data after adjusting the treatment equipment to form a sample data. These sample data record the specific results of each treatment optimization and the patient's response. Over time, the system accumulates a large amount of sample data, which provides detailed information about the patient's treatment effect and equipment adjustment.

[0046] After obtaining multiple sample data, the system will select the N sample data closest to the current moment. Selecting the most recent sample data helps ensure that the model can reflect the latest patient conditions and treatment effects. These sample data are input into the deep reinforcement learning model, and the system will retrain the model. The retraining process improves the performance and accuracy of the model by adjusting and optimizing the model parameters using the latest sample data. In this way, the deep reinforcement learning model can continuously learn and adapt to the dynamic changes of patients, improve the generation and optimization of treatment plans, and further improve the treatment effect and the health management level of patients.

[0047] In summary, the cyclic execution steps and retraining mechanism of the present invention form a closed-loop feedback system that can continuously optimize the treatment plan and improve the adaptability and accuracy of the system. This dynamic adjustment and optimization method not only improves the personalization of treatment, but also significantly enhances the ability to respond to real-time changes in patients, making the treatment process more efficient and safer.

[0048] In one example, after obtaining the patient's historical medical data, the plan determination method also includes: inputting the patient's historical medical data into a preset large language model; integrating and analyzing the patient's historical medical data through the large language model to obtain the patient's medical summary report, and using the patient's medical summary report to replace the patient's historical medical data and inputting it into a preset deep reinforcement learning model.

[0049] In this example, integrating a large language model to process the patient's historical medical data is a key technology that enhances the data quality and relevance of the deep reinforcement learning model when generating treatment plans. Specifically, this process is divided into several key steps, each of which ensures the validity of the data and the accuracy of the treatment plan.

[0050] First, obtaining the patient's historical medical data is the starting point of the whole process. This data usually includes electronic medical records, diagnostic records, medical history information, etc., covering the patient's past health records and treatment experiences. These historical data provide basic information for understanding the patient's long-term health status and treatment response, but simply using these raw data may lead to difficulties in information processing and analysis, especially when the data volume is large and the structure is complex. Next, inputting the patient's historical medical data into the preset large language model is a key step. Large language models, such as the GPT series models, have powerful natural language processing capabilities and can understand and generate complex language information. By inputting the patient's historical medical data into the large language model, the model will deeply analyze and integrate the data. Specifically, the large language model can identify and extract key information and integrate this information into a comprehensive medical summary report. This summary report not only summarizes the patient's historical health status, but also highlights important medical information such as major diseases, treatment history, known allergic reactions, medication use, etc.

[0051] The purpose of this step is to convert the raw data into a more structured and easy-to-interpret medical summary report. Such a report can present the patient's health information more clearly, facilitating further analysis and decision-making. At the same time, it also helps remove noise and redundant information in historical data, making subsequent data processing more efficient and accurate.

[0052] Finally, the generated medical summary report replaces the original historical medical data and is re-input into the deep reinforcement learning model. This operation is intended to use the refined data processed by the large language model to guide the deep reinforcement learning model. The role of the deep reinforcement learning model is to generate and optimize treatment plans based on the patient's historical data and real-time feedback. Using medical summary reports instead of raw data can improve the quality of model input data, making the deep reinforcement learning model more accurate and relevant when generating treatment plans. This processing method helps the deep reinforcement learning model better understand the patient's health background, so as to develop personalized treatment plans that better meet the patient's needs.

[0053] Through this method, the entire system not only improves the efficiency of data processing, but also enhances the accuracy of treatment plans. The integrated analysis capabilities of the large language model ensure the quality and relevance of the input data, while the deep reinforcement learning model uses this high-quality data to optimize the treatment plan. This multi-level data processing and integration mechanism significantly improves the system's performance in complex medical decision-making, making the formulation of treatment plans more scientific and efficient.

[0054] like Figure 2 As shown, the embodiment of the present invention further provides a solution determination device based on reinforcement learning, comprising: The acquisition unit 1 is used to acquire the patient's historical medical data and real-time feedback data, wherein the historical medical data includes at least any one of the following: electronic medical records and diagnosis records, and the real-time feedback data includes at least: physiological detection data.

[0055] Generation unit 2 is used to input the patient's historical medical data and real-time feedback data into a preset deep reinforcement learning model, generate multiple preliminary treatment plans, and perform multi-objective optimization processing on the multiple preliminary treatment plans to obtain the optimal comprehensive treatment plan.

[0056] In this embodiment, for the specific implementation of each unit in the above device embodiment, please refer to the above method embodiment, which will not be repeated here.

[0057] Reference Figure 3 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0058] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0059] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0060] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0061] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0062] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A solution determination method based on reinforcement learning, characterized in that: The following steps are involved: Obtain the patient's historical medical data and real-time feedback data, wherein the historical medical data includes at least any of the following: electronic medical records and diagnostic records, and the real-time feedback data includes at least: physiological detection data: The patient's historical medical data and real-time feedback data are input into a preset deep reinforcement learning model to generate multiple preliminary treatment plans, and the multiple preliminary treatment plans are subjected to multi-objective optimization processing to obtain the optimal comprehensive treatment plan.

2. The method for determining a solution according to claim 1, characterized in that: After obtaining the optimal comprehensive treatment plan, the plan determination method further includes: Based on the optimal comprehensive treatment plan, adjusting the treatment equipment used by the patient, wherein the treatment equipment at least includes: a blood purification system; And the steps of "obtaining real-time feedback data from the patient, inputting the real-time feedback data from the patient into a preset deep reinforcement learning model, generating multiple preliminary treatment plans, performing multi-objective optimization processing on the multiple preliminary treatment plans to obtain an optimal comprehensive treatment plan, and adjusting the treatment equipment used for the patient based on the obtained optimal comprehensive treatment plan" are executed cyclically.

3. The method for determining a solution according to claim 1, characterized in that: The multiple preliminary treatment plans are subjected to multi-objective optimization processing to obtain the best comprehensive treatment plan, including: Defining multiple optimization objectives, converting the optimization objectives into objective functions, and constructing an optimization algorithm framework based on the objective functions of the multiple optimization objectives to obtain a multi-objective optimization algorithm, wherein the multiple optimization objectives at least include: maximizing toxin clearance rate, minimizing side effects, and maintaining blood balance; The multi-objective optimization algorithm is used to iteratively calculate the multiple preliminary treatment plans to obtain the value of each objective function under each of the preliminary treatment plans. Based on the value of each objective function under each of the preliminary treatment plans, a set of preliminary treatment plans that balance multiple optimization objectives are output as the comprehensive treatment plan.

4. The method for determining a solution according to claim 1, characterized in that: After inputting the patient's real-time feedback data into the deep reinforcement learning model, the solution determination method further includes: The preset deep reinforcement learning model is used to detect whether there is any abnormality in the real-time feedback data. If there is any abnormality, an alarm prompt is sent.

5. The method for determining a solution according to claim 1, characterized in that: The step of cyclically executing "obtaining real-time feedback data from a patient, inputting the real-time feedback data from the patient into a preset deep reinforcement learning model, generating a plurality of preliminary treatment plans, performing multi-objective optimization processing on the plurality of preliminary treatment plans to obtain an optimal comprehensive treatment plan, and adjusting the treatment device used for the patient based on the obtained optimal comprehensive treatment plan" includes: Step 1, obtaining the real-time feedback data of the patient, and inputting the real-time feedback data of the patient into a preset deep reinforcement learning model to generate multiple preliminary treatment plans; Step 2, performing multi-objective optimization processing on the multiple preliminary treatment plans to obtain an optimal comprehensive treatment plan, and adjusting the treatment equipment used for the patient based on the obtained optimal comprehensive treatment plan; Execute step 1 and step 2 in a loop, and associate the comprehensive treatment plan with the real-time feedback data detected by the patient after adjusting the treatment device based on the comprehensive treatment plan to form a sample data; After obtaining multiple sample data, select N sample data closest to the current moment and input them into the preset deep reinforcement learning model, and retrain the preset deep reinforcement learning model based on the N sample data to improve the performance of the deep reinforcement learning model.

6. The method for determining a solution according to claim 1, characterized in that: After obtaining the patient's historical medical data, the solution determination method further includes: Inputting the patient's historical medical data into a preset large language model; The patient's historical medical data is integrated and analyzed through the large language model to obtain the patient's medical summary report, and the patient's medical summary report is used to replace the patient's historical medical data and input into a preset deep reinforcement learning model.

7. A solution determination device based on reinforcement learning, characterized in that: include: The acquisition unit is used to acquire the patient's historical medical data and real-time feedback data, wherein the historical medical data includes at least any of the following: electronic medical records and diagnostic records, and the real-time feedback data includes at least: physiological detection data: A generation unit is used to input the patient's historical medical data and real-time feedback data into a preset deep reinforcement learning model, generate multiple preliminary treatment plans, and perform multi-objective optimization processing on the multiple preliminary treatment plans to obtain the optimal comprehensive treatment plan.

8. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images