Intelligent Management System and Method for Anticoagulation Regulation in Extracorporeal Circulation

Abstract

This invention belongs to the field of healthcare informatics and provides an intelligent management system and method for anticoagulation regulation in extracorporeal circulation. The system includes: constructing a feature vector characterizing a patient's current anticoagulation needs; calculating the expected cumulative reward for each strategy agent by exploring the long-term effects of different heparin infusion adjustment strategies on the patient's individual state; dynamically modeling the short-term relationship between different heparin infusion adjustment strategies and the patient's real-time coagulation activation time deviation from the target value and the safe range of heparin concentration using a probability density function, generating a specific heparin infusion rate adjustment amount; receiving the heparin infusion rate adjustment amount and updating the heparin infusion rate in real time accordingly; and feeding back the patient's real-time coagulation activation time deviation from the target value and the current heparin concentration to the global optimization module and the local optimization module to continuously correct the strategy parameters.

Description

Technical Field

[0001] This invention belongs to the field of healthcare informatics, and specifically relates to an intelligent management system and method for anticoagulation regulation of extracorporeal circulation. Background Technology

[0002] During cardiopulmonary bypass (CPB), heparin is widely used as the primary anticoagulant to maintain the patency and stability of blood flow in the CPB tubing, preventing circulatory disturbances caused by blood clotting within the tubing. This process requires extremely strict control of anticoagulation intensity. It is essential to ensure that the activation time (ACT) remains within a predetermined safe range while avoiding complications such as bleeding due to heparin overdose or tubing blockage due to insufficient anticoagulation. Because each patient exhibits significant differences in heparin pharmacokinetics, physiological state, and surgical environment, anticoagulation regimens often need to be dynamically adjusted based on individual circumstances.

[0003] Current anticoagulation control methods mainly rely on experience-driven manual decision-making or rule-based control algorithms. On the one hand, clinicians typically adjust the heparin infusion rate manually based on intermittently monitored ACT values, heparin concentrations, and past medication experience; on the other hand, some systems introduce PID controllers or fuzzy control logic to automatically adjust parameters according to the degree of deviation from the target value. These methods achieve basic closed-loop control of the anticoagulation process to a certain extent, but their core still relies on preset rules and static feedback, making it difficult to adapt to complex and variable physiological changes.

[0004] However, existing technologies generally suffer from the following problems: First, control systems struggle to model and predict the cumulative effects of different anticoagulation regimens over time, leading to delayed or unstable strategy responses. Second, they lack effective adaptation mechanisms to system state uncertainties and individual patient differences, making them prone to regulatory failure during sudden physiological changes. Third, control strategies lack self-learning and continuous optimization capabilities, failing to dynamically adjust model parameters based on clinical feedback. Therefore, there is an urgent need to construct an intelligent anticoagulation regulation system with long-term optimization capabilities, short-term adaptive capabilities, and a real-time feedback learning mechanism to achieve precise management and intervention of anticoagulation risks during extracorporeal circulation. Summary of the Invention

[0005] To address the problems in the prior art, this invention provides an intelligent management system for anticoagulation regulation in extracorporeal circulation, comprising:

[0006] The patient status acquisition module is used to collect and update individual patient status information related to anticoagulation therapy in real time, and construct a feature vector to characterize the patient's current anticoagulation needs;

[0007] The global optimization module is used to construct multiple parallel strategy agents. By exploring the long-term effects of different heparin infusion adjustment strategies on the individual patient status, the expected cumulative reward of each strategy agent is calculated, and the global anticoagulation therapy strategy is updated accordingly.

[0008] The local optimization module is used to dynamically model the short-term relationship between different heparin infusion adjustment strategies and the deviation of the patient's real-time coagulation activation time from the target value and the safe range of heparin concentration through a probability density function. Based on the principle of maximizing strategy entropy, the global anticoagulation treatment strategy is corrected in real time, and specific heparin infusion rate adjustment amounts are generated.

[0009] The heparin precision infusion module is used to receive the heparin infusion rate adjustment amount and update the heparin infusion rate in real time accordingly.

[0010] The feedback update module is used to feed back the patient's real-time coagulation activation time deviation from the target value and the current heparin concentration to the global optimization module and the local optimization module, and continuously correct the strategy parameters of the global optimization module and the local optimization module.

[0011] Furthermore, the action input generated by the strategy agent is fed into a simulated interactive environment that closely resembles the patient's physiological state.

[0012] Furthermore, the simulated interactive environment receives the current state and the agent output action, generates the next state by executing the state transition function, and outputs a reward signal related to the state. This reward signal is used to measure the positive or negative impact of the heparin adjustment action on the anticoagulation effect.

[0013] Furthermore, after each policy agent completes its policy optimization, it uploads the updated policy network weight parameters or gradient change vectors to the central policy updater.

[0014] The updater is equipped with an aggregation processing module, which receives parameter inputs from various agents and performs parameter fusion operations according to preset rules.

[0015] The new round of global policy network weights obtained after aggregation will be distributed to each policy agent for the next round of policy iteration training.

[0016] Furthermore, in the construction of multiple parallel policy agents, each policy agent has the same network structure, parameter size, and input / output dimensions, but the training process is completely independent, with different initial parameters, random seeds, policy update trajectories, and interaction paths.

[0017] Furthermore, the multiple parallel strategy proxies are deployed on multiple computing nodes in a distributed manner.

[0018] Furthermore, the policy agent is constructed using an asynchronous advantage actor-critic structure. Each policy agent independently contains a policy network and a value function network, which are used to generate control outputs and evaluate state reward potential, respectively.

[0019] The policy network takes the current state vector as input and outputs the heparin infusion rate adjustment value.

[0020] The value function network takes the same state information as input and outputs an estimate of the reward in that state;

[0021] The agent interacts with the simulation environment based on the action output by the current policy. After one round of training, it constructs an advantage value based on the difference between the actual reward and the value function output, and optimizes the policy parameters based on this value.

[0022] Furthermore, the probability density function is:

[0023]

[0024] in, Indicates in the strategy parameters Under these conditions, the system is in state Take action at the time The probability density function;

[0025] Indicates the system in state Adjustment amount of heparin generated below;

[0026] Represents a value with mean as variance is It follows a normal distribution.

[0027] Furthermore, before the strategy is executed, the system starts from the distributed... Multiple candidate actions are generated through sampling, and each candidate action is input into an online physiological response prediction model.

[0028] The prediction model is a lightweight state transition neural network, trained based on historical patient data. It has a feedforward network structure, receives input, and outputs the estimated state value for the next time step.

[0029] The present invention also provides an intelligent management method for anticoagulation regulation of extracorporeal circulation, which uses the aforementioned intelligent management system for anticoagulation regulation of extracorporeal circulation to perform anticoagulation regulation of extracorporeal circulation.

[0030] The intelligent management system for anticoagulation regulation proposed in this invention can dynamically generate forward-looking and adaptive anticoagulation strategies based on the multimodal physiological state information of individual patients. The system introduces a two-layer decision-making structure of global optimization and local correction, enabling the control of heparin infusion rate to not only consider long-term therapeutic effects and short-term response performance, but also automatically adapt to nonlinear fluctuations in the patient's condition, significantly improving the stability and safety of anticoagulation control.

[0031] By introducing reinforcement learning mechanisms and a simulated environment interaction model, the system enables its control strategy to continuously evolve. During multiple surgeries or long-term operation, it can continuously optimize strategy parameters based on feedback, effectively overcoming the limitations of traditional experience-based methods in adapting to new patients or complex cases. Furthermore, through a real-time feedback mechanism and risk-reward function design, the system can precisely control the anticoagulation effect while suppressing unnecessary heparin dosage, thereby reducing the risk of bleeding while ensuring unobstructed blood flow, and improving patient prognosis and overall treatment safety. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a functional structure diagram of the system of the present invention. Detailed Implementation

[0034] The invention will now be described in preferred form with reference to the accompanying drawings and specific embodiments.

[0035] This embodiment solves the above problems through the following steps:

[0036] like Figure 1 As shown in one embodiment, the present invention discloses an intelligent management system for anticoagulation regulation for extracorporeal circulation, belonging to the field of healthcare informatics. This system is used to adaptively optimize the infusion strategy of anticoagulants based on the real-time coagulation status and dynamic changes of individual patients during extracorporeal circulation, so as to precisely adjust the heparin infusion rate, thereby achieving dynamic balance control of coagulation function, avoiding insufficient or excessive anticoagulation, ensuring the safety of patients' blood circulation during extracorporeal circulation surgery, and improving clinical treatment efficacy and safety.

[0037] To achieve the above objectives, the system includes the following modules:

[0038] The patient status acquisition module is used to collect and update individual patient status information related to anticoagulation therapy in real time, and construct a feature vector to characterize the patient's current anticoagulation needs;

[0039] The global optimization module is used to construct multiple parallel strategy agents. By exploring the long-term effects of different heparin infusion adjustment strategies on the individual patient status, the expected cumulative reward of each strategy agent is calculated, and the global anticoagulation therapy strategy is updated accordingly.

[0040] The local optimization module is used to dynamically model the short-term relationship between different heparin infusion adjustment strategies and the deviation of the patient's real-time coagulation activation time from the target value and the safe range of heparin concentration through a probability density function. Based on the principle of maximizing strategy entropy, the global anticoagulation treatment strategy is corrected in real time, and specific heparin infusion rate adjustment amounts are generated.

[0041] The heparin precision infusion module is used to receive the heparin infusion rate adjustment amount and update the heparin infusion rate in real time accordingly.

[0042] The feedback update module is used to feed back the patient's real-time coagulation activation time deviation from the target value and the current heparin concentration to the global optimization module and the local optimization module, and continuously correct the strategy parameters of the global optimization module and the local optimization module.

[0043] The system can monitor the dynamic changes in an individual patient's coagulation function in real time and precisely optimize the heparin infusion rate based on reinforcement learning technology. This allows the anticoagulation strategy to dynamically adapt to the physiological changes of different patients during cardiopulmonary bypass, significantly reducing the risk of thrombosis caused by insufficient anticoagulation and the risk of bleeding caused by excessive anticoagulation, thus effectively improving the safety and stability of cardiopulmonary bypass surgery.

[0044] The aforementioned system further realizes the synergistic effect of long-term optimization and short-term fine-tuning of anticoagulation therapy strategies, effectively reducing the subjectivity and lag issues existing in traditional manual experience-based anticoagulation control methods, making heparin dosage more precise and rational, thereby reducing medical costs while ensuring medical safety and improving the treatment effect and prognosis of patients.

[0045] To more clearly illustrate the technical solution and implementation of the present invention, the following will describe in detail the various components and their cooperative relationships in conjunction with specific embodiments.

[0046] First, the patient status acquisition module in this invention will be described.

[0047] The patient status acquisition module is a fundamental component for realizing intelligent management of anticoagulation regulation in this embodiment of the invention. Its function is to sense and record multi-source physiological information closely related to the anticoagulation strategy during the patient's extracorporeal circulation in real time, and to structure this information into feature vectors that can be processed by the reinforcement learning module, so as to ensure that the subsequent strategy generation process has sufficient and reliable status basis.

[0048] Specifically, the parameters collected by the patient status acquisition module include, but are not limited to, the following categories:

[0049] (1) Static individual characteristic parameters: including patient weight (in kg), used to convert heparin dose units (IU / kg / min) and serve as the basic variable for the heparin pharmacokinetic model; at the same time, preoperative basic coagulation function indicators, such as thromboelastography (TEG) clotting time (R value), clot formation time (K value), clot formation rate (α angle), maximum amplitude (MA value), etc., can be collected to characterize the patient's prior coagulation status and anticoagulation sensitivity;

[0050] (2) Intraoperative real-time dynamic indicators: including but not limited to real-time activated clotting time (ACT), body temperature (°C), plasma heparin concentration (IU / mL), anti-Xa factor activity, blood flow velocity, circuit pressure, etc. Among them, ACT is the main anticoagulation effect feedback indicator, which can be updated every 5 to 10 minutes; body temperature is a key parameter affecting the activity of coagulation factors, and it is usually negatively correlated with the patient's prothrombin time; anti-Xa activity can reflect the actual effect of heparin.

[0051] (3) Operating status parameters: such as the extracorporeal circulation pump speed, perfusion fluid temperature, pump suction pressure and other system operating parameters, can be selected as auxiliary status inputs to explain non-physiological interference factors of ACT or heparin concentration changes.

[0052] To achieve unified acquisition and processing of the aforementioned multi-source data, the patient status acquisition module preferably integrates a multi-channel data acquisition unit, supporting interface with clinical equipment through the following methods:

[0053] It connects to the data interface (such as RS-232 or CAN bus) of the extracorporeal circulation machine to extract the equipment's operating parameters and real-time flow information;

[0054] Connect to intelligent coagulation testing devices (such as TEG analyzers and ACT analyzers) to obtain real-time coagulation parameters;

[0055] It integrates with temperature sensors, pressure sensors, and biochemical analyzers to achieve continuous sensing of patient body temperature, circuit pressure, and heparin concentration.

[0056] The raw data collected above, after format conversion, missing data completion, numerical normalization, and unit unification, is organized into a time-synchronized state vector.

[0057] After obtaining the individual patient status characteristics, the system enters the anticoagulation strategy generation stage, the first stage of which is the operation of the global optimization module.

[0058] During cardiopulmonary bypass, a patient's coagulation status is influenced by a combination of physiological variables, and these changes typically exhibit time lag and path dependence. Short-term adjustments to anticoagulation therapy often fail to represent its long-term effects. Without systematic modeling of the cumulative effects of different anticoagulation regimens over time, issues such as poor strategy stability, delayed response, and intervention failure can easily arise. To improve the sustainability, stability, and predictability of anticoagulation strategies throughout the entire control process, a mechanism is needed to evaluate the effectiveness of heparin infusion and optimize strategy selection over a longer timescale. Therefore, a global optimization module is constructed to build multiple parallel strategy agents. By exploring the long-term effects of different heparin infusion adjustment strategies on the individual patient status, the expected cumulative reward of each strategy agent is calculated, and the global anticoagulation therapy strategy is updated accordingly.

[0059] In this invention, a policy agent refers to a policy unit that runs independently in a parallel computing structure. Each agent independently generates a set of heparin infusion actions based on the same input state information and executes the policy in its independent interactive environment to obtain reward feedback.

[0060] The global anticoagulation therapy strategy refers to the basic strategy formed within the global optimization module after integrating the results of multiple strategy proxy explorations, which is used to provide a stable decision-making benchmark for heparin infusion.

[0061] In the specific implementation process, within each strategy decision cycle, the system preferably uses a preset time step as the starting point to obtain the patient's current state vector and synchronously inputs this state vector into multiple strategy agents in the global optimization module. Each strategy agent independently constructs its own strategy network, preferably a feedforward neural network or a recursive network with a time memory structure. The network uses the current state vector as the input node and outputs a set of continuous action values. These action values ​​represent the adjustment amount of heparin infusion rate per unit time, and the unit can be set to International Units per kilogram per minute (IU / kg / min).

[0062] During the operation of the policy agent, the state vector is first standardized by a normalization module to ensure that the input data has a uniform distribution range. Then, after computation by each hidden layer node of the policy network, a set of action values ​​is finally generated at the output layer. The action value can be a single scalar (representing the rate of increase or decrease) or a two-dimensional vector (representing the additive adjustment amount and the multiplicative adjustment coefficient, respectively), used to describe the heparin adjustment operation recommended in the current state.

[0063] The actions generated by each policy agent are not directly used to control the actual device, but are instead input into a simulated interactive environment that closely resembles the patient's physiological state. This simulated environment is preferably an approximate physiological response model trained based on historical patient data or a state transition engine constructed by combining clinical pharmacokinetic models. The simulated environment receives the current state. With proxy output action The next state is generated by executing the state transition function. It outputs a reward signal associated with that state. This reward signal is used to measure the positive or negative impact of heparin adjustment on the anticoagulation effect.

[0064] The state transition function can be constructed in the following two ways:

[0065] Optional implementation schemes include: using random forest, gradient boosting tree, or lightweight neural network models to predict physiological indicators such as ACT changes and heparin concentration fluctuations, so as to approximate the physiological response process of real patients under similar conditions.

[0066] Optional implementation schemes also include: constructing a simplified heparin-coagulation dynamics model, considering the heparin metabolic half-life, weight adjustment factors, and the interaction strength between anticoagulants and anticoagulants, and combining it with an ODE (ordinary differential equation) solver to form a continuous state simulator.

[0067] As the policy agent continues to execute in the simulation environment, it records state-action-reward triplet data for multiple consecutive time steps and accumulates the expected reward values ​​for multiple time steps based on the reward function, forming the so-called long-term reward expectation. The reward function preferably adopts a negative bias structure, that is, the loss metric is constructed based on the absolute deviation between the current ACT value and the target ACT value. At the same time, auxiliary indicators such as heparin concentration over-limit penalty terms and action smoothing penalty terms can be added to enhance the safety and continuity of policy convergence.

[0068] The long-term reward expectation is preferably calculated using the time difference method or the Monte Carlo reward estimation method. After each simulation period, the system uses the accumulated reward as the target reward to perform backpropagation and parameter updates on the policy network. The parameter update method preferably employs the policy gradient method, proximal policy optimization (PPO), or asynchronous policy update mechanism.

[0069] After each policy agent completes its policy optimization, it uploads the updated policy network weight parameters or gradient change vector to the central policy updater. The updater includes an aggregation module that receives parameter inputs from each agent and performs parameter fusion operations according to preset rules. Parameter aggregation methods may include simple averaging, reward-based weighted averaging, or experience pool sampling updates. The resulting new global policy network weights are then distributed to each policy agent for the next round of policy iteration training.

[0070] In one specific implementation, the global optimization module constructs six policy agents, labeled Agent1 through Agent6. Each policy agent runs in an independent thread and contains a policy network and a value function network. The policy network is used to generate heparin infusion adjustment actions, and the value function network is used to evaluate the long-term reward of the action in the current state.

[0071] Each policy network is a three-layer feedforward neural network. The input layer has a size of 12, corresponding to the dimension of the input state vector, specifically including: patient weight, preoperative R value, K value, α angle, MA value, current body temperature, current ACT value, current heparin concentration, loop pressure, blood pump speed, anti-Xa activity, and surgical time progression encoding. The hidden layer has two layers, each containing 128 ReLU activation nodes. The output layer has one node, using the Tanh activation function to generate standardized action values ​​ranging from -1 to 1, representing the relative adjustment ratio of the heparin infusion rate. The output action values ​​are linearly mapped to the actual adjustment increment, in units of IU / kg / min, with a numerical range of -3.0 to +3.0.

[0072] Each policy agent employs a three-layer feedforward neural network with consistent parameter sizes and identical input / output dimensions. However, their training processes are completely independent, with different initial parameters, random seeds, policy update trajectories, and interaction paths. They explore different regions in the policy space, thus covering more potential solutions. This helps avoid the "policy trap" where the global policy gets stuck in a suboptimal solution early in training. Patient states in extracorporeal circulation are highly individualized and dynamically uncertain. The different exploration styles of the agents help the system generate diverse feedback, making the final global policy more robust and improving its adaptability to marginal and abnormal states.

[0073] Each agent completes action verification by interacting with a local physiological response simulator. The simulator is constructed using a data-driven state transition function, which is a trained lightweight neural network model trained on historical surgical data. The inputs are the current state vector and the heparin adjustment action, and the outputs are the ACT value and heparin concentration at the next time step. The simulator maintains a constant environment during operation to accurately simulate the dynamic behavioral sequence of the strategy under current conditions.

[0074] Within each simulation cycle, the Agent k From the current state Departure, generate action Input the action into the simulator to obtain With rewards This process lasts for several time steps (preferably 10 steps), forming a complete trajectory. After trajectory collection is complete, the reward is calculated using the advantage function. The advantage function is defined as the difference between the current reward and the output of the value function, used to reduce the variance of the policy gradient estimation and improve stability. The reward value is used to calculate the loss function of the policy network and update the agent's internal parameters through the backpropagation algorithm.

[0075] After each round of training, each agent sets its policy network parameters. The advantage value is sent to the global policy aggregator. The aggregator collects parameter updates from all agents and performs a weighted average operation, where the weights are determined by the standardized cumulative reward corresponding to that agent; that is, agents with better performance have higher policy weights. This aggregation forms a new round of global policy parameters. This will be broadcast and synchronized to all agents for the next round of training.

[0076] To prevent the strategy from overfitting to a particular class of patient data, the system incorporates a random state perturbation mechanism. During the initialization phase of each training round, the system adds Gaussian perturbation noise to the input states, including body temperature, anti-Xa, and loop pressure. The noise has a mean of 0 and a standard deviation set to 15% of the historical standard deviation of the parameters. This mechanism improves the robustness of the strategy and enhances its generalization ability to unseen states.

[0077] This module introduces multiple parallel-running strategy agents to simulate, evaluate, and learn the long-term dynamic effects of heparin infusion adjustment behavior under different strategy paths. This enables the system to fully identify the delayed and cumulative effects of different adjustment schemes on the coagulation process during the strategy generation stage, thereby avoiding the local optima caused by short-term gains from a single strategy.

[0078] Compared to traditional anticoagulation strategies that rely on single-path trial or empirical threshold adjustment, the policy agent mechanism in this invention enables the system to possess significant policy diversity and exploration capabilities. Different agents independently construct policy networks and use their respective interaction environments for simulation in asynchronous operation, allowing the system to cover a wider state and policy space. After each round of training, the agent parameters are weighted and merged through a central aggregator, ensuring that the final global policy inherits excellent policy behavior while retaining the overall steady-state control characteristics of the system, thereby achieving stable evolution and long-term performance optimization at the policy level.

[0079] Furthermore, the proxy simulation environment built through a physiological simulator allows the system to avoid directly affecting the patient's actual physiological state during the training phase, thus completing the initial training and testing of the strategy non-invasively and safely. This mechanism is particularly suitable for real-world scenarios with drastic fluctuations in intraoperative condition and limited response time windows, effectively alleviating the pressure and uncertainty faced by clinicians in the rapid decision-making process and improving the controllability and emergency response capability of the system's anticoagulation strategy.

[0080] Furthermore, the state perturbation mechanism introduced by the system before each round of strategy training can improve the generalization ability of the generated strategy in new states, significantly enhance the system's adaptability to different patient types and different pathological conditions, and have good cross-individual universality and deployment transferability.

[0081] In summary, by constructing a global optimization module and its multiple strategy agents, this invention achieves long-term dynamic modeling, parallel optimization, and multi-path fusion of heparin infusion strategies during extracorporeal circulation. The global anticoagulation strategy generated by the system not only has higher stability, foresight, and robustness, but can also serve as the basic strategy support for subsequent fine-tuning by local optimization modules, significantly improving the clinical safety and intelligent decision-making level of the entire anticoagulation regulation system.

[0082] In a preferred implementation, the multiple policy agents are deployed in a distributed manner on multiple computing nodes, or simulated asynchronous operation using logical threads within a single computing unit, thereby constructing a structurally isolated parallel policy system. Each policy agent exists as an independent entity within the system architecture, possessing its own independent state trajectory, policy parameters, value function evaluation path, and physiological simulation environment. They do not share any state, action, or reward information during the runtime phase, exchanging parameter gradients or policy weights only during the policy aggregation phase. Through this design, the system can generate highly differentiated and uncorrelated policy behaviors within the same policy input space, effectively avoiding the local optimum trap that may result from a single control path.

[0083] Each agent runs connected to a local lightweight simulation environment. The actions it generates are applied to the agent's environment model, simulating the ACT response trajectory and heparin concentration change curve over several future time steps. The agents use the accumulated reward values ​​from these trajectories to construct a long-term reward objective and independently update the parameters of the policy network. Throughout the training cycle, each agent completes policy evolution and asynchronously sends the optimization results back to the central aggregator. The aggregator performs weighted parameter fusion based on the expected rewards obtained by each agent, constructing a unified master policy network, and then sends this network back to each agent thread for the next round of distribution and training.

[0084] Through the aforementioned architectural design, the system significantly enhances the diversity of policy generation while maintaining policy concurrency efficiency, thereby improving its adaptability to complex state spaces. Multiple independent policy agents explore different heparin adjustment trajectories in parallel, avoiding homogenization of gradient update paths at the parameter level. This effectively reduces the probability of convergence jitter or gradient oscillations during training, thus improving the stability and reliability of the global anticoagulation strategy. In periods of drastic fluctuations in patient condition or nonlinear enhancement of the anticoagulation response mechanism, this structure can rapidly generate multiple candidate strategies and complete strategy optimization, providing structured support for subsequent real-time fine-tuning.

[0085] In another preferred implementation, the system employs an asynchronous advantage actor-critic structure to construct policy agents. Each policy agent independently contains a policy network and a value function network, used to generate control outputs and evaluate the potential reward of a state, respectively. The policy network takes the current state vector as input and outputs a heparin infusion rate adjustment value; the value function network takes the same state information as input and outputs a reward estimate for that state. The agent interacts with the simulation environment based on the action output by the current policy. After one round of training, an advantage value is constructed based on the difference between the actual reward obtained and the value function output, and the policy parameters are optimized based on this value.

[0086] This optimized architecture decouples state value estimation from policy behavior evaluation, enabling each policy agent to more accurately identify the behavioral path with the greatest potential for improvement in a given state. The introduction of advantage values ​​further enhances the agent's ability to identify significant behavioral contributions, making policy updates more selective and targeted. Multiple agents execute training in asynchronous threads at independent time steps, avoiding the waiting and blocking issues and data race problems inherent in synchronous update mechanisms, significantly improving the overall system's training throughput and operational efficiency.

[0087] In anticoagulation control, this asynchronous structure exhibits faster convergence speed and smaller strategy oscillation amplitude. Especially in clinical settings where ACT deviates significantly from the target range and rapid correction of heparin injection rates is required, this structure can quickly generate and fuse multiple strategies and rapidly switch to the optimal strategy path, improving the system's response speed to sudden anticoagulation abnormalities. Because each agent retains the independence of its strategy parameters during training, the system can maintain its multi-strategy exploration capability even when facing uncertain inputs, providing a solid foundation for stable system operation.

[0088] Considering that the patient's condition during surgery has obvious short-term fluctuations, this invention further includes a local optimization module to achieve real-time fine-tuning based on the global strategy.

[0089] During cardiopulmonary bypass, while a global anticoagulation strategy can provide stable guidance for heparin adjustment on a macroscopic timescale, significant short-term fluctuations in patient status, including sudden intraoperative temperature changes, hemodilution, drug intervention, or changes in perfusion parameters, can lead to the failure of the global strategy within a short period if a rapid response mechanism for these short-term disturbances is lacking. This can result in abnormal fluctuations in ACT (Activation Time to Coagulation) or excessive heparin concentration, affecting the immediacy and safety of anticoagulation control. To achieve fine-grained compensation and adjustment for real-time status changes and improve the immediate adaptability of the anticoagulation strategy, a strategy adjustment mechanism with short-term dynamic response capability needs to be introduced on top of the global strategy. Therefore, a local optimization module is introduced to dynamically model the short-term relationship between different heparin infusion adjustment strategies and the deviation of the patient's real-time coagulation activation time from the target value and the safe range of heparin concentration using a probability density function. Based on the principle of maximizing strategy entropy, the global anticoagulation treatment strategy is corrected in real time, and specific heparin infusion rate adjustments are generated.

[0090] The local optimization module refers to the computational module in the system strategy layer used to model the relationship between short-term state changes and control strategies. Its function is to dynamically correct the output of the global anti-coagulation strategy to adapt it to the state disturbances in the current cycle.

[0091] Policy entropy is a measure of the uncertainty of policy output, used to control the degree of exploration in the policy generation process. Maximizing policy entropy means introducing a certain degree of randomness into the policy output, thereby improving the policy's response flexibility and diversity.

[0092] In the specific implementation of the local optimization module, the system constructs a short-term physiological state vector in real time for each strategy execution cycle and generates heparin infusion rate adjustment actions based on a probabilistic strategy modeling framework. This module is used to refine the basic anticoagulation strategy output by the global optimization module, thereby enhancing the system's immediate response capability to rapid intraoperative status fluctuations.

[0093] In this invention, the local optimization module employs a parameterized strategy distribution modeling method, receiving the patient's current state in each cycle. The state vector has a dimension of 12 and includes:

[0094] The current activation time (ACT) value, plasma heparin concentration, core body temperature, loop pressure, pump rate, blood dilution ratio, intraoperative perfusion fluid temperature, anti-Xa factor activity, ACT rate of change within the time window, blood flow velocity, pre-cycle control output value, and system intervention response delay indicator are all normalized and input into the strategy modeling network.

[0095] The policy modeling network is a two-layer fully connected neural network. The input layer has 12 nodes, the first hidden layer has 256 nodes with ReLU activation, the second hidden layer has 128 nodes with Tanh activation, and the output layer has a dual-head structure, outputting the mean of the policy action. and logarithmic variance The system uses this parameter to construct the probability density function of the action:

[0096]

[0097] in, Indicates in the strategy parameters Under these conditions, the system is in state Take action at the time The probability density function defines the distribution of actions in the current state; Indicates the system in state The amount of heparin adjusted (in IU / kg / min) generated represents the degree of fine-tuning of the current baseline infusion rate. : indicates a value with a mean of variance is The normal distribution is used to model the state. The action distribution under the given conditions. By outputting a normal distribution defined by the mean and variance, the policy modeling network can not only provide stable control suggestions, but also express the uncertainty of the policy in the current state, which is used to assess the degree of exploration of the control system.

[0098] Before the strategy is executed, the system starts from the distribution. Mid-sampling generates multiple candidate actions Among them, those with superscript Indicates the first Actions at a given moment, without superscript. hour, The action at any given time is represented, and each candidate action is input into an online physiological response prediction model. The prediction model is a lightweight state transition neural network, trained based on historical patient data, and has a feedforward network structure. Output the state estimate for the next time step. Key predictive parameters include coagulation activation time in the next cycle. Heparin concentration in the next cycle .

[0099] For each sampling action, the system defines an immediate reward function based on the target anticoagulation effect:

[0100]

[0101] in,

[0102] This indicates the target coagulation activation time set during the procedure (e.g., 280 seconds).

[0103] This indicates the maximum permissible heparin concentration threshold (e.g., 2.0 IU / mL).

[0104] This is an indicator function; it takes a value of 1 when the predicted concentration exceeds the limit, and 0 otherwise.

[0105] , To adjust the weights, the preferred values ​​are 1.0 and 5.0.

[0106] All sampling actions and their corresponding rewards are used to construct the maximum entropy objective function:

[0107]

[0108] The entropy term is:

[0109]

[0110] This objective function maximizes policy returns while maintaining sufficient policy distribution width, ensuring that the system still has a variety of action choices in regions of state uncertainty. It is a common policy optimization objective function in reinforcement learning, representing the action. according to The sampling expectation; through the sampling expectation; The gradient is calculated and backpropagation is performed; the optimizer then updates the parameters of the policy network. The Adam algorithm is preferred for optimization, with a learning rate set to 3×10⁻⁶. -4 The weight decay factor is 10. -5 .

[0111] Ultimately, the system will determine the expected value of the policy distribution. As a local adjustment amount Output, and combine with policy actions from the global optimization module. The fusion process is performed. The fusion method uses a residual structure:

[0112]

[0113] in To adjust the intensity locally, the system adaptively sets the intensity based on the current state deviation. If the ACT deviation exceeds 15%, the intensity is increased. To improve the corrected response, the value should be increased to above 0.8; if the system is in a stable period, then... Keep it below 0.2 to avoid excessive intervention.

[0114] The local optimization module, by constructing a strategy generation mechanism based on probabilistic modeling, enables the system to maintain sufficient response flexibility when facing short-term, rapidly fluctuating physiological states. During extracorporeal circulation, changes in the patient's coagulation state not only exhibit high individual variability but also often display characteristics such as nonlinearity, rapid mutation, and sensitivity to external disturbances. Relying solely on global strategy output can easily lead to problems such as lagging control response and insufficient adjustment granularity, making precise intervention impossible during state mutations. However, by introducing a modeling mechanism based on strategy distribution, the system can not only output deterministic regulatory actions but also simultaneously express the uncertainty and dynamic risk of the strategy under the current state, thereby giving the control strategy dynamic adjustment flexibility and feedback fault tolerance.

[0115] By using a parameterized probability density function-based action generation structure, the local optimization module can establish a continuous spatial distribution mapping of the adjustment behavior in the current state and dynamically adjust the action selection range based on this mapping. The introduction of a policy entropy maximization mechanism allows the system to maintain high action diversity even when the state space is not fully explored or when there is a multi-peak response trend, thus preventing the policy from getting trapped in early convergence or suboptimal control regions. Furthermore, through the combined effect of an online reward calculation mechanism and action-state simulation feedback, the system can quickly evaluate the impact of multiple candidate actions on ACT bias and heparin concentration fluctuations based on real-time prediction signals, thereby achieving immediate correction of the policy distribution.

[0116] Furthermore, the adjustment actions output by the local optimization module and the basic policy output generated by the global optimization module adopt a residual fusion approach in structure. This ensures that the output of the local policy does not directly overwrite the original policy, but rather is superimposed on the original policy as a correction. Therefore, while preserving long-term stability, it effectively introduces short-term control capabilities. This structure avoids conflicts in control between global and local policies, enabling the system to have continuous control capabilities from long-term policy stability to short-term immediate response. Fusion control not only improves the policy's fault tolerance in edge states but also effectively enhances the system's generalization ability to non-standard input states.

[0117] The aforementioned mechanism constructs a rapid response control scheme for dynamic state fluctuations during extracorporeal circulation, enabling the system to simultaneously consider long-term effects and short-term deviation corrections in anticoagulation strategy control. By introducing probability distribution and strategy entropy, the model's behavioral expressiveness and adjustment flexibility are enhanced, demonstrating good adaptability and robustness in multi-patient state spaces. In multi-round simulation experiments and clinical data-driven validation, this mechanism effectively shortened ACT recovery time, reduced the frequency of heparin concentration exceeding limits, and significantly reduced the intraoperative intervention failure rate caused by control system lag, demonstrating the system's high adaptability and control precision in complex pathological state changes.

[0118] In a preferred implementation, the local optimization module is constructed using a reinforcement learning method based on a soft policy architecture. Preferably, the Soft Actor-Critic (SAC) algorithm is chosen as the policy optimization framework to achieve stable control over heparin infusion behavior under real-time fluctuations in the patient's state. This method introduces a maximum entropy enhancement mechanism on top of the original reinforcement learning structure, enabling the policy to actively retain the ability to explore the potential optimal policy space while obtaining high rewards, thereby enhancing the overall system's generalization and adaptability in dynamic environments.

[0119] In its implementation, the local optimization module comprises two main network structures: a policy network and a value function network. The policy network outputs parameters of the action distribution, specifically the expected value (mean) of the heparin adjustment in the continuous action space and the degree of action uncertainty (variance). This network is modeled using a Gaussian distribution, with the output node structure designed as a two-branch system, corresponding to the mean component and the standard deviation component, respectively. The mean is used to generate the optimal heparin adjustment proposal for the current state, while the variance reflects the degree of uncertainty of the policy in that state. The system samples from this distribution to generate the actual heparin adjustment actions used for simulating feedback.

[0120] The value function network estimates the expected reward between the current state and the sampled action pair, and introduces an entropy term to construct a maximum entropy value function. In the SAC algorithm, the optimization objective of policy update is no longer simply to maximize the expected reward, but to maximize "expected reward plus entropy value," so that the obtained policy has higher information entropy while obtaining the expected benefit, thereby encouraging the policy to retain a certain degree of exploratory nature. This design is particularly critical in the context of clinical control, because the physiological state space of patients undergoing cardiopulmonary bypass is often non-stationary, with many critical state regions. Policies that rely solely on maximizing reward are prone to getting trapped in local optima and ignoring potentially better adjustment paths.

[0121] During strategy optimization, the system takes the current patient state vector as input, outputs a set of policy distribution parameters through the policy network, samples the heparin adjustment action based on this distribution, and inputs it into the simulator or actual feedback environment to obtain the next state and immediate reward value. This reward value comprehensively considers indicators such as the deviation from the ACT target, the heparin concentration safety boundary, and the smoothness of the adjustment amplitude, and is used to jointly optimize the policy network and value function network during training. During training, SAC employs two independent Q functions to reduce estimation bias and introduces a target network for delayed updates, thereby improving the stability of the overall policy estimation.

[0122] This optimized structure, in the process of generating control strategies, no longer relies on a single optimal action path. Instead, it maintains the breadth of the strategy space through a maximum entropy mechanism, enabling the system to tolerate and adjust to multiple feasible control paths even under conditions of nonlinear state disturbances, modeling errors, or observation noise. This structure enhances the system's robustness and generalization ability, allowing it to stably generate adjustment actions with medical safety boundary constraints when dealing with rapid hemodynamic changes or non-standard individual states, significantly improving the system's feasibility and reliability in real clinical environments.

[0123] Furthermore, in multiple rounds of simulation experiments, this soft-strategy structure demonstrated superior adjustment sensitivity compared to traditional hard-strategy models within a short time window. Particularly in the early stages of anomalies in anticoagulation, it generated effective responses more quickly, intervened earlier, shortened the ACT recovery time, and reduced the fluctuation amplitude of heparin concentration, exhibiting a significant improvement in strategy response rate and control accuracy under dynamic high-risk conditions. This optimized implementation, by combining strategy distribution modeling, entropy regularization, and delayed Q estimation mechanisms, provides stronger adaptability and control flexibility to the local optimization module, significantly enhancing the system's ability to adapt to individualized anticoagulation interventions.

[0124] After strategy optimization, the control signal output by the local optimization module is transmitted to the heparin precision infusion module as a dynamic adjustment result. This control signal indicates the heparin infusion rate adjustment to be implemented in the current state, thereby achieving real-time closed-loop control of anticoagulation therapy. This control signal is typically represented as a standardized continuous numerical value, which, after mapping and physical quantity conversion, is directly used to drive the output adjustment unit of the infusion device, thus achieving precise adjustment of the drug dosage.

[0125] In the specific implementation process, the system first performs a numerical normalization inverse transformation on the adjustment signal output by the local optimization module. The normalization coefficient is set during the system initialization phase based on the patient's weight, infusion pump type, and dose response curve, ensuring that the output range of the control signal strictly falls within the device's allowable operating range. After inverse normalization, the resulting infusion rate value is expressed in IU / kg / min (International Units per kilogram per minute), indicating the dose of heparin to be injected into the extracorporeal circulation tubing per unit time.

[0126] The heparin precision infusion module is preferably integrated into a digitally controlled infusion pump drive system. This system supports high-frequency control signal input and has sub-milliliter-level flow resolution. Upon receiving a control signal, the module adjusts the operating frequency of its internal stepper motor or piezoelectric drive mechanism according to the signal's amplitude, thereby changing the propulsion rate of the heparin reservoir and achieving dynamic control of the infusion flow rate. To ensure control accuracy and response speed, the module is also equipped with a flow closed-loop feedback mechanism, which fine-tunes the flow based on the difference between the real-time collected infusion volume and the target infusion volume, ensuring that the actual infusion rate remains highly consistent with the expected value.

[0127] In a preferred implementation, the heparin precision infusion module shares a clock synchronization signal with the system's status acquisition module and is equipped with a timestamp calibration mechanism to ensure strict temporal alignment between the control signal and patient status data, thereby avoiding dose overshoot or anticoagulation delays caused by control lag or response drift. Furthermore, the heparin precision infusion module preferably employs a redundant control logic design, with the main control unit and backup unit automatically switching in case of critical equipment failure or data anomalies, ensuring a continuous and stable drug delivery process in the intraoperative environment.

[0128] By seamlessly integrating the control signals generated by the local strategy module into the actual heparin infusion device control chain, this system achieves a closed-loop connection throughout the entire process, from state recognition and strategy generation to drug execution. This mechanism significantly improves the response speed and dosage accuracy of anticoagulation control, effectively reduces the risk of anticoagulation failure caused by adjustment lag, operational errors, or individual differences, and ensures that coagulation function is maintained within a safe and stable range throughout the entire cardiopulmonary bypass process, ultimately improving patient safety during surgery and the quality of postoperative recovery.

[0129] To achieve adaptive iteration and online updating of the control strategy, this invention further constructs a feedback update module to feed treatment results back to the decision-making system, continuously optimizing the adaptability and accuracy of global and local anticoagulation regulation strategies. Because patients' physiological responses during cardiopulmonary bypass exhibit high individual variability and time-varying characteristics, fixed strategies often fail to maintain optimal effectiveness throughout the entire surgical cycle. Therefore, the system needs to dynamically modify the control strategy based on actual treatment results to achieve continuous evolution and multi-round adaptive adjustment of strategy parameters.

[0130] The feedback update module constructs a feedback channel for strategy correction by collecting the patient's anticoagulation response data in real time during each control cycle. Preferably, the feedback data includes, but is not limited to, the following key indicators: the deviation between the current activation time (ACT) and the target ACT, the trend of actual heparin concentration changes, whether the safety threshold is exceeded, the heparin dose accumulation curve, changes in blood circuit pressure, and anti-Xa activity indicators. All of the above data are obtained through a status acquisition module within the system on a high-frequency sampling basis and include precise timestamps to ensure that the correspondence with the control signals does not deviate.

[0131] In its implementation, the feedback update module comprises two main functional units: a global feedback channel and a local feedback channel. The global feedback channel primarily serves the global optimization module. Its core task is to summarize the policy execution effects across multiple decision cycles and reconstruct the objective function of each policy agent during policy training in the form of long-term rewards. This process preferably employs temporal difference learning or Monte Carlo path reward estimation techniques, using state-action-result triples constructed from real execution trajectories to back-update the policy network and value function network parameters in each policy agent, thereby iteratively enhancing the policy's predictive ability for long-term anticoagulation efficacy.

[0132] The local feedback channel focuses on the fine-tuning response relationship between control signals and immediate treatment effects within short cycles. Specifically, within a short time window after each control signal is executed, the system analyzes in real time whether the ACT change caused by the adjustment meets expectations and whether the heparin concentration fluctuates within the target range, and calculates the deviation based on the loss function structure. The feedback update module uses this process to adjust the probabilistic policy distribution parameters in the local optimization module, especially key parameters such as the mean, variance, and policy entropy weights of the action distribution, thereby achieving adaptive correction to the current state and improving the accuracy and stability of the policy response.

[0133] Preferably, the feedback update module also includes an anomaly monitoring mechanism. When the ACT value continuously deviates from the target range for several consecutive cycles, or when the heparin concentration exhibits a non-linear abnormal fluctuation trend, the module can trigger a control freeze strategy, suspending the current strategy parameter update process and switching to manual intervention mode or low-risk control mode. It also records the complete state-action-outcome trajectory for subsequent retrospective analysis. This mechanism helps avoid the accumulation of erroneous strategies caused by model misjudgment or sensor abnormalities, ensuring patient safety at the system level.

[0134] By introducing and integrating the aforementioned feedback update mechanism, this system achieves closed-loop training and online adaptation of the control strategy, breaking the inherent limitations of the traditional control scheme of "static design - offline deployment - manual intervention". This enables anticoagulation regulation to be corrected and continuously optimized in real time based on the dynamic feedback of the patient's physiological response during the operation, significantly improving the system's ability to cover individualized treatment pathways under complex, nonlinear, and multi-factor influences, and effectively enhancing the practicality, robustness, and intelligence of the anticoagulation control system in the clinical environment of extracorporeal circulation.

[0135] Furthermore, this invention also provides an intelligent management method for anticoagulation regulation in extracorporeal circulation. The method is based on the structure and function of the aforementioned intelligent management system for anticoagulation regulation. It relies on the collaborative work of the patient status acquisition module, global optimization module, local optimization module, heparin precision infusion module, and feedback update module to construct a complete anticoagulation control process covering preoperative initialization, intraoperative regulation, and intraoperative update.

[0136] In this method, the system first acquires multiple individual status indicators, including weight, basic coagulation function, real-time ACT value, current heparin concentration, body temperature, blood dilution ratio and circuit pressure, through the patient status acquisition module, and constructs a status vector based on these indicators to describe the patient's current anticoagulation requirements.

[0137] Subsequently, the state vector is synchronously input into the global optimization module, where multiple parallel policy agents explore the long-term effects of different heparin adjustment schemes in that state. Each agent executes its policy in its independent simulation environment, evaluates the reward, updates the policy parameters, and aggregates the results to the central policy aggregator to generate the current global baseline anticoagulation policy.

[0138] Based on the global strategy, the system further calls the local optimization module, uses a probabilistic modeling network to construct a strategy distribution function for heparin adjustment starting from the current state, and introduces a strategy entropy adjustment mechanism to achieve short-cycle adaptive strategy adjustment, outputting a specific heparin infusion rate correction.

[0139] This correction is then converted into a drug pump control signal in real time by the heparin precision infusion module, driving the heparin delivery system to complete the actual adjustment operation. After the adjustment is completed, the feedback update module collects the new ACT value and heparin concentration, and links them with the decision record of the previous control cycle to correct the strategy network structure in the global optimization module and the local optimization module, so as to realize the dynamic iteration and continuous evolution of the strategy.

[0140] This method enables precise modeling, dynamic prediction, and real-time correction of anticoagulation protocols throughout the entire extracorporeal circulation process. It ensures that anticoagulation control at each stage is precisely matched to the patient's condition, thereby effectively reducing the risk of thrombosis and bleeding complications, and improving the overall safety and therapeutic effect of the extracorporeal circulation process.

[0141] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0142] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0143] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0144] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. For some module structures not specifically defined in this invention, the content described in the prior art shall prevail. The prior art mentioned in the foregoing background and specific embodiments sections can be considered as part of this invention and used to understand the meaning of some technical features or parameters.

Claims

1. An anticoagulation regulation intelligent management system for extracorporeal circulation, characterized in that, The system comprises the following modules: a patient state acquisition module for acquiring and updating patient individual state information related to anticoagulation therapy in real time, and constructing a feature vector for representing the current anticoagulation needs of the patient; a global optimization module for constructing a plurality of parallel strategy agents, calculating the expected cumulative reward of each strategy agent by respectively exploring the long-term effect of different heparin infusion adjustment strategies on the patient individual state, and updating the global anticoagulation therapy strategy accordingly; a local optimization module for dynamically modeling the short-term relationship between different heparin infusion adjustment strategies and the deviation of the patient's coagulation activation time from the target value and the heparin concentration safety range based on a probability density function, real-time correcting the global anticoagulation therapy strategy based on the principle of maximizing strategy entropy, and generating a specific heparin infusion rate adjustment amount; a heparin precise infusion module for receiving the heparin infusion rate adjustment amount and updating the heparin infusion rate in real time accordingly; a feedback update module for feeding back the patient's real-time coagulation activation time deviation from the target value and the current heparin concentration to the global optimization module and the local optimization module to continuously correct the strategy parameters of the global optimization module and the local optimization module; the probability density function is: wherein, represents the probability density function of the action taken by the system at state when the policy parameters are the system generates a heparin adjustment amount in the state the system generates a heparin adjustment amount in the state represents a normal distribution with mean , variance . Before policy execution, the system generates a plurality of candidate actions from a distribution wherein represents the th time instant, each candidate action is input to an online physiological response prediction model;​ The prediction model is a light-weight state transition neural network, trained based on historical patient data, structured as a feed-forward network, receiving as input and outputting an estimate of the state at the next time instant where the key prediction terms include the clotting activation time at the next cycle and the heparin concentration at the next cycle ; For each sampled action, the system defines an immediate reward function based on the target anticoagulation effect: wherein, representing the target clotting activation time set intraoperatively, represents the threshold of the maximum heparin concentration allowed, is an indicator function that takes the value 1 if the predicted concentration is out of limits, and 0 otherwise, , to adjust the weights; all sampled actions and their corresponding rewards are used to construct a maximum entropy objective function: For the objective optimization function, denote According to the sampling expectation; wherein the entropy term is: 。 2. The extracorporeal circulation-oriented anticoagulation regulation intelligent management system according to claim 1, characterized in that, After each strategy agent completes its strategy optimization, it uploads the updated strategy network weight parameters or gradient change vector of this round to the central strategy updater; the updater is constructed with an aggregation processing module for receiving parameter input from each agent and performing parameter fusion operations according to pre-set rules; the new global strategy network weight obtained after aggregation will be distributed to each strategy agent for the next round of strategy iteration training.

3. The extracorporeal circulation anticoagulation regulation intelligent management system according to claim 1, characterized in that, In the construction of the plurality of parallel strategy agents, the network structure, parameter size, input and output dimensions of each strategy agent are the same, but the training process is completely independent, and the initial parameters, random seeds, strategy update trajectories and interaction paths are not the same.

4. The extracorporeal circulation anticoagulation regulation intelligent management system according to claim 1, characterized in that, The plurality of parallel strategy agents are respectively deployed on multiple computing nodes in a distributed manner.

5. The extracorporeal circulation anticoagulation regulation intelligent management system according to claim 1, characterized in that, The strategy agents are constructed using the asynchronous advantage actor critic structure, each strategy agent independently includes a policy network and a value function network for generating control output and evaluating state return potential, respectively; the policy network inputs the current state vector and outputs the heparin infusion rate adjustment value; the value function network inputs the same state information and outputs the return estimate in that state; the agent interacts with the simulated environment with the action output by the current policy, and after a round of training, the advantage value is constructed based on the difference between the actual reward obtained and the value function output, and the strategy parameters are optimized based on this.

6. An anticoagulation regulation intelligent management method for extracorporeal circulation, characterized in that, Using the extracorporeal circulation anticoagulation control intelligent management system according to any one of claims 1-5 for extracorporeal circulation anticoagulation control.

Images