Thermal management control system for pulsatile blood pump based on rotational speed modulation

Abstract

This invention discloses a pulsed blood pump thermal management control system based on speed modulation. The control system is configured to perform the following steps: generating an initial state vector based on historical key motor parameters, blood pump output flow rate, blood temperature, and aortic differential pressure; using a strategy network to output a speed adjustment vector based on the initial state vector, thereby controlling the motor speed; generating a current state vector based on the current key motor parameters, blood pump output flow rate, blood temperature, and aortic differential pressure, and calculating an immediate reward; updating the value network based on the state vectors at n time points and the immediate reward; and updating the strategy network based on the prediction reward error of the value network. This invention achieves temperature rise minimization under constraints such as blood pump output flow rate, blood temperature, and aortic differential pressure, ensuring that the blood pump provides stable blood flow support during long-term operation while effectively preventing adverse biological effects.

Description

Technical Field

[0001] This invention belongs to the field of implantable medical device control technology, and particularly relates to a pulsed blood pump thermal management control system based on speed modulation. Background Technology

[0002] For patients with advanced heart failure, pulsatile blood pumps are a vital life-saving tool. Their operation requires not only sufficient blood flow but also mimicking the pulsatile characteristics of a heartbeat. Pulsatile blood pumps typically rely on brushless motors to deliver blood. Their speed control strategy often employs a rapid acceleration followed by a quick deceleration to simulate the heartbeat process. This causes a large amount of mechanical energy to be rapidly converted into heat energy, leading to localized temperature rise and heat loss. Overheating of the motor can cause protein denaturation in the blood, damage to blood cell membranes, and even blood cell lysis. Simultaneously, the rapid temperature increase can interfere with local vascular temperature control mechanisms, increasing the risk of tissue thermal damage, inflammation, and local tissue necrosis.

[0003] Traditional motor heat loss models primarily calculate based on factors such as copper loss, iron loss, and mechanical loss. However, under the operating conditions of pulsating blood pumps, due to complex nonlinear effects and fluctuations in environmental parameters, these models struggle to accurately capture actual heat loss. Furthermore, pulsating speed regulation exhibits significant time-varying characteristics, with heat loss generation and dissipation showing a dynamic evolution trend. Traditional static models cannot predict instantaneous heat loss (or thermal power) and cumulative heat energy over the entire cycle in real time. Moreover, existing control strategies often employ fixed parameters or simple feedback control, failing to fully consider the time-varying issues caused by individual patient differences and dynamic physiological states. Existing methods (such as particle swarm optimization algorithms) can only perform short-term planning, easily getting trapped in local optima, potentially leading to excessive heat accumulation under specific operating conditions, severely impacting the long-term stable operation of the blood pump. Summary of the Invention

[0004] The purpose of this invention is to provide a pulsed blood pump thermal management control system based on speed modulation, in order to solve at least one of the following problems: traditional motor thermal loss models are difficult to capture the complex nonlinear and temporal characteristics generated by pulsed speed regulation, resulting in deviations in the prediction of instantaneous thermal power and cumulative thermal energy over the entire cycle; existing methods can only perform short-term planning, which may lead to excessive heat accumulation under specific operating conditions, seriously affecting the long-term stable operation of the blood pump.

[0005] This invention solves the above-mentioned technical problems through the following technical solution: a pulsed blood pump thermal management control system based on speed modulation, wherein the control system is configured to perform the following steps:

[0006] Step 1: Generate the initial state vector based on the key parameters of the motor, the blood pump output flow, the blood temperature, and the aortic pressure difference at historical moments;

[0007] Step 2: Utilize the strategy network to output the rotational speed adjustment vector based on the initial state vector;

[0008] Step 3: Adjust the vector control of the motor speed according to the rotational speed;

[0009] Step 4: Obtain the current key parameters of the motor, blood pump output flow, blood temperature, and aortic pressure differential;

[0010] Step 5: Generate the current state vector based on the current key parameters of the motor, the blood pump output flow, the blood temperature, and the aortic pressure difference, and calculate the immediate reward;

[0011] Step 6: Based on the state vector at the current moment, repeat steps 2 to 5 n-1 times to obtain the speed adjustment vector, state vector and instant reward at n moments;

[0012] Step 7: Update the value network based on the state vectors at n time points and the immediate reward;

[0013] Step 8: Update the policy network based on the prediction reward error of the value network;

[0014] Step 9: Repeat steps 2 to 8 based on the new starting state vector until the entire cycle has been traversed;

[0015] Step 10: Repeat steps 1 to 9 multiple times until the termination condition is met, and obtain the trained policy network.

[0016] Furthermore, the key parameters of the motor include speed and cumulative thermal energy. The cumulative thermal energy is calculated based on the motor's thermal power, which is equal to the sum of the base thermal power and the thermal power correction value.

[0017] Furthermore, the basic thermal power is calculated using a physical model of motor thermal power, and the expression of the physical model of motor thermal power is:

[0018] ;

[0019] in, Indicates the basic thermal power of the motor; Indicates electromagnetic torque. Represents the torque constant. Indicates the stator winding resistance; This represents the core hysteresis loss coefficient. Indicates the operating frequency. denoted by , where m represents the magnetic flux density and m represents the exponent of hysteresis loss in the core loss. This describes the material parameters that describe the eddy current losses in the iron core; Indicates the motor speed. This represents the empirical coefficient related to the first-order term of the rotational speed. This represents the empirical coefficient related to the quadratic term of the rotational speed, and t represents time.

[0020] Furthermore, the thermal power correction value is obtained by predicting the motor operating indicators using a thermal power correction prediction model; the process of obtaining the thermal power correction prediction model includes:

[0021] Construct a sample dataset; wherein each sample in the sample dataset includes an input quantity and an output quantity, the input quantity being a motor operating index, and the output quantity being a thermal power correction value, the thermal power correction value being equal to the difference between the actual measured thermal power value and the baseline thermal power;

[0022] Construct an LSTM-TCN neural network model;

[0023] The LSTM-TCN neural network model is trained using the sample dataset to obtain the thermal power correction prediction model.

[0024] Furthermore, the output of the LSTM-TCN neural network model is:

[0025] ;

[0026] ;

[0027] ;

[0028] in, This indicates the predicted thermal power correction value; and These represent the weights and biases of the fully connected layer, respectively. Indicates the output of TCN; Indicates the position of the LSTM output sequence. H represents the hidden layer dimension of the LSTM; Indicates the kernel weights of the TCN; Indicates the activation function; Indicates the kernel size of the TCN; Indicates the bias of TCN; Indicates in At time t, the hidden state vector output by the LSTM is... Indicates the coefficient of thermal expansion; Let k represent the LSTM input sequence, and k represent the number of elements in the LSTM input sequence.

[0029] Furthermore, the formula for calculating the instant reward is as follows:

[0030] ;

[0031] in, Indicates the immediate reward at the current moment; This represents the difference in motor thermal power between the current moment and the previous moment. , This represents the motor's thermal power at the current moment. This represents the motor's thermal power at the previous moment; , and These represent the penalty factors for the corresponding terms; This indicates the blood pump output flow rate at the current moment. and These represent the minimum and maximum output flow rates of the blood pump, respectively. This indicates the current blood temperature. Indicates the upper limit of the safe temperature; This represents the aortic pressure difference at the current moment. and These represent the minimum and maximum values ​​of the aortic pressure gradient, respectively. This symbol represents a conditional judgment. It takes the value 1 when the condition is met and 0 when the condition is not met.

[0032] Furthermore, the loss function of the value network is:

[0033] ;

[0034] ;

[0035] in, The loss function represents the value network. The parameters represent the value network; N represents the number of moments in the entire cycle. Represents the state vector of the value network at time t. Output the estimated reward from time t to time t+N; This represents the sum of the cumulative discounted reward calculated based on the immediate rewards at n time points and the estimated reward at the next Nn time points; Indicates the discount factor; Represents the immediate reward at time t+k; This represents the estimated reward from time t+n to time t+N, determined by the value network.

[0036] Furthermore, the loss function of the policy network is:

[0037] ;

[0038] ;

[0039] in, The loss function of the policy network is represented. Represents the parameters of the policy network; This represents the error in predicting rewards within the value network. Represents the state vector Lower output speed adjustment vector The probability of; Represents the speed adjustment vector; This represents the sum of the cumulative discounted reward calculated based on the immediate rewards at n time points and the estimated reward at the next Nn time points; Represents the state vector of the value network at time t. Output the estimated reward from time t to time t+N.

[0040] Furthermore, the control system is also configured to perform the following steps:

[0041] Acquire key parameters of the motor, blood pump output flow, blood temperature, and aortic pressure differential at the start time;

[0042] The state vector at the start time is generated based on the key parameters of the motor, the blood pump output flow, the blood temperature, and the aortic pressure difference.

[0043] The trained policy network outputs a rotation speed adjustment vector based on the state vector at the initial moment.

[0044] Motor speed is controlled by adjusting the speed vector.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0046] This invention minimizes the cumulative heat energy over a period of time by using the state vector and the parameters of the designed instant reward online update strategy network. At the same time, it ensures that the blood pump output flow, blood temperature, and aortic pressure difference meet predetermined requirements. In other words, it minimizes the temperature rise under multiple constraints such as blood pump output flow, blood temperature, and aortic pressure difference, ensuring that the pulsating blood pump can provide stable blood flow support during long-term operation and effectively prevent adverse biological effects caused by local overheating. This significantly improves the safety and reliability of the pulsating blood pump during long-term operation.

[0047] This invention calculates motor thermal power using a physical model and a thermal power correction prediction model. It utilizes the prior information provided by the physical model and captures complex nonlinear and temporal features through deep learning. It does not require a large amount of training data, which not only improves the prediction accuracy of motor thermal power, but also solves the problem that traditional motor thermal loss models are unable to capture complex nonlinear and temporal features, resulting in deviations in the prediction of instantaneous thermal power and cumulative thermal energy over the entire cycle. Attached Figure Description

[0048] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only one embodiment of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart of the modeling process for the speed control of the pulsating blood pump motor in an embodiment of the present invention;

[0050] Figure 2 This is a flowchart of the speed control of the pulsating blood pump motor in an embodiment of the present invention. Detailed Implementation

[0051] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0053] The pulsed blood pump thermal management control system based on speed modulation provided in this embodiment of the invention is configured to execute a pulsed blood pump motor speed control modeling step and a pulsed blood pump motor speed control step. Wherein, as... Figure 1 As shown, the modeling steps for the speed control of a pulsed blood pump motor include:

[0054] Step 1: Generate the initial state vector based on the key parameters of the motor, the blood pump output flow, the blood temperature, and the aortic pressure difference at historical moments.

[0055] To ensure that speed control fully reflects the current operating state of the system, a state vector needs to be constructed. To minimize the thermal power of the blood pump motor, ensure temperature control safety, and meet physiological requirements for blood pump flow rate and pulsation, the state vector of this invention is generated based on key motor parameters, blood pump output flow rate, blood temperature, and aortic pressure differential. Key motor parameters include speed and accumulated thermal energy. Therefore, the state vector can be expressed as:

[0056] (1)

[0057] in, This represents the state vector at time t; This indicates the motor speed, which is acquired by a speed sensor. Indicates electromagnetic torque; This indicates the accumulated heat energy of the motor; The pump output flow rate at time t (unit: L / min) is fed back from the cardiovascular dynamics model; The blood temperature at time t (in °C) is fed back from the cardiovascular dynamics model. The aortic pressure gradient at time t (unit: mmHg) is fed back from the cardiovascular dynamics model. The accumulated heat energy is calculated based on the motor's thermal power; the specific calculation formula is as follows:

[0058] (2)

[0059] in, This represents the cumulative heat energy from time zero to time t. This represents the motor thermal power at time i. Indicates the speed adjustment interval.

[0060] Traditional motor heat loss models primarily rely on factors such as copper loss, iron loss, and mechanical loss for calculation. However, due to complex nonlinear effects and fluctuations in environmental parameters, traditional motor heat loss models struggle to accurately capture the actual heat loss in pulsating blood pumps. Furthermore, pulsating speed regulation exhibits significant time-varying characteristics, with heat loss generation and dissipation showing a dynamic evolution trend. Traditional motor heat loss models, being static, cannot predict instantaneous heat loss (or thermal power) and cumulative heat energy over the entire cycle in real time.

[0061] To address the aforementioned technical problems, this invention realizes the thermal power of the motor through a physical model of motor thermal power and a thermal power correction prediction model. Calculation:

[0062] (3)

[0063] in, This represents the basic thermal power of the motor, calculated from the physical model of motor thermal power. This represents the thermal power correction value, which is predicted by the thermal power correction prediction model.

[0064] In a specific embodiment of the present invention, the expression for the physical model of motor thermal power is as follows:

[0065] (4)

[0066] in, Indicates electromagnetic torque. , This represents the blood load determined by the cardiovascular dynamics model. This represents the acceleration determined by the cardiovascular dynamics model. This represents friction and fluid damping; This represents the torque constant, used to convert current into torque, and its specific value is 0.01 Nm / A; Indicates the stator winding resistance; This represents the core hysteresis loss coefficient, with a specific value of 0.002. This indicates the operating frequency, which is directly proportional to the rotational speed. , This represents the proportionality coefficient, which is related to design factors such as the number of pole pairs in the motor. Magnetic flux density represents the magnetic flux intensity within the motor and depends on the motor design, for example... Approximately 0.35T; m represents the exponent of hysteresis loss in the core loss, and its value ranges from 1.5 to 2.5. In this embodiment, the value of m is 2. The material parameters describing the eddy current losses in the iron core are shown in this embodiment. The value is 1×10 -4 ; Indicates the motor speed; This represents an empirical coefficient related to the first-order term of the rotational speed. In this embodiment, The value is 1×10 -3 W / (rpm); This represents the empirical coefficient related to the quadratic term of the rotational speed. In this embodiment, The value is 5×10 -7 W / (rpm²).

[0067] A thermal power correction prediction model is used to correct the basic thermal power calculated by the physical model of motor thermal power, thereby improving the accuracy of motor thermal power prediction. This model can be applied to several key scenarios: in real-time monitoring and early warning, it is deployed in blood pump controllers to output corrected thermal power prediction values ​​in real time, and to issue timely warning signals if the value exceeds a safety threshold; in equipment design optimization, through joint training with CFD simulation data, it provides feedback on local thermal risks in motor structure and heat dissipation design, offering optimization basis for material selection and structural layout; in clinical safety assessment, the thermal power correction prediction model can quantitatively analyze the thermal risks of the motor during long-term operation, improving the accuracy of thermal safety assessment and providing reliable support for implantation risk judgment.

[0068] The thermal power correction prediction model is a data-driven deep learning model. In a specific embodiment of this invention, the process of obtaining the thermal power correction prediction model includes:

[0069] Step 1.1: Construct the sample dataset.

[0070] Each sample in the sample dataset includes an input and an output. The input is the motor operating index, and the output is the thermal power correction value, which is equal to the difference between the actual measured thermal power and the baseline thermal power.

[0071] The actual measured values ​​of thermal power under different motor operating parameters can be obtained through experiments or simulations. Experiments include animal experiments and in vitro experiments. Animal experiments involve implanting blood pumps into animals after obtaining ethical permission to obtain actual measured values ​​of thermal power under different motor operating parameters. In vitro experiments involve constructing a simulated circulatory system, operating the blood pump under different motor operating parameters, and obtaining actual measured values ​​of thermal power. Simulations involve generating thermal power data using CFD and finite element thermal analysis.

[0072] In a specific embodiment of the present invention, the motor operating parameters include speed, electromagnetic torque, magnetic flux density, and operating frequency. Therefore, the input quantity for each sample can be expressed as:

[0073] (5)

[0074] Organize the motor operating parameters at time t and the k times preceding it to generate an input sequence, which can be represented as:

[0075] (6)

[0076] in, This represents the input sequence. To reflect the number of elements and time period in the input sequence, the input sequence can also be represented as... .by As input to the LSTM-TCN neural network model (i.e., Long Short-Term Memory Network - Temporal Convolutional Neural Network), this method enables the LSTM-TCN neural network model to learn the impact of historical information on the current state.

[0077] Step 1.2: Construct the LSTM-TCN neural network model.

[0078] The thermal power correction prediction model adopts a two-level network structure, namely a Long Short-Term Memory (LSTM) network and a Temporal Convolutional Neural Network (TCN). The LSTM is used to learn long-term dependency information in the input sequence, and the TCN uses one-dimensional convolution to extract local (short-term) temporal features, thereby further enhancing the features output by the LSTM. Finally, the features extracted by the TCN are mapped to a single scalar output, namely the thermal power correction value, through a fully connected layer.

[0079] By utilizing the memory units and gating mechanism of LSTM, long-term trends in indicators such as rotational speed in the input sequence can be captured. LSTM excels at capturing long-term temporal dependencies and avoids the gradient vanishing problem, making it suitable for describing the dependencies of data over a relatively long time span. After inputting the input sequence into the LSTM, the output hidden state vector is:

[0080] (7)

[0081] in, The hidden state vector is the output of the LSTM, i.e., the output sequence of the LSTM; LSTM stands for Long Short-Term Memory Network; H represents the hidden layer dimension of the LSTM. In this embodiment, the value of H is 100, and H determines the size of the hidden state. Represents the real number field.

[0082] TCN, through dilated convolution, can amplify the receptive field, thereby efficiently extracting local features and capturing short-term transient fluctuations in the input. TCN can compensate for the shortcomings of LSTM in capturing local information, making the model more sensitive to short-term dynamic changes. The one-dimensional convolution operation of TCN can be fitted with a general formula, which can be expressed as:

[0083] (8)

[0084] in, Indicates the output of TCN; Indicates the position of the LSTM output sequence. ; This represents the convolutional kernel weights of the TCN, used for local feature extraction; The activation function is represented by the ReLU function in this embodiment to increase nonlinearity. The kernel size of the TCN represents the number of consecutive signal points considered in each convolution operation. In this embodiment, The value of is 3; This represents the bias of the TCN, the translation parameter used for compensation; Indicates in At time t, the hidden state vector output by the LSTM; This represents the dilation coefficient, used to control the sampling stride of the convolution kernel, thereby expanding the receptive field. In this embodiment, The value is 2. In this embodiment, the channel data of the TCN convolutional layer output feature map is set to 64.

[0085] Local features extracted by TCN After global aggregation, it is mapped to a single scalar output through a fully connected layer. This output is used to correct the base thermal power calculated by the physical model of motor thermal power. The specific formula is as follows:

[0086] (9)

[0087] in, This indicates the predicted thermal power correction value; and These represent the weights and biases of the fully connected layer, respectively.

[0088] Step 1.3: Train the LSTM-TCN neural network model using the sample dataset to obtain the thermal power correction prediction model.

[0089] The thermal power correction prediction model can be used to predict the thermal power correction value, and then the motor thermal power at any time can be calculated according to formula (3). The calculation of motor thermal power is realized by using the physical model of motor thermal power and the thermal power correction prediction model. It not only utilizes the prior information provided by the physical model, but also captures complex nonlinear and temporal characteristics through deep learning, thereby improving the prediction accuracy of motor thermal power.

[0090] Step 2: Use the policy network (Actor) to output the rotational speed adjustment vector based on the initial state vector.

[0091] In blood pump control systems, to minimize temperature rise while meeting physiological requirements such as blood flow, temperature control, and pulsation, precise rotational speed planning is necessary based on the blood pump's operating cycle. This invention employs a parameterization method to discretize the operating cycle, dividing the entire cycle into N discrete time points, and using these discrete parameters to construct a rotational speed parameter vector. :

[0092] (10)

[0093] in, Let represent the motor speed at the i-th discrete time, with a value ranging from 3000 to 9000 rpm. To achieve smooth speed control, a continuous speed function can be constructed using linear or spline interpolation methods: This parametric design fully reflects the optimization needs throughout the entire working cycle, ensuring smooth speed changes while providing an adjustable set of parameters for subsequent online optimization.

[0094] To achieve intelligent optimization of motor speed regulation during blood pump operation, this invention proposes a motor speed control based on reinforcement learning. This control method is based on state perception and reward feedback, and automatically learns the optimal long-term speed scheduling strategy to ensure that the cumulative heat energy within the cycle is minimized while meeting the physiological requirements of blood pump output flow (i.e., blood flow), temperature control and pulse pressure difference.

[0095] The reinforcement learning method of this invention employs a policy network-value network algorithm. The task of the policy network (Actor) is to output actions based on the state vector, that is, the adjustment value of the discrete speed parameter vector (i.e., the speed adjustment vector); the value network (Critic) is used to evaluate the expected cumulative reward that the system can obtain in the future from the current state.

[0096] Suppose the entire working cycle is divided into N discrete moments, and the rotational speed parameter vector is... The rotational speed adjustment vector output by the policy network in one operation is:

[0097] (11)

[0098] in, This represents the adjustment amount for the i-th speed, which is usually limited to a certain range (e.g., ±500 rpm).

[0099] The policy network in this embodiment adopts a feedforward multilayer neural network structure, and its design parameters include:

[0100] State vector dimension :Right now The dimension of this embodiment It equals 6;

[0101] Hidden layer dimensions The number of nodes in the hidden layer of the policy network, in this embodiment Equals 50;

[0102] Hidden layers The number of hidden layers in the policy network, in this embodiment It equals 5;

[0103] Output Dimension : equals the number of discrete speed planning time points, i.e., the dimension of the speed adjustment vector, in this embodiment It equals 20.

[0104] The optimization objective of the policy network is to maximize the probability of dominant actions (actions that minimize heat output), so its loss function... Defined as:

[0105] (12)

[0106] (13)

[0107] in, The loss function of the policy network is represented. This represents the parameters of the policy network, namely the weights and biases of the hidden layers; This represents the prediction reward error of the value network, specifically the prediction reward error at n time points, indicating the rotational speed adjustment vector. Advantages; Represents the state vector Lower output speed adjustment vector The probability of; Represents the speed adjustment vector; This represents the sum of the cumulative discounted reward calculated based on the immediate rewards at n time points and the estimated reward at the next Nn time points; Represents the state vector of the value network at time t. The output is the estimated reward from time t to time t+N, which is the estimated reward of the value network from time t to the end of the cycle.

[0108] If the current action Make A negative value indicates that the estimated reward for the current action is higher than the actual reward. In this case, the vector is adjusted by increasing the rotation speed. The probability of being selected decreases if the initial state vector is input to the policy network on the controller, and the policy network outputs the speed adjustment vector. .

[0109] Step 3: Adjust the vector according to the rotational speed Control the motor speed.

[0110] Step 4: Obtain the current key parameters of the motor, blood pump output flow, blood temperature, and aortic pressure difference.

[0111] Key parameters of the motor include speed and accumulated heat energy. The controller adjusts the speed vector based on the output of the strategy network. The speed of the blood pump motor is controlled, and an interaction request is initiated between the thermal power prediction model (including the physical model of motor thermal power and the thermal power correction prediction model) and the cardiovascular dynamics model. The thermal power prediction model feeds back the motor thermal power at the current moment to the controller, and then calculates the cumulative thermal energy at the current moment according to formula (2). The speed sensor feeds back the current motor speed to the controller. The cardiovascular dynamics model feeds back the current blood pump output flow rate to the controller. Blood temperature and aortic pressure gradient Alternatively, the above parameters can be directly measured through animal experiments to replace cardiovascular dynamics model simulation and achieve feedback to the controller.

[0112] Step 5: Generate the current state vector (as shown in formula (1)) based on the current key parameters of the motor, the blood pump output flow, the blood temperature and the aortic pressure difference, and calculate the immediate reward.

[0113] To guide the policy network in learning a rotation speed scheduling strategy that satisfies both clinical safety and energy efficiency, this invention designs a multi-objective fusion immediate reward function. The immediate reward function scores the current action at each time step (i.e., time step) as the immediate reward in reinforcement learning, used to construct the discounted cumulative reward or discounted cumulative reward over the period. The expression for the immediate reward function is:

[0114] (14)

[0115] in, Indicates the immediate reward at the current moment; This represents the difference in motor thermal power between the current moment and the previous moment, i.e. , This represents the motor's thermal power at the current moment. This represents the motor's thermal power at the previous moment; the lower the thermal power consumption, the higher the reward. , and These represent the penalty factors for the corresponding items. In this embodiment, the penalty factor... , and For trainable parameters, their lower bounds are all limited to 1. ; This indicates the blood pump output flow rate at the current moment. and These represent the minimum and maximum output flow rates of the blood pump, respectively. This indicates the current blood temperature. Indicates the upper limit of the safe temperature; This represents the aortic pressure difference at the current moment. and These represent the minimum and maximum values ​​of the aortic pressure gradient, respectively. This symbol represents a conditional judgment. It takes the value 1 when the condition is met and 0 when the condition is not met.

[0116] The second term in formula (14) is a constraint term for the blood pump output flow rate: if If the threshold is exceeded, a penalty is imposed, with a value of 1. The second term is... The third item is the upper temperature limit constraint (thermal damage protection): if If the temperature exceeds the upper limit of the safe temperature range (39℃ in this embodiment), a penalty is imposed with a value of 1. The third item is... The fourth item is the pulse pressure difference constraint (cardiac load regulation): if If the risk exceeds the clinically permissible safety range, a penalty will be imposed, with a value of 1. The third item is... .

[0117] Step 6: Based on the state vector at the current moment, repeat steps 2 to 5 n-1 times to obtain the speed adjustment vector, state vector and instant reward at n moments.

[0118] The policy network is used to output a speed adjustment vector based on the current state vector, and then control the motor speed. The key parameters of the motor, blood pump output flow, blood temperature and aortic pressure difference at the next moment are obtained. Then, the state vector of the next moment is generated based on the key parameters of the motor, blood pump output flow, blood temperature and aortic pressure difference at the next moment, and the instant reward is calculated. This process is repeated n-1 times, so that the speed adjustment vector, state vector and instant reward at n moments can be obtained.

[0119] Step 7: Update the value network based on the state vectors at n time points and the immediate reward.

[0120] To optimize the action selection direction output by the policy network, a value network is designed to evaluate the expected cumulative reward, or expected cumulative reward, that the system can obtain in the future from its current state. During training, the value network provides a benchmark for the policy network, gradually learning the "reward capacity" corresponding to each state by minimizing the difference between its estimated reward and the actual reward. Unlike the policy network, the value network outputs not an action, but a scalar reward value. This indicates the level of the current "reward". The higher the value, the more "promising" the corresponding state is. For the purpose of this invention, that is, to minimize the thermal power consumption of the motor while satisfying multiple constraints such as blood pump output flow rate, blood temperature and aortic pressure difference.

[0121] Since it is impossible to wait for the system to complete its entire cycle before calculating the true reward each time, this invention adopts a compromise solution—an n-step reward. This solution combines the immediate reward (discounted cumulative reward) from the first n time steps with the estimated reward from the next (Nn) time steps to define the true reward. :

[0122] (15)

[0123] in, The sum of the discounted cumulative reward calculated based on the immediate rewards at n times and the estimated rewards at the next Nn times is the actual reward. The discount factor is used to emphasize the importance of adjacent moments and weaken the impact of moments that are far apart on the reward. In this embodiment, the discount factor is set to 0.8. This represents the immediate reward at time t+k, which is the actual reward at the previous t+k times. This indicates that the estimated reward from time t+n to time t+N, i.e., the estimated value of the subsequent cumulative reward, is determined by the value network and is considered constant, not participating in the backpropagation of the neural network. From formula (15), it can be seen that... It equals the sum of the cumulative discounted reward calculated based on the immediate rewards at n times and the estimated reward from time t+n to time t+N by the value network.

[0124] The learning objective of a value network is to output an estimated reward. The goal is to approximate the mathematical expectation of the actual reward as closely as possible. Since it's impossible to wait for the system to complete its entire cycle before calculating the actual reward each time, this invention aims to... As the true reward throughout the entire cycle, the loss function of the value network can be defined as:

[0125] (16)

[0126] in, The loss function represents the value network. These represent the parameters of the value network. The loss function estimates the reward. As close as possible The desired outcome is to reduce the variance in policy network updates. In this embodiment, the value network and policy network use the same network structure, the only difference being the output dimension. Before training, the parameters of the policy network and value network are randomly initialized, with a period length of 1 second, discrete time N set to 20, and a discount factor... Set it to 0.8.

[0127] After accumulating the speed adjustment vector, state vector, and immediate reward for every n time moments, the true reward is constructed according to formula (15), and then the loss value of the value network is calculated according to formula (16). The parameters of the value network are then updated based on this loss value. In this embodiment, the gradient update method is used to update the parameters of the value network:

[0128] (17)

[0129] in, This represents the learning rate of the value network, with an initial value set to 1e. -3 ; This indicates the direction of gradient updates in the value network.

[0130] Step 8: Based on the predicted reward error of the value network Update the policy network.

[0131] The prediction reward error of the value network is calculated according to formula (13). Then, according to formula (12), the dominant direction of the action probability is maximized, and the parameters of the policy network are updated by gradient:

[0132] (18)

[0133] in, This represents the learning rate of the policy network, with an initial value set to 3e. -4 ; This represents the gradient of the action with respect to the parameters of the policy network.

[0134] Step 9: Repeat steps 2 to 8 based on the new starting state vector until the entire cycle has been traversed.

[0135] After each update (or training) is completed, the sliding window slides one position forward on the periodic sequence (i.e., N time points), using the new time point as the starting point to generate a new initial state vector. Steps 2 to 8 are repeated until the entire period has been traversed, thus completing the traversal of N time points. For example, if the starting time of the previous update or training was the i-th time point (the initial state vector was...), then... If the starting time of the current update or training is the (i+1)th time (the initial state vector is...), then the starting time of the current update or training is the (i+1)th time (the initial state vector is...). Repeat steps 2 through 8 until the entire cycle is completed.

[0136] During the training phase, the controller is not limited to sampling from the beginning of the cycle. Instead, it adopts a sliding window strategy, using the state vector at any time as the starting state vector to construct a sliding sample trajectory and sampling the state-action sequence for n consecutive time moments. This strategy can effectively cover the state changes within the complete cycle N, which helps to capture local differences and long-term dependency features, and achieve full learning of the state-action mapping. Thus, during the deployment phase, it can dynamically match the patient's heart rate and physiological changes.

[0137] Step 10: Repeat steps 1 to 9 multiple times until the termination condition is met, and obtain the trained policy network.

[0138] Repeat steps 1 through 9 multiple times to complete multiple cycles of updates or training. The termination condition in this embodiment is that the value network output is stable, the reward converges, and the control policy output by the policy network tends to be optimal.

[0139] The pulsed blood pump thermal management control system based on speed modulation of the present invention is also configured to execute the pulsed blood pump motor speed control step. For example... Figure 2 As shown, the steps for controlling the speed of the pulsed blood pump motor include:

[0140] Step 11: Obtain key motor parameters at the start time ( , ), blood pump output flow Blood temperature and aortic pressure gradient .

[0141] Step 12: Based on the key motor parameters at the start time ( , ), blood pump output flow Blood temperature and aortic pressure gradient Generate the state vector at the start time. As shown in formula (1).

[0142] Step 13: Utilize the trained policy network based on the initial state vector Output speed adjustment vector.

[0143] The trained policy network is deployed on the controller, using the initial state vector. As input to the trained policy network, the trained policy network directly outputs all speed adjustments within the current cycle.

[0144] Step 14: Adjust the motor speed according to the speed adjustment vector.

[0145] The controller adjusts the speed according to the speed adjustment vector, executing speed adjustments sequentially at each time step, eliminating the need for sliding sampling or reward estimation. Therefore, only the policy network is retained during deployment, and the value network is not used. The controller operates using single-cycle inference, characterized by fast response, low resource overhead, and strong adaptability, making it suitable for real-time blood pump speed regulation tasks. The parameters of the policy network are fixed after deployment and are no longer updated.

[0146] The above description only discloses specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or modifications that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A pulsed blood pump thermal management control system based on speed modulation, characterized in that, The control system is configured to perform the following steps: Step 1: Generate the initial state vector based on the key parameters of the motor, the blood pump output flow, the blood temperature, and the aortic pressure difference at historical moments; Step 2: Utilize the strategy network to output the rotational speed adjustment vector based on the initial state vector; Step 3: Adjust the vector control of the motor speed according to the rotational speed; Step 4: Obtain the current key parameters of the motor, blood pump output flow, blood temperature, and aortic pressure differential; Step 5: Generate the current state vector based on the current key parameters of the motor, the blood pump output flow, the blood temperature, and the aortic pressure difference, and calculate the immediate reward; Step 6: Based on the state vector at the current moment, repeat steps 2 to 5 n-1 times to obtain the speed adjustment vector, state vector and instant reward at n moments; Step 7: Update the value network based on the state vectors at n time points and the immediate reward; Step 8: Update the policy network based on the prediction reward error of the value network; Step 9: Repeat steps 2 to 8 based on the new starting state vector until the entire cycle has been traversed; Step 10: Repeat steps 1 to 9 multiple times until the termination condition is met to obtain the trained policy network; The key parameters of the motor include speed and cumulative thermal energy. The cumulative thermal energy is calculated based on the motor's thermal power, which is equal to the sum of the base thermal power and the thermal power correction value.

2. The pulsed blood pump thermal management control system based on speed modulation according to claim 1, characterized in that, The basic thermal power is calculated using a physical model of motor thermal power, and the expression of the physical model of motor thermal power is as follows: ； in, Indicates the basic thermal power of the motor; Indicates electromagnetic torque. Represents the torque constant. Indicates the stator winding resistance; This represents the core hysteresis loss coefficient. Indicates the operating frequency. denoted by , where m represents the magnetic flux density and represents the exponent of hysteresis loss in the core loss. This describes the material parameters that describe the eddy current losses in the iron core; Indicates the motor speed. This represents the empirical coefficient related to the first-order term of the rotational speed. This represents the empirical coefficient related to the quadratic term of the rotational speed, and t represents time.

3. The pulsed blood pump thermal management control system based on speed modulation according to claim 1, characterized in that, The thermal power correction value is obtained by predicting the motor operating indicators using a thermal power correction prediction model; The process of obtaining the thermal power correction prediction model includes: Construct a sample dataset; wherein each sample in the sample dataset includes an input quantity and an output quantity, the input quantity being a motor operating index, and the output quantity being a thermal power correction value, the thermal power correction value being equal to the difference between the actual measured thermal power value and the baseline thermal power; Construct an LSTM-TCN neural network model; The LSTM-TCN neural network model is trained using the sample dataset to obtain the thermal power correction prediction model.

4. The pulsed blood pump thermal management control system based on speed modulation according to claim 3, characterized in that, The output of the LSTM-TCN neural network model is: ； ； ； in, This indicates the predicted thermal power correction value; and These represent the weights and biases of the fully connected layer, respectively. Indicates the output of TCN; Indicates the position of the LSTM output sequence. H represents the hidden layer dimension of the LSTM; Indicates the kernel weights of the TCN; Indicates the activation function; Indicates the kernel size of the TCN; Indicates the bias of TCN; Indicates in At time t, the hidden state vector output by the LSTM is... Indicates the coefficient of thermal expansion; Let k represent the LSTM input sequence, and k represent the number of elements in the LSTM input sequence.

5. The pulsed blood pump thermal management control system based on speed modulation according to claim 1, characterized in that, The formula for calculating the instant reward is as follows: ； in, Indicates the immediate reward at the current moment; This represents the difference in motor thermal power between the current moment and the previous moment. , This represents the motor's thermal power at the current moment. This represents the motor's thermal power at the previous moment; , and These represent the penalty factors for the corresponding terms; This indicates the blood pump output flow rate at the current moment. and These represent the minimum and maximum output flow rates of the blood pump, respectively. This indicates the current blood temperature. Indicates the upper limit of the safe temperature; This represents the aortic pressure difference at the current moment. and These represent the minimum and maximum values ​​of the aortic pressure gradient, respectively. This symbol represents a conditional judgment. It takes the value 1 when the condition is met and 0 when the condition is not met.

6. The pulsed blood pump thermal management control system based on speed modulation according to claim 1, characterized in that, The loss function of the value network is: ； ； in, The loss function represents the value network. The parameters represent the value network; N represents the number of moments in the entire cycle. Represents the state vector of the value network at time t. Output the estimated reward from time t to time t+N; This represents the sum of the cumulative discounted reward calculated based on the immediate rewards at n time points and the estimated reward at the next Nn time points; Indicates the discount factor; Represents the immediate reward at time t+k; This represents the estimated reward from time t+n to time t+N, determined by the value network.

7. The pulsed blood pump thermal management control system based on speed modulation according to claim 1, characterized in that, The loss function of the policy network is: ； ； in, The loss function of the policy network is represented. Represents the parameters of the policy network; This represents the error in predicting rewards within the value network. Represents the state vector Lower output speed adjustment vector The probability of; Represents the speed adjustment vector; This represents the sum of the cumulative discounted reward calculated based on the immediate rewards at n time points and the estimated reward at the next Nn time points; Represents the state vector of the value network at time t. Output the estimated reward from time t to time t+N.

8. The pulsed blood pump thermal management control system based on speed modulation according to claim 1, characterized in that, The control system is also configured to perform the following steps: Acquire key parameters of the motor, blood pump output flow, blood temperature, and aortic pressure differential at the start time; The state vector at the start time is generated based on the key parameters of the motor, the blood pump output flow, the blood temperature, and the aortic pressure difference. The trained policy network outputs a rotation speed adjustment vector based on the state vector at the initial moment. Motor speed is controlled by adjusting the speed vector.

Images