Kidney replacement treatment strategy recommendation method based on normal distribution comprehensive evaluation

Abstract

The invention discloses a kidney replacement treatment strategy recommendation method based on normal distribution comprehensive evaluation, and belongs to the field of medical big data and artificial intelligence, and the method comprises the following steps: S1, preprocessing historical data, and obtaining a patient data matrix; s2, screening a to-be-evaluated matrix according to a missing rate theta-majority principle; s3, performing filling by utilizing matrix decomposition; s4, performing scoring by adopting a CNS; s5, establishing a Q value model by taking a score difference of adjacent moments as a reward function, taking an RRT working mode as an action and taking physical sign monitoring as a state; s6, training the model to obtain an optimal strategy and a worst strategy of the current round; s7, carrying out hyperrho ep treatment scheme and secondary pep treatment scheme labeling; s8, calculating a comprehensive comparison matrix, and screening out a global optimal strategy; s9, repeating the steps S6 to S8 until a termination criterion is met; and S10, realizing patient kidney replacement treatment strategy recommendation. According to the method, the optimal AI strategy recommendation can be provided in the statistical sense, and medical staff can be assisted to take effective measures.

Description

Technical Field

[0001] The present invention relates to a method for recommending a renal replacement therapy strategy based on a comprehensive evaluation of normal distribution, belongs to the field of medical big data and artificial intelligence, and is particularly suitable for recommending a renal replacement therapy strategy based on a comprehensive evaluation of normal distribution. Background Art

[0002] The rapid development of artificial intelligence (AI) and machine learning (ML) technologies has brought revolutionary impacts to the medical field. These technologies have made significant progress in medical image classification, biomolecular structure analysis, genomics, automatic diagnosis, and rehabilitation outcome prediction. Renal replacement therapy (RRT), as a vital life support technology, has become an important treatment method for millions of patients with kidney disease worldwide. It is estimated that about 2.284 million people worldwide are at risk of premature death due to lack of RRT. With the development of technology and the increase in medical needs, it is estimated that by 2030, the number of patients using RRT worldwide will exceed 5.4 million. The Clinical Practice Guidelines for the Evaluation and Management of Chronic Kidney Disease (KidneyDisease: Improving Global Outcomes, KDIGO) have begun to focus on the potential applications of AI technology and recognize its great potential in improving patient outcomes. Specifically, the application of artificial intelligence in the field of renal ultrasound is particularly significant, including automatic segmentation of the kidney, volume measurement, function prediction, and disease diagnosis. Looking ahead to renal replacement therapy, especially the application of continuous renal replacement therapy (CRRT), AI technology has begun to show its research potential in the fields of predicting CRRT needs, predicting mortality and renal function recovery, identifying risk factors, dynamic monitoring and predicting complications. Despite this, most of the AI-related innovative research on CRRT is still in the design or early verification stage.

[0003] Choosing the appropriate RRT mode, such as CRRT, intermittent renal replacement therapy (IRRT), etc., will directly affect the patient's treatment effect, cost, comfort, and even the possibility of kidney recovery. Compared with IRRT, CRRT is more likely to provide a better chance of renal function recovery, which will have a significant impact on the patient's quality of life and medical expenses, and can effectively reduce dependence on long-term dialysis. In addition, in the treatment of severe acute kidney injury (AKI) in critically ill patients, the choice of RRT mode is even more important, which will directly affect key outcomes such as mortality and renal function recovery.

[0004] Traditional patient health assessment methods usually only use partial statistical information and physiological indicators, such as Sequential Organ Failure Assessment (SOFA) score, Acute Physiology and Chronic Health Evaluation II (APACHE II) score, Simplified Acute Physiology Score II (SAPS II), etc. These methods each focus on different evaluation dimensions and are difficult to be used directly to comprehensively evaluate the health status of patients corresponding to the RRT mode under AI control. At the same time, it is worth noting that medical and health data usually have quality problems such as high noise and missing data. The traditional methods for dealing with such problems are mainly data cleaning and various interpolations. Although the common data cleaning method of deleting data exceeding a certain threshold is simple, it can easily lead to the loss of key features of the data, and common interpolation methods are difficult to meet the accurate extraction of features.

[0005] Therefore, there is currently a lack of an efficient, real-time, and accurate evaluation standard and intelligent recommendation method for renal replacement therapy strategies, especially for situations such as missing data. Summary of the invention

[0006] In view of this, the present invention provides a method for recommending renal replacement therapy strategies based on comprehensive evaluation of normal distribution, which aims to first reconstruct missing data using matrix decomposition technology, and then construct a reinforcement learning reward function using comprehensive normalization score (CNS), and then directly evaluate the changes in the dynamic health status of patients. Finally, a multi-index evaluation system of a comprehensive comparison matrix method is used to more accurately identify the "best" and "worst" AI strategies, realize renal replacement therapy strategy recommendation, and ultimately achieve intelligent recommendation of renal replacement therapy strategies as well as the timeliness and accuracy of early warning.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] Recommendation method for renal replacement therapy strategy based on comprehensive evaluation of normal distribution, combined with Figure 1 , characterized in that the method comprises the following steps:

[0009] S1: Preprocess the vital sign monitoring according to patients, characteristics and time series to obtain the patient data matrix;

[0010] S2: Filter the patient matrix to be evaluated according to the θ-majority principle of missing rate for each feature in the statistical patient data matrix;

[0011] S3: Use matrix decomposition method to fill missing values ​​in the patient evaluation matrix;

[0012] S4: Use the CNS scoring method to score the filled matrix for evaluation at each moment, and incorporate the score at that moment into the patient's time series data as a new feature;

[0013] S5: The difference in CNS scores at adjacent moments is used as the reward function, the RRT working mode is the action, and the patient characteristics are used as the state to establish a Q-value model;

[0014] S6: Input the patient's time series data into the Q value model for training, obtain the best strategy and the worst strategy for the current round by comparing the Q values, and record the Q value model parameters;

[0015] S7: Calculate the positive incentive rate ρ of each patient in the current round ep , according to ρ ep The patient time series data are marked as superρ ep Treatment options and times ep Treatment options;

[0016] S8: Traverse the best strategy, the worst strategy and the corresponding super ρ selected in all rounds ep , times epLabel the treatment plans, calculate the comprehensive comparison matrix for each batch, and compare and select the global best strategy and the global worst strategy;

[0017] S9: Repeat steps S6 to S8 until the Q-value model meets the η-optimal termination criterion;

[0018] S10: Use the Q-value model parameters corresponding to the global optimal strategy and the global worst strategy to build a Q-value model to recommend renal replacement therapy strategies for patients.

[0019] Further, the preprocessing described in step S1 is:

[0020] S101: Screening the patient's vital sign monitoring according to the unique identification ID;

[0021] S102: Sort the data of each patient in time sequence and perform normalization processing;

[0022] S103: Construct the time series data of each patient into a matrix M according to the behavioral characteristics and the columns as time, wherein unknown elements are set as missing values.

[0023] Further, the specific calculation in step S2 is:

[0024] S201: Calculate the missing rate of each feature in the patient's time series data, that is, missing rate = number of missing values / total number of time series data;

[0025] S202: According to the θ-majority principle, the rows with missing rate less than θ in the matrix M are extracted row by row and reconstructed into the matrix to be evaluated M c , where θ is the threshold hyperparameter.

[0026] Further, the matrix decomposition method described in step S3 is specifically:

[0027] (1) According to the matrix M to be evaluated c The dimensions of the matrix are randomly generated, and W; where M c is an m×n dimensional matrix, F and W are m×r and r×n matrices respectively, and r is the hyperparameter to be learned;

[0028] (2) By iteratively updating, and W, until M * =F·W and the matrix to be evaluated M c The error of the elements at the corresponding positions of the known elements in is less than the threshold requirement;

[0029] (3) According to M * For the evaluation matrix M c Fill in the corresponding elements at the positions of unknown elements.

[0030] Preferably, the error in step (2) is the matrix M to be evaluated c All known elements and corresponding positions M * The first-norm, second-norm, or a linear combination of the element-wise differences.

[0031] The CNS scoring method is specifically:

[0032] S401: Obtaining the medically defined normal value reference interval corresponding to each feature in the matrix to be evaluated;

[0033] S402: For any feature, assuming that the normal value reference interval is [l, u], use statistical methods to calculate the baseline interval [min, max] of the feature; where l and u are the lower bound and upper bound respectively, and min and max are the maximum value and minimum value of the feature in the matrix to be evaluated respectively;

[0034] S403: If any value of l or u is negative, both the reference interval and the baseline interval are shifted, otherwise, step S404 is executed; the shift operation is specifically: l=l+|min|, u=u+|min|, min=0, max=max+|min|;

[0035] S404: Use a power function to geometrically scale the reference interval and the baseline interval; specifically, calculate a real number k such that:

[0036] l′=l k , u′=u k , min′=min k , max′=max k ;

[0037] Where u′-max′=min′-l′;

[0038] S405: The centers of the reference interval and the baseline interval are translated to the origin respectively, specifically:

[0039]

[0040] S406: Using the hyperbolic sine function and Normalize the reference interval and the baseline interval; the normalization operation is specifically as follows:

[0041] If max s >3·u s , then the normalization is:

[0042] l * =tanh α (l s ),u * =tanhα (u s ), min*=tanh α (min s ), max * =tanh α (max s );

[0043] Otherwise, normalization is:

[0044] l * =sinh α (l s ),u * =sinh α (u s ), min * =sinh α (min s ), max * =sinh α (max s );

[0045] Among them, α is a hyperparameter;

[0046] S407: Standard normal distribution reference interval and baseline interval; specifically:

[0047]

[0048] S408: For any element v corresponding to the feature in the matrix to be evaluated, the corresponding standard normal distribution is calculated according to the normalization branch of S406: or

[0049] S409: Calculate the feature normalized score vector (d r， d m， d e ), where d r is the ratio of the number of elements of the feature that are not in the reference interval to all elements of the feature in the matrix to be evaluated, d m and d e are all v not in the interval [-1, 1] * The absolute mean and covariance of

[0050] S4010: Calculate CNS score according to the standardized score vector: CNS(vs)=0.6d r +0.3d m +0.1d e , where vs is the calculated d r , d m and d e The corresponding features.

[0051] Further, the CNS score difference of the states at adjacent times in step S5 is: t =CNS(s t )-CNS(s t+1 ), the Q value model is a Double Deep Q Network (DDQN) model, which is composed of a Q network, a target network and an experience buffer pool; the Q network input is the current state s in the experience buffer pool t and the current action α t , the output is the predicted Q value Q(s t , a t ; θ), where θ is the Q network training parameter; get the next state s from the experience buffer pool t+1 , input the Q network to obtain the Q values ​​obtained by taking each action in this state, take the action a′ corresponding to the largest Q value among all Q values, and compare it with the next state s in the experience buffer pool t +1 are input into the target network to obtain the optimal Q value Q(s t+1 , a′; θ′), where θ′ is the target network training parameter; the time series data in the data set is converted into (s t , a t , r t ,s t+1 ) in the form of experience stored in the experience buffer pool; during training, a batch of indexes will be randomly given, and then the experience corresponding to the index will be extracted from the experience buffer pool; where the state s t Corresponding to the vital sign monitoring data at time t, action a t Corresponding to the RRT working mode at time t, the reward r t Corresponding to the state s at time t t With status s t+1 The cxSOFA score is poor, status s t+1 Corresponding to the state s at time t t Execute action a t The next state after the transition.

[0052] Furthermore, the network update process of DDQN is divided into the following parts:

[0053] [1]. Network parameter synchronization: Update the parameters θ of the Q network to the parameters θ′ of the target network. A soft update method is adopted, that is, an update rate T is introduced, and the weighted average value of the Q network parameters and the target network parameters is assigned to the target network, that is, θ′=τ*θ+(1-τ)θ′;

[0054] [2]. Gradient update: Update the hidden layer parameters of the Q network and use the MSE algorithm to calculate the loss function L = (r t +γ·Q(s t+1 ,a′;θ′)-Q(s t , a t ;θ)) 2 ; Among them, r t is the reward function value at time t, that is, the difference in cxSOFA scores between time t and time t-1; γ is the discount factor and is a hyperparameter.

[0055] Furthermore, the Q value model described in step S5 is a Deep Q network (DQN) model, which is composed of a Q network, a target network and an experience buffer pool; the Q network is an updated network, which is trained to generate the best state and action value by predicting the Q value, and its input is the current state s in the experience buffer pool. t and the current action a t , the output is the predicted Q value Q(s t , a t ; θ), where θ is the Q network training parameter; the target network obtains the next state s from the experience buffer pool t+1 , and calculate the best Q value maxQ(s) from all actions a′ executed in this state t+1 , a′; θ′), where θ′ is the Q network training parameter; the experience buffer pool will (s t , a t , r t ,s t+1 ) is stored as experience. During training, a batch of indexes will be randomly given, and then the experience corresponding to the index will be extracted from the experience buffer pool. t Corresponding to the physical sign monitoring at time t as the state, action a t Corresponding to the RRT working mode at time t.

[0056] Furthermore, the update process of the DQN network is divided into the following parts:

[0057] [1] Network parameter synchronization: The parameters θ of the Q network are updated to the parameters θ′ of the target network. The update method adopted is soft update, that is, an update rate τ is introduced, and the value of the weighted average of the Q network parameters and the target network parameters is assigned to the target network. The specific update formula is: θ′=τ*θ+(1-τ)θ′.

[0058] [2] Gradient update: Update the hidden layer parameters of the Q network and use the MSE algorithm to calculate the loss function L = (r t +γ·maxQ(s t+1,a′;θ′)-Q(s t , a t ;θ)) 2 ; Among them, r t is the reward function value at time t, that is, the score difference between time t and time t-1; γ is the discount factor, which is a hyperparameter.

[0059] Furthermore, the Q-value model described in step S5 is a Behavior Cloning with Q-values ​​(BCQ) model, which is composed of a Q network, a target network, a behavior clone, and an experience replay pool; the Q network is an updated network, which is trained to generate the best state and action value by predicting the Q value, and its input is the current state s in the experience buffer pool t and the current action a t , the output is the predicted Q value Q(s t , a t ; θ), where θ is the Q network training parameter; the target network obtains the next state s from the experience buffer pool t+1 , and calculate the best Q value maxQ(s) from all actions α′ executed in this state t+1 , a′; θ′), where θ′ is the Q network training parameter; the experience buffer pool will (s t , a t , r t ,s t+1 ) is stored as experience. During training, a batch of indexes will be randomly given, and then the experience corresponding to the index will be extracted from the experience buffer pool; the behavior cloning is to learn action selection by imitating expert strategies, that is, directly learn the mapping from state to action without trial and error. t Corresponding to the physical sign monitoring at time t as the state, action a t Corresponding to the RRT working mode at time t.

[0060] Furthermore, the update process of the DQN network is divided into the following parts:

[0061] [1] Network parameter synchronization: The parameters θ of the Q network are updated to the parameters θ′ of the target network. The update method adopted is soft update, that is, an update rate τ is introduced, and the value of the weighted average of the Q network parameters and the target network parameters is assigned to the target network. The specific update formula is: θ′=τ*θ+(1-τ)θ′.

[0062] [2] Gradient update: Update the hidden layer parameters of the Q network and use the MSE algorithm to calculate the loss function L = (r t +γ·maxQ(s t+1 ,a′;θ′)-Q(s t , at ;θ)) 2 ; Among them, r t is the reward function value at time t, that is, the score difference between time t and time t-1; γ is the discount factor, which is a hyperparameter.

[0063] Further, the comparison of Q values ​​in step S6 obtains the best strategy and the worst strategy for the current round, specifically: for any state s, calculate the output Q value under state s and action a, that is, Q(s, a), according to and Select the strategy with the largest and smallest Q value, that is, the best strategy π * , the worst strategy π′, where A is the action space.

[0064] Further, the step S7 is specifically as follows:

[0065] S701: Count the number of positive excitations N of the patient's time series data in the current round g , that is, r t The number of states ≥ 0;

[0066] S702: Calculate positive excitation rate: ρ ep =N g / n; where n is the total number of states of the patient time series data in the current round;

[0067] S703: Set a threshold τ∈[0.5,1] as the positive excitation rate threshold. When ρ ep When ≥τ, the patient time series data at the corresponding time is called superρ ep Treatment plan, adding P B Otherwise, it is called sub-ρ ep Treatment plan, adding P G middle.

[0068] Further, in combination with Table 1 and Table 2, the step S8 is specifically as follows:

[0069] S801: Calculate vital signs monitoring data t In the optimal strategy π * Action of π * (s t ) and the super ρ at time t eep Treatment plana t Action similarity = in, idx(*) represents the subscript of *, n is the total number of states of the patient time series data in the current round, p∈P G , P G is the set of time series data with good healing effect in the current round, and P G The number of time series data included is denoted as |P G |;

[0070] S802: Calculate vital signs monitoring data t In the optimal strategy π * Action of π * (s t ) and the times ρ at time t ep Treatment plana t Action similarity = p∈P B , P B is the set of time series data with poor healing effect in the current round, and P B The number of time series data included is denoted as |P B |;

[0071] S803: Calculate vital sign monitoring data t In the worst strategy π′, the action π′(s t ) and the super ρ at time t ep Treatment plana t Action similarity =

[0072] S804: Calculate vital signs monitoring data t In the worst strategy π′, the action π′(s t ) and the times ρ at time t ep Treatment plana t Action similarity =

[0073] S805: Calculate the difference between the best and the worst ° =OG-OB, inferior extreme difference Ξ w =WB-WG, comprehensive confidence and the combined bias μ = (OG-WG)-(wB-OB);

[0074] S806: Traverse all rounds, for any two rounds where i≠j, if o (π i )≥Ξ o (π j ) and w (π i )≥Ξ w (π j ), we assume that strategy π i Compared with strategy π j Otherwise, if σ(π i )≥σ(π j ), and also believe that strategy π i Compared with strategy π j To be better, the best global strategy is The global worst strategy is π′ i .

[0075] Preferably, the method for selecting the global best strategy described in step S806 can introduce a temporary variable to store the global best strategy, and only compare the current round with the temporary variable to determine the global best strategy.

[0076] Furthermore, the η-optimal termination criterion described in step S9 is specifically: set a hyperparameter η≥1, if the global best strategy and the global worst strategy selected by step S8 are obtained in the most recent η rounds, then continue the training of the next round of Q value model; otherwise, terminate the training.

[0077] The step S10 specifically includes: selecting corresponding Q-value model parameters to construct a Q-value model according to the global best strategy and the global worst strategy corresponding to when the termination criterion is reached, thereby achieving a recommendation for a renal replacement therapy strategy.

[0078]

[0079] Beneficial effects of the present invention: The present invention provides a method for recommending renal replacement therapy strategies based on reinforcement learning. First, the missing values ​​in the data set are filled by matrix decomposition, and then a CNS scoring method is proposed to design an appropriate reward function to make the strategy, and the η-optimal termination criterion is used to determine the convergence of the Q-value model. At the same time, a comprehensive comparison matrix is ​​introduced to accurately evaluate the Q-value model, and the optimal network parameters are intelligently screened out, ensuring the timeliness and accuracy of the model output strategy. The method of the present invention abandons the stereotyped model based on the SOFA scoring mechanism, and gives the optimal AI strategy recommendation in a statistical sense, which can more intelligently assist medical personnel in taking effective treatment measures. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to illustrate the purpose and technical solution of the present invention, the present invention provides the following drawings for illustration:

[0081] Figure 1 It is a flowchart of the method of the present invention;

[0082] Figure 2 This is a DDQN model architecture diagram of an embodiment of the present invention;

[0083] Figure 3 This is a comparison chart of hospitalization duration according to an embodiment of the present invention;

[0084] Figure 4 A comparison chart of mortality rates according to an embodiment of the present invention;

[0085] Figure 5 It is the comprehensive comparison matrix result of the embodiment of the present invention. DETAILED DESCRIPTION

[0086] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0087] The accompanying drawings are only used for illustrative explanations and are only schematic diagrams, and should not be construed as limitations of the present invention.

[0088] Example 1: MIMIC-IV (a large public medical database) https: / / physionet.org / content / mimiciv / 2.2 / ) and eICU( https: / / physionet.org / content / eicu-crd-demo / 2.0.1 / ) is used to organize the patient data disclosed by the patient, which includes: the patient's unique ID, age, PaO2 / FiO2 oxygenation index, Vaso value, weight, total bilirubin, ALT, albumin, UO24, AST, etc., a total of 51 features. In order to better assist medical staff to improve the actual treatment of RRT, this embodiment proposes a renal replacement therapy strategy recommendation method based on reinforcement learning, which includes the following steps:

[0089] Step 1: Preprocess the vital sign monitoring according to patients, characteristics and time series to obtain the patient data matrix.

[0090] The pre-processing is as follows:

[0091] S101: Screening the patient's vital sign monitoring according to the unique identification ID;

[0092] S102: Sort the data of each patient in time sequence and perform normalization processing;

[0093] S103: construct the time series data of each patient into a matrix M according to the behavioral characteristics and the columns as time, wherein unknown elements are set as missing values.

[0094] Step 2: According to the missing rate of each feature in the statistical patient data matrix, as shown in Table 3, the θ-majority principle is used to screen the patient matrix to be evaluated.

[0095] The specific calculation described in step 2 is:

[0096] S201: Calculate the missing rate of each feature in the patient's time series data, that is, missing rate = number of missing values / total number of time series data;

[0097] S202: According to the θ-majority principle, the rows with missing rate less than θ in the matrix M are extracted row by row and reconstructed into the matrix to be evaluated M c , where θ is the threshold hyperparameter; it is recommended that θ ≥ 0.4.

[0098] Table 3 Missing characteristic elements

[0099]

[0100] Step 3: Use matrix decomposition method to fill missing values ​​in the patient evaluation matrix.

[0101] The matrix decomposition method described in step 3 is specifically:

[0102] (1) According to the matrix M to be evaluated c The dimensions of the matrix are randomly generated, and W; where M c is an m×n dimensional matrix, F and W are m×r and r×n matrices respectively, and r is the hyperparameter to be learned;

[0103] (2) By iteratively updating, and W, until M * =F·W and the matrix to be evaluated M c The error of the elements at the corresponding positions of the known elements in is less than the threshold requirement;

[0104] (3) According to M * For the evaluation matrix M c Fill in the corresponding elements at the positions of unknown elements.

[0105] Preferably, the error in step 2 is the matrix M to be evaluated c All known elements and corresponding positions M * The first-norm, second-norm, or a linear combination of the element-wise differences.

[0106] Step 4: Use the CNS scoring method to score the filled matrix for evaluation at each moment, and incorporate the score at that moment into the patient's time series data as a new feature.

[0107] The CNS score calculation process is as follows:

[0108] S401: Obtaining the medically defined normal value reference interval corresponding to each feature in the matrix to be evaluated;

[0109] S402: Select any scorable feature, assume that the normal value reference interval is [l, u], and use statistical methods to calculate the baseline interval [min, max] of the feature; where l and u are the lower and upper bounds, respectively, and min and max are the maximum and minimum values ​​of the feature in the matrix to be evaluated, respectively, as shown in Table 4, where min0 and max 0 is the maximum and minimum value of the feature in the matrix to be evaluated before filling, min * and max * To fill the maximum and minimum values ​​of the feature in the matrix to be evaluated, out is the sum of the maximum and minimum values ​​before and after filling, @ indicates that the feature does not need matrix decomposition for recovery and does not perform CNS score calculation. * Indicates that there is no normal reference interval.

[0110] Table 4. Distribution range of characteristics

[0111]

[0112] S403: If any value of l or u is negative, both the reference interval and the baseline interval are shifted, otherwise, step S404 is executed; the shift operation is specifically: l=l+|min|, u=u+|min|, min=0, max=max+|min|;

[0113] S404: Use a power function to geometrically scale the reference interval and the baseline interval; specifically, calculate a real number k such that:

[0114] l′=l k , u′=u k , min′=min k , max′=max k ;

[0115] Where u′-max′=min′-l′;

[0116] S405: The centers of the reference interval and the baseline interval are translated to the origin respectively, specifically:

[0117]

[0118] S406: Using the hyperbolic sine function and Normalize the reference interval and the baseline interval; the normalization operation is specifically as follows:

[0119] If max s >3·u s , then the normalization is:

[0120] l * =tanh α (ls),u * =tanh a (u s ), min * =tanh α (mins ), max * =tanh α (max s );

[0121] Otherwise, normalization is:

[0122] l * =sinh α (ls),u * =sinh α (u s ), min * =sinh α (min s ), max * =sinh α (max s );

[0123] Among them, α is a hyperparameter;

[0124] S407: Standard normal distribution reference interval and baseline interval; specifically:

[0125]

[0126] For example, if the reference interval of AST is [9, 40], and its baseline interval is [0, 4620.17], we first calculate a real number k = 0.126953 through the power function, and get

[0127] l′=1.322, u′=1.597, min′=0.0, max′=2.919,

[0128] Then, after translation in S405,

[0129] l s =-0.138,u s =0.138, min s =-1.460, max s =1.460;

[0130] Due to max s >3·u s , so the normalization is:

[0131] l * =tanh α (l s )=-0.137,u * =tanh α (u s )=0.137,

[0132] min *=tanh α (min s )=-0.898,max * =tanh α (max s )=0.898;

[0133] After standard normal distribution:

[0134] L=--1, U=1, MIN=-6.556, MAX=6.556;

[0135] S408: For any element v corresponding to the feature in the matrix to be evaluated, the corresponding standard normal distribution is calculated according to the normalization branch of S406: or

[0136] S409: Calculate the feature standardized score vector (d r , d m , d e ), where d r is the ratio of the number of elements of the feature that are not in the reference interval to all elements of the feature in the matrix to be evaluated, d m and d e are all v not in the interval [1, 1] * The absolute mean and covariance of

[0137] S4010: Calculate CNS score according to the standardized score vector: CNS(vs)=0.6d r +0.3d m +0.1d e , where vs is the calculated d r , d m and d e The corresponding features.

[0138] Step 5: Use the difference in CNS scores at adjacent moments as the reward function, the RRT working mode as the action, and the patient characteristics as the state to establish a Q-value model.

[0139] The CNS score difference between the states at adjacent moments is: t =CNS(s t )-CNS(s t+1 ), the Q value model is a Double Deep Q Network (DDQN) model, which is composed of a Q network, a target network and an experience buffer pool; the Q network input is the current state s in the experience buffer pool t and the current action a t , the output is the predicted Q value Q(s t , at ; θ), where θ is the Q network training parameter; get the next state s from the experience buffer pool t+1 , input the Q network to obtain the Q values ​​obtained by taking each action in this state, take the action a′ corresponding to the largest Q value among all Q values, and compare it with the next state s in the experience buffer pool t+1 Input the target network together to obtain the optimal Q value Q(s t+1 , a′; θ′), where θ′ is the target network training parameter; the time series data in the data set is converted into (s t , a t , r t ,s t+1 ) in the form of experience stored in the experience buffer pool; during training, a batch of indexes will be randomly given, and then the experience corresponding to the index will be extracted from the experience buffer pool; where the state s t Corresponding to the vital sign monitoring data at time t, action α t Corresponding to the RRT working mode at time t, the reward r t Corresponding to the state s at time t t With status s t+1 The cxSOFA score is poor, status s t+1 Corresponding to the state s at time t t Execute action a t The next state after the transition.

[0140] Furthermore, the network update process of DDQN is divided into the following parts:

[0141] (1) Network parameter synchronization: Update the parameters θ of the Q network to the parameters θ′ of the target network using a soft update method, that is, introduce an update rate τ, and assign the weighted average of the Q network parameters and the target network parameters to the target network, that is, θ′=τ*θ+(1-τ)θ′;

[0142] (2) Gradient update: Update the hidden layer parameters of the Q network and use the MSE algorithm to calculate the loss function L = (r t +γ·Q(s t+1 ,a′;θ′)-Q(s t , a t ;θ)) 2 ; Among them, r t is the reward function value at time t, that is, the difference in cxSOFA scores between time t and time t-1; γ is the discount factor and is a hyperparameter.

[0143] Step 6: Input the patient's time series data into the DDQN model for training. By comparing the Q values, the best and worst strategies for the current round are obtained, and the DDQN model parameters are recorded.

[0144] For any state s, calculate the output Q value under state s and action a, that is, Q(s, a), according to and Select the strategy with the largest and smallest Q value, that is, the best strategy π * , the worst strategy π′, where A is the action space.

[0145] Step 7: Calculate the positive incentive rate ρ of each patient in the current round ep , according to ρ ep The patient time series data are marked as superρ ep Treatment options and times ep Treatment options.

[0146] The positive incentive rate calculation method is as follows:

[0147] S701: Count the number of positive excitations N of the patient's time series data in the current round g , that is, r t The number of states ≥ 0;

[0148] S702: Calculate positive excitation rate: ρ ep =N g / n; where n is the total number of states of the patient time series data in the current round;

[0149] S703: Set a threshold τ∈[0.5, 1] ​​as the positive excitation rate threshold. When ρ ep When ≥τ, the patient time series data at the corresponding time is called superρ ep Treatment plan, adding P B Otherwise, it is called sub-ρ ep Treatment plan, adding P G middle.

[0150] Step 8: Traverse the best strategy, worst strategy and corresponding super ρ selected in all rounds ep , times ep The treatment plans are labeled, and the comprehensive comparison matrix of each batch is calculated to compare and screen out the global best strategy and the global worst strategy.

[0151] The process of combining the comprehensive comparison matrix to select the global best strategy and the worst strategy is specifically as follows:

[0152] S801: Calculate vital signs monitoring data t In the optimal strategy π * Action of π * (s t) and the super ρ at time t ep Treatment plana t Action similarity = in, idx(*) represents the subscript of *, n is the total number of states of the patient time series data in the current round, p∈P G , P G is the set of time series data with good healing effect in the current round, and P G The number of time series data included is denoted as |P G |;

[0153] S802: Calculate vital signs monitoring data t In the optimal strategy π * Action of π * (s t ) and the times ρ at time t ep Treatment plana t Action similarity = p∈P B , P B is the set of time series data with poor healing effect in the current round, and P B The number of time series data included is denoted as |P B |;

[0154] S803: Calculate vital sign monitoring data t In the worst strategy π′, the action π′(s t ) and the super ρ at time t ep Treatment plana t Action similarity =

[0155] S804: Calculate vital signs monitoring data t In the worst strategy π′, the action π′(s t ) and the times ρ at time t ep Treatment plana t Action similarity =

[0156] S805: Calculate the difference between the best and the worst ° =OG-OB, inferior extreme difference E w =WB-WG, comprehensive confidence and comprehensive bias μ = (OG-WG)-(WB-OB);

[0157] S806: Traverse all rounds, for any two rounds where i≠j, if ° (π i )≥Ξ ° (π j ) and E w (πi )≥Ξ w (π j ), we assume that strategy π i Compared with strategy π j Otherwise, if σ(π i )≥σ(π j ), and also believe that strategy π i Compared with strategy π j To be better, the best global strategy is The global worst strategy is π′ i .

[0158] Preferably, the method for selecting the global best strategy described in step S806 can introduce a temporary variable to store the global best strategy, and only compare the current round with the temporary variable to determine the global best strategy.

[0159] Step 9: Repeat steps S6 to S8 until the DDQN model meets the η optimal termination criterion.

[0160] The η-optimal termination criterion is: set a hyperparameter η≥1, if the global best strategy and the global worst strategy selected by step S6 are obtained in the most recent η rounds, then continue the next round of DDQN model training; otherwise, terminate the training.

[0161] Step 10: Use the DDQN model parameters corresponding to the global optimal strategy to build a DDQN model to recommend renal replacement therapy strategies for patients.

[0162] In order to compare the CNS score with the SOFA score, the difference between the CNS score and the SOFA score at adjacent moments was used as the reward function and applied to the DQN, DDQN, and BCQ algorithms. The CNS-DQN, CNS-DDQN, CNS-BCQ, SOFA-DQN, SOFA-DDQN, and SOFA-BCQ models were established. The CNS-DQN, CNS-DDQN, CNS-BCQ, SOFA-DQN, SOFA-DDQN, and SOFA-BCQ models were trained according to the above steps 1 to 7, respectively. The mortality rate and hospital stay of each group of patients were calculated, as shown in Figure 2 . Figure 3 , 4 As shown in the figure, the results show that the performance of the CNS model is always better than that of the SOFA model. The t-test was used to confirm that the results were statistically significant, and the alpha value was set to 0.05. If the p value is less than 0.05, there is a statistically significant difference between the means of the two groups. Figure 5 The p-values ​​of the best AI policies for DQN, DDQN, and BCQ are 0.029, 0.017, and 0.042, respectively. Figure 3The p-values ​​for the best AI strategies for DQN, DDQN, and BCQ were 0.004, 0.009, and 0.035, respectively. All p-values ​​were <0.05, indicating significant results. Among all models, CNS-BCQ performed best, reducing mortality from 20.2% to 13.2%, a 34.6% reduction, and reducing the average length of stay from 171 hours to 148.24 hours, a 22.8% reduction in average length of stay.

[0163] Table 5 Comprehensive comparison matrix results and mortality rate and average length of hospital stay

[0164]

[0165] Among them, OG, OB, WG, WB, σ, and μ are the indicators, comprehensive confidence, and comprehensive bias in the comprehensive comparison matrix, respectively; MoR is the mortality rate; and AiHs is the average length of hospital stay.

[0166] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. A method for recommending renal replacement therapy strategies based on comprehensive evaluation of normal distribution, characterized in that: The method comprises the following steps: S1: Preprocess the vital sign monitoring according to patients, characteristics and time series to obtain the patient data matrix; S2: Filter the patient matrix to be evaluated according to the θ-majority principle of missing rate for each feature in the statistical patient data matrix; S3: Use matrix decomposition method to fill missing values ​​in the patient evaluation matrix; S4: Use the CNS scoring method to score the filled matrix for evaluation at each moment, and incorporate the score at that moment into the patient's time series data as a new feature; S5: The difference in CNS scores at adjacent moments is used as the reward function, the RRT working mode is the action, and the patient characteristics are used as the state to establish a Q-value model; S6: Input the patient's time series data into the Q value model for training, obtain the best strategy and the worst strategy for the current round by comparing the Q values, and record the Q value model parameters; S7: Calculate the positive incentive rate ρ of each patient in the current round ep , according to ρ ep The patient time series data are marked as superρ ep Treatment options and times ep Treatment options; S8: Traverse the best strategy, the worst strategy and the corresponding super ρ selected in all rounds ep , times ep Label the treatment plans, calculate the comprehensive comparison matrix for each batch, and compare and select the global best strategy and the global worst strategy; S9: Repeat steps S6 to S8 until the Q-value model meets the η-optimal termination criterion; S10: Use the Q-value model parameters corresponding to the global best strategy and the global worst strategy to build a Q-value model to recommend a renal replacement therapy strategy for patients; The CNS scoring method described in step S2 is specifically: S401: Obtaining the medically defined normal value reference interval corresponding to each feature in the matrix to be evaluated; S402: For any feature, assuming that the normal value reference interval is [l, u], use statistical methods to calculate the baseline interval [min, max] of the feature; where l and u are the lower bound and upper bound respectively, and min and max are the maximum value and minimum value of the feature in the matrix to be evaluated respectively; S403: If any value of l or u is negative, both the reference interval and the baseline interval are shifted, otherwise, step S404 is executed; the shift operation is specifically: l=l+|min|, u=u+|min|, min=0, max=max+|min|; S404: Use a power function to geometrically scale the reference interval and the baseline interval; specifically, calculate a real number k such that: l′=l k ,u′u k ,min′=min k ,max′=max k ; Where u′-max′=min′-l′; S405: The centers of the reference interval and the baseline interval are translated to the origin respectively, specifically: S406: Using the hyperbolic sine function and Normalize the reference interval and the baseline interval; the normalization operation is specifically as follows: If max s >3·u s , then the normalization is: l * fishy α (l s ),u * fishy α (u s ), min * fishy α (min s ), max * fishy α (max s ); Otherwise, normalization is: l * birth a (l s ),u * birth α (u s ), min * birth α (min s ), max * birth α (max s ); Among them, α is a hyperparameter; S407: Standard normal distribution reference interval and baseline interval; specifically: U=1, MAX=max * / u * ; S408: For any element v corresponding to the feature in the matrix to be evaluated, the corresponding standard normal distribution is calculated according to the normalization branch of S406: or S409: Calculate the feature normalized score vector (d r , d m , d e ), where d r is the ratio of the number of elements of the feature that are not in the reference interval to all elements of the feature in the matrix to be evaluated, d m and d e are all v not in the interval [-1, 1] * The absolute mean and covariance of S4010: Calculate CNS score according to the standardized score vector: CNS(vs)=0.6d r +0.3d m +0.1d e , where vs is the calculated d r ,d m and d e The corresponding features; The Q value model is one of a DDQN model, a DQN model or a BCQ model; The CNS score difference between the states at adjacent times in step S5 is: t =CNS(s t )-CNS(s t+1 ).

2. The method for recommending renal replacement therapy strategies based on normal distribution comprehensive evaluation according to claim 1, characterized in that: The pre-processing described in step S1 is: S101: Screening the patient's vital sign monitoring according to the unique identification ID; S102: Sort the data of each patient in time sequence and perform normalization processing; S103: construct the time series data of each patient into a matrix M according to the behavioral characteristics and the columns as time, wherein unknown elements are set as missing values.

3. The method for recommending renal replacement therapy strategies based on normal distribution comprehensive evaluation according to claim 1, characterized in that: The preprocessing described in step S1 is: The specific calculation described in step S2 is: S201: Calculate the missing rate of each feature in the patient's time series data, that is, missing rate = number of missing values / total number of time series data; S202: According to the θ-majority principle, the rows with missing rate less than θ in the matrix M are extracted row by row and reconstructed into the matrix to be evaluated M c , where θ is the threshold hyperparameter.

4. The method for recommending renal replacement therapy strategies based on normal distribution comprehensive evaluation according to claim 1, characterized in that: The matrix decomposition method described in step S3 is specifically: (1) According to the matrix M to be evaluated c Two matrices F and W are randomly generated with dimensions of c is an m×n dimensional matrix, F and W are m×r and r×n matrices respectively, and r is the hyperparameter to be learned; (2) Update F and W iteratively until M * =F·W and the matrix to be evaluated M c The error of the elements at the corresponding positions of the known elements in is less than the threshold requirement; (3) According to M * For the evaluation matrix M c Fill in the corresponding elements at the positions of unknown elements.

5. The method for recommending renal replacement therapy strategies based on normal distribution comprehensive evaluation according to claim 4, characterized in that: The error in step (2) is the matrix M to be evaluated c All known elements and corresponding positions M * The first-norm, second-norm, or a linear combination of the element-wise differences.

6. The method for recommending renal replacement therapy strategies based on normal distribution comprehensive evaluation according to claim 1, characterized in that: The comparison of Q values ​​in step S6 obtains the best strategy and the worst strategy for the current round, specifically: for any state s, calculate the output Q value under state s and action a, i.e., Q(s, a), according to and Select the strategy with the largest and smallest Q value, that is, the best strategy π * , the worst strategy π′; where A is the action space.

7. The method for recommending renal replacement therapy strategies based on normal distribution comprehensive evaluation according to claim 1, characterized in that: Step S7 is specifically as follows: S701: Count the number of positive excitations N of the patient's time series data in the current round g , that is, r t The number of states ≥ 0; S702: Calculate positive excitation rate: ρ ep =N g / n; where n is the total number of states of the patient time series data in the current round; S703: Set a threshold τ∈[0.5,1] as the positive excitation rate threshold. When ρ ep ≥τ, add the patient time series data at the corresponding time to the super ρ ep The set of treatment options P G Otherwise, add to the sub-ρ ep The set of treatment options P B middle.

8. The method for recommending renal replacement therapy strategies based on normal distribution comprehensive evaluation according to claim 1, characterized in that: The step S8 is specifically as follows: S801: Calculate vital signs monitoring data t In the optimal strategy π * Action of π * (s t ) and the super ρ at time t ep Treatment plana t Action similarity = in, idx(*) represents the subscript of *, n is the total number of states of the patient time series data in the current round, p∈P G , P G For the current round of Super League ρ ep The collection of time series data of treatment plans, P G The number of time series data included is denoted as |P G |; S802: Calculate vital signs monitoring data t In the optimal strategy π * Action of π * (s t ) and the times ρ at time t ep Treatment plana t The action similarity of p∈P B , P B is the number ρ in the current round ep The collection of time series data of treatment plans, P B The number of time series data included is denoted as |P B |; S803: Calculate vital sign monitoring data t In the worst strategy π′, the action π′(s t ) and the super ρ at time t ep Treatment plana t The action similarity of S804: Calculate vital signs monitoring data t In the worst strategy π′, the action π′(s t ) and the times ρ at time t ep Treatment plana t The action similarity of S805: Calculate the difference between the best and the worst o =OG-OB, inferior extreme difference Ξ w =WB-WG, comprehensive confidence and the combined bias μ = (OG-WG)-(WB-OB); S806: Traverse all rounds, for any two rounds where i≠j, if o (π i )≥Ξ o (π j ) and w (π i )≥Ξ w (π j ), we assume that strategy π i Compared with strategy π j Otherwise, if σ(π i )≥σ(π j ), and also believe that strategy π i Compared with strategy π j To be better, the best global strategy is The global worst strategy is π′ i .

9. The method for recommending renal replacement therapy strategies based on normal distribution comprehensive evaluation according to claim 1, characterized in that: The η-optimal termination criterion described in step S9 is specifically: set a hyperparameter η≥1. If the global best strategy and the global worst strategy selected by step S8 are obtained in the most recent η rounds, continue the training of the next round of Q value model; otherwise, terminate the training.

10. The method for recommending renal replacement therapy strategies based on normal distribution comprehensive evaluation according to claim 8, characterized in that: The method for selecting the global best strategy described in step S806 can be achieved by introducing a temporary variable to store the global best strategy, and only comparing the current round with the temporary variable to determine the global best strategy.

Images