A method and system for blood glucose prediction and control

Abstract

The application provides a blood glucose prediction and control method and system, and relates to the technical field of pharmacokinetics mechanism, and the blood glucose prediction and control method comprises the following steps: obtaining a current time blood glucose state and an expected external intervention amount of target personnel according to a reinforcement learning model; inputting the current time blood glucose state and the expected external intervention amount into a reversible neural network in a forward direction to obtain a next time blood glucose state; inputting the current time blood glucose state and an expected blood glucose state into the reversible neural network in a reverse direction to obtain a reference external intervention amount required from the current time blood glucose state to the expected blood glucose state; adjusting the expected external intervention amount according to the reference external intervention amount; and when the next time blood glucose state reaches the expected blood glucose state, taking the adjusted expected external intervention amount as a final external intervention amount. The expected external intervention amount is adjusted through the reversible neural network, and the generalization ability of machine learning to fresh data such as frequently changing external intervention amounts is improved by counterfactual thinking.

Description

Technical Field

[0001] This invention relates to the field of pharmacokinetic mechanism technology, and more specifically, to a method and system for blood glucose prediction and control. Background Technology

[0002] In digital medicine, machine learning methods are often used to process the correlation between medical data. A certain data is used as the input of the machine learning model, and the model outputs the corresponding strategy or result. For example, based on external intervention data such as carbohydrate intake or insulin injection, the blood glucose changes in the target patient can be obtained.

[0003] In existing technologies, most machine learning models require training and testing data to maintain the assumption of independent and identically distributed data. Since these models are correlation-driven, they cannot distinguish between causality and spurious correlation in the data. Therefore, these models are particularly unstable in out-of-distribution generalization prediction, resulting in weak generalization ability and thus failing to make reasonable predictions about changes in unknown data. Summary of the Invention

[0004] The technical problem addressed by this invention is how to improve the generalization ability when making predictions based on medical data.

[0005] This invention provides a method for blood glucose prediction and control, comprising:

[0006] Based on the reinforcement learning model, the current blood glucose status of the target person and the expected amount of external intervention are obtained;

[0007] The current blood glucose status and the expected external intervention amount are positively input into the reversible neural network to obtain the blood glucose status at the next moment.

[0008] The current blood glucose state and the desired blood glucose state are inversely input into the reversible neural network to obtain the reference external intervention amount required to change the current blood glucose state to the desired blood glucose state;

[0009] The projected external intervention amount is adjusted based on the reference external intervention amount;

[0010] When the blood glucose level at the next moment reaches the desired blood glucose level, the adjusted expected external intervention amount will be used as the final external intervention amount.

[0011] Optionally, the reversible neural network includes multiple basic blocks, each of which has two complementary affine coupling layers;

[0012] The step of positively inputting the current blood glucose status and the expected external intervention amount into a reversible neural network to obtain the blood glucose status at the next moment includes:

[0013] The expected external intervention amount is divided into two parts of latent intervention states using the orthogonal weight standardization method.

[0014] The intervention hidden state is positively input into the affine coupling layer of the basic block, and the blood glucose state at the next moment is obtained through the positive reasoning of the affine coupling layer.

[0015] Optionally, the step of inversely inputting the current blood glucose state and the desired blood glucose state into the reversible neural network to obtain the reference external intervention amount required to change the current blood glucose state to the desired blood glucose state includes:

[0016] The current blood glucose state is divided into two latent blood glucose states using the orthogonal weight standardization method.

[0017] The latent blood glucose state is input in reverse to the affine coupling layer of the basic block, and the reference external intervention quantity is obtained through the reverse reasoning of the affine coupling layer.

[0018] Optionally, the basic block includes symmetric and asymmetric basic blocks. The step of forward inputting the intervention hidden state into the affine coupling layer of the basic block, and obtaining the next-time blood glucose state through forward reasoning of the affine coupling layer, includes:

[0019] The intervention hidden state is positively input into the affine coupling layer of the basic block;

[0020] The output vector of the affine coupling layer is obtained by performing forward inference using the symmetric basic blocks, the asymmetric basic blocks, and the forward inference expression.

[0021] The forward inference expression is:

[0022] l 1:d =a t W;

[0023] U 1:b =l 1:b ;

[0024] U b+1:d =l b+1:d +s 1:d-b ;

[0025] Among them, l 1:d and l b+1:d These are the two intervention hidden states described in the two parts, U 1:b and U b+1:d These refer to the two parts describing the latent blood glucose state. a t For the expected amount of external intervention, s 1:d-bThe blood glucose variable is the blood glucose state at the current moment after being affected by the expected amount of external intervention;

[0026] The blood glucose state at the next moment is obtained based on the blood glucose variable after the expected external intervention amount has been applied to the current blood glucose state.

[0027] Optionally, the step of inputting the latent blood glucose state back into the affine coupling layer of the basic block, and obtaining the reference external intervention quantity through back reasoning of the affine coupling layer, includes:

[0028] The hidden blood glucose state is input in reverse to the affine coupling layer of the basic block;

[0029] The output vector of the affine coupling layer is obtained by performing reverse reasoning using the symmetric basic block, the asymmetric basic block, and the reverse reasoning expression.

[0030] The reverse reasoning expression is:

[0031] l 1:b =U 1:b ;

[0032] l b+1:d =U b+1:d -s 1:d-b ;

[0033]

[0034] Among them, l 1:d and l b+1:d These are the two intervention hidden states described in the two parts, U 1:b and U b+1:d These refer to the two parts describing the latent blood glucose state. s 1:d-b The blood glucose variable is the result of the predicted external intervention amount affecting the current blood glucose status. The reference external intervention amount.

[0035] Optionally, adjusting the expected external intervention amount based on the reference external intervention amount includes:

[0036] Based on the reference external intervention amount, the expected external intervention amount is adjusted through a reinforcement learning model;

[0037] The expected external intervention amount is used as an action in the agent policy of the reinforcement learning model. The curiosity reward of the expected external intervention amount in the reinforcement learning model is obtained through the expected blood glucose state and the blood glucose state at the next moment.

[0038] The reinforcement learning model and the reversible neural network are trained based on the difference between the curiosity reward, the blood glucose state at the next moment, and the expected blood glucose state.

[0039] When the reversible neural network and the reinforcement learning model have been trained, the expected external intervention amount output by the trained reinforcement learning model is used as the adjusted expected external intervention amount.

[0040] Optionally, training the reinforcement learning model and the reversible neural network based on the difference between the curiosity reward, the blood glucose state at the next time step, and the expected blood glucose state includes:

[0041] Based on the difference between the blood glucose state at the next moment and the expected blood glucose state, the inverse loss function and the forward loss function of the reversible neural network are obtained.

[0042] Based on the inverse loss function, the forward loss function, and the curiosity reward, a comprehensive loss function is obtained;

[0043] The invertible neural network and the reinforcement learning model are trained based on the comprehensive loss function, the weight values ​​of the invertible neural network, and the weighting values ​​of the reinforcement learning model, wherein the weight values ​​are any values ​​in the range of 0 to 1, and the weighting values ​​are greater than 0.

[0044] When the comprehensive loss function reaches its minimum value, it is determined that the training of the invertible neural network and the reinforcement learning model is complete.

[0045] Optionally, obtaining the comprehensive loss function based on the inverse loss function, the forward loss function, and the curiosity reward includes:

[0046] The total reward is obtained by summing the curiosity reward and the external reward of the reinforcement learning model;

[0047] The comprehensive loss function is obtained based on the total reward, the inverse loss function, and the forward loss function;

[0048] The comprehensive loss function is:

[0049]

[0050] Where β is the weight value, λ is the weighted value, and r t Let be the sum of the rewards at time t. Let be the gradient loss of the proxy strategy.

[0051] Optionally, it further includes: constructing the reversible neural network, wherein constructing the reversible neural network includes:

[0052] Based on the causal graph of external intervention amount and blood glucose status and the Floyd algorithm, the topological order of the causal graph is obtained;

[0053] Based on the topological order, a hierarchical structure of the initial reversible neural network is obtained, wherein the hierarchical structure includes multiple layers, and the network nodes of each layer are no less than the network nodes of the previous layer;

[0054] Based on the input and output dimensions of the i-th layer of the initial reversible neural network, a preset dimension alternation process is performed to obtain the reversible neural network.

[0055] The present invention also provides a blood glucose prediction and control system, including a computer-readable storage medium storing a computer program and a processor, wherein the computer program is read and executed by the processor to implement any of the blood glucose prediction and control methods described above.

[0056] The blood glucose monitoring and prediction method and system of this invention obtains the blood glucose status of a target individual at the current moment and the expected external intervention amount through a reinforcement learning model. The external intervention amount can be external factors such as carbohydrate intake or insulin injection volume. The expected external intervention amount is used to predict the target individual's blood glucose status at the next moment, thus achieving blood glucose prediction. Simultaneously, to ensure the target individual's blood glucose status reaches the desired level, a reversible neural network performs causal inference based on the current blood glucose status and the desired blood glucose status to obtain the specific external intervention amount required for the process from the current blood glucose status to the desired blood glucose status, i.e., the reference external intervention amount. The expected external intervention amount is then adjusted using the reference external intervention amount. This allows for self-reflection on the expected external intervention amount before each iteration of the reinforcement learning network, reducing blind trial and error of the model. The expected blood glucose status is used to adjust the expected external intervention amount through the reversible neural network. By utilizing counterfactual thinking, the machine learning's ability to generalize to frequently changing external intervention amounts and other fresh data is improved. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating a blood glucose monitoring and prediction method in one embodiment of the present invention;

[0058] Figure 2 This is a schematic diagram of the reversible neural network structure of the blood glucose monitoring and prediction method in another embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Combination Figure 1 As shown, the present invention provides a method for blood glucose prediction and control, comprising:

[0061] S1: Based on the reinforcement learning model, obtain the target person's current blood glucose status and the expected amount of external intervention.

[0062] Specifically, the expected external intervention amount can be external intervention factors that can cause changes in blood glucose levels in the target person, such as the amount of carbohydrates required or the dose of insulin injected. After obtaining the target person's current blood glucose status, the target person's blood glucose status is used as the environment. Through a reinforcement learning model, the target person learns by maximizing the rewards provided by the environment to obtain a strategy to stabilize blood glucose within a preset range. The corresponding expected external intervention amount is obtained based on the strategy.

[0063] S2: Input the current blood glucose status and the expected external intervention amount into the reversible neural network to obtain the blood glucose status at the next moment.

[0064] Specifically, the current blood glucose status and the expected external intervention amount are positively input into the reversible neural network. The reversible neural network structure has causal awareness. This structure incorporates causal reasoning and counterfactual reasoning into a unified framework to realize the positive causal reasoning of the neural network. That is, based on the current blood glucose status and the expected external intervention amount, positive causal reasoning is used to obtain the blood glucose status at the next moment after the expected external intervention amount is received, thus realizing the prediction of blood glucose status.

[0065] S3: Input the current blood glucose state and the desired blood glucose state into the reversible neural network in reverse order to obtain the reference external intervention amount required to change the current blood glucose state to the desired blood glucose state.

[0066] Specifically, since the reversible neural network structure has the ability to reason backwards, the current blood glucose state and the expected blood glucose state are input backwards into the reversible neural network to perform counterfactual reasoning. The reversible neural network outputs the reference external intervention amount required to get from the blood glucose state at the next moment to the expected blood glucose state. This reference external intervention amount is the thinking of the reinforcement learning model before executing the expected external intervention amount, so as to achieve the unity of the thinking of the reversible neural network and the action of the reinforcement learning model.

[0067] S4: Adjust the expected external intervention amount based on the reference external intervention amount.

[0068] Specifically, after obtaining the reference external intervention amount, the expected external intervention amount obtained from the strategy of the reinforcement learning model can be optimized based on the training of the reversible neural network and the reinforcement learning model, so as to achieve precise control of blood glucose in the target personnel.

[0069] S5: When the blood glucose level at the next moment reaches the desired blood glucose level, the adjusted expected external intervention amount will be used as the final external intervention amount.

[0070] Specifically, the above process involves a thought process before taking action. This thought process, combined with the autonomous learning of the reinforcement learning model, signifies the completion of the thought process when the blood glucose level reaches the desired level at the next moment, at which point the desired goal can be achieved. The final external intervention output can then be used to control the blood glucose of the target individual, reducing the model's blind trial and error.

[0071] In a preferred embodiment of the present invention, during the forward reasoning process, the reversible framework of the reversible neural network inputs the blood glucose state s at time t. t and the expected amount of external intervention a t Output the predicted blood glucose level at the next moment. In formal terms, this is represented as:

[0072]

[0073] During the reverse inference process, the model inputs the blood glucose state s at time t. t And the expected blood glucose level at the next moment Output from s t Shift to the expected state The necessary reference external intervention amount Formal representation:

[0074]

[0075] The blood glucose monitoring and prediction method of this invention obtains the target person's blood glucose status at the current moment and the expected external intervention amount through a reinforcement learning model. The external intervention amount can be external factors such as carbohydrate intake or insulin injection amount. The expected external intervention amount is used to predict the target person's blood glucose status at the next moment, thereby achieving blood glucose prediction for the target person. At the same time, in order to make the target person's blood glucose status reach the desired blood glucose status, a reversible neural network performs causal inference based on the current blood glucose status and the desired blood glucose status to obtain the specific external intervention amount required for the process from the current blood glucose status to the desired blood glucose status, i.e., the reference external intervention amount. The expected external intervention amount is then adjusted by the reference external intervention amount. This achieves self-reflection on the expected external intervention amount before each iteration of the reinforcement learning network provides the expected external intervention amount, thereby reducing the model's blind trial and error. The expected blood glucose status is used to adjust the expected external intervention amount through the reversible neural network. By using counterfactual thinking, the machine learning's ability to generalize to fresh data such as frequently changing external intervention amounts is improved.

[0076] In this embodiment of the invention, the reversible neural network includes multiple basic blocks, each of which has two complementary affine coupling layers;

[0077] The step of positively inputting the current blood glucose status and the expected external intervention amount into a reversible neural network to obtain the blood glucose status at the next moment includes:

[0078] The expected external intervention amount is divided into two parts of latent intervention states using the orthogonal weight standardization method.

[0079] The intervention hidden state is positively input into the affine coupling layer of the basic block, and the blood glucose state at the next moment is obtained through the positive reasoning of the affine coupling layer.

[0080] In this embodiment, combined with Figure 2 As shown, the reversible neural network of this invention is composed of several basic blocks, which are further divided into symmetrical and asymmetrical forms. The asymmetrical blocks have the same overall structure as the symmetrical blocks, and each block is a reversible structure, following the architecture of a reversible neural network. When the reversible neural network performs forward inference, it first uses an orthogonal weight standardization method to reduce the expected external intervention amount a. t Decomposed into two parts of hidden state representation:

[0081] and l b+1:d =[a t W] b+1:d ∈R d-b ,in

[0082] Wherein, l represents b+1:dWith partial variables (s1,…,s) d-b Linear summation is performed on a to characterize it. t For (s1,…,s) d-b The linear causal effect of ) and another part represents l 1:b With linear interaction (s1,…,s) d-b ) are fed together into a standard affine coupling layer to characterize a t For (s1,…,s) d-b The nonlinear causal effect of the affine coupling layer in the basic block of this invention can be superimposed multiple times to simulate arbitrarily complex function transformations. The blood glucose state at the next moment is obtained through forward inference of the affine coupling layer. For asymmetric blocks, during forward inference, the joint latent representation of the expected external intervention and the directly affected state is received. 1:d and the remaining state vector s d-b+1:m Output the result vector of all state variables after external intervention (i.e., at time t+1). This will give you the blood sugar level at the next moment.

[0083] The blood glucose monitoring and prediction method of the present invention combines causal relationships between variables by using basic block splicing and neural network construction, thereby enabling blood glucose prediction based on causal relationships and improving the generalization ability of machine learning to fresh data such as frequently changing external interventions.

[0084] In this embodiment of the invention, the step of inversely inputting the current blood glucose state and the desired blood glucose state into the reversible neural network to obtain the reference external intervention amount required to change the blood glucose state from the current blood glucose state to the desired blood glucose state includes:

[0085] The current blood glucose state is divided into two latent blood glucose states using the orthogonal weight standardization method.

[0086] The latent blood glucose state is input in reverse to the affine coupling layer of the basic block, and the reference external intervention quantity is obtained through the reverse reasoning of the affine coupling layer.

[0087] In this embodiment, combined with Figure 2 As shown, traditional invertible neural networks can only handle cases where the input dimension and output dimension are the same, i.e., |A|. t |+|S t |=|S t+1 The basic block of this invention can use an affine coupling layer to adjust the dimension of the input information, enabling the model to handle any case where the input dimension is greater than or equal to the output dimension, i.e., during forward reasoning: |A t |+|S t |>|S t+1|, In reverse abductive reasoning: |S t |+|S t+1 |>|A t Based on this, the current blood glucose state is divided into two parts: the latent blood glucose state x of the reversible block. t ∈R D Divided into two halves, that is and By tracing back the cause through the affine coupling layer, the reference external intervention quantity is obtained.

[0088] In a preferred embodiment of the present invention, the current blood glucose state is divided into two parts: a latent blood glucose state and a reversible block. t ∈R D Divided into two halves, that is and Transformation coefficients exp(s) using affine functions i ) and t i For i∈{1,2}, use element-wise multiplication (⊙) and addition:

[0089]

[0090]

[0091]

[0092]

[0093] Where s and t can be Any complex function, given the next state These expressions are reversible:

[0094]

[0095]

[0096]

[0097]

[0098] In this process, s and t are implemented through several fully connected layers with LeakyReLU activation functions, and x is implemented through several consecutive basic blocks. t and x t+1 Any state transition between them.

[0099] The blood glucose monitoring and prediction method of this invention proposes a basic asymmetric form block whose structure is consistent with that of a symmetric form block. The key is that the affine coupling layer can be used to adjust the dimension of the input information, so that the model can handle any case where the input dimension is greater than or equal to the output dimension, thereby realizing reverse reasoning through the affine coupling layer.

[0100] In this embodiment of the invention, the basic block includes symmetric and asymmetric basic blocks. The step of forward inputting the intervention latent state into the affine coupling layer of the basic block, and obtaining the next-time blood glucose state through forward inference of the affine coupling layer, includes:

[0101] The intervention hidden state is positively input into the affine coupling layer of the basic block;

[0102] The output vector of the affine coupling layer is obtained by performing forward inference using the symmetric basic blocks, the asymmetric basic blocks, and the forward inference expression.

[0103] The forward inference expression is:

[0104] l 1:d =a t W;

[0105] U 1:b =l 1:b ;

[0106] U b+1:d =l b+1:d +s 1:d-b ;

[0107] Among them, l 1:d and l b+1:d These are the two intervention hidden states described in the two parts, U 1:b and U b+1:d These refer to the two parts describing the latent blood glucose state. a t For the expected amount of external intervention, s 1:d-b The blood glucose variable is the blood glucose state at the current moment after being affected by the expected amount of external intervention;

[0108] The blood glucose state at the next moment is obtained based on the blood glucose variable after the expected external intervention amount has been applied to the current blood glucose state.

[0109] In this embodiment, by adjusting the dimension of the input information through the aforementioned asymmetric basic blocks, the model can handle any case where the input dimension is greater than or equal to the output dimension, thereby achieving back-causation through the affine coupling layer. In the basic blocks of this invention, since the affine coupling layer can be stacked multiple times to simulate arbitrarily complex function transformations, U... 1:b and U b+1:dAfter being fed into a standard affine coupling layer, the output vector O can be obtained through two linear transformations. 1:d Used to characterize the state variable s 1:d-b Subject to external intervention t The hidden state representation after the action.

[0110] The blood glucose monitoring and prediction method of the present invention simulates arbitrarily complex function transformations by superimposing affine coupling layers multiple times, thereby realizing forward reasoning about blood glucose status and predicting blood glucose levels.

[0111] In this embodiment of the invention, the step of inputting the latent blood glucose state back into the affine coupling layer of the basic block, and obtaining the reference external intervention quantity through back inference of the affine coupling layer, includes:

[0112] The hidden blood glucose state is input in reverse to the affine coupling layer of the basic block;

[0113] The output vector of the affine coupling layer is obtained by performing reverse reasoning using the symmetric basic block, the asymmetric basic block, and the reverse reasoning expression.

[0114] The reverse reasoning expression is:

[0115] l 1:b =U 1:b ;

[0116] l b+1:d =U b+1:d -s 1:d-b ;

[0117]

[0118] Among them, l 1:d and l b+1:d These are the two intervention hidden states described in the two parts, U 1:b and U b+1:d These refer to the two parts describing the latent blood glucose state. s 1:d-b The blood glucose variable is the result of the predicted external intervention amount affecting the current blood glucose status. The reference external intervention amount.

[0119] In this embodiment, for asymmetric blocks, during forward reasoning, the joint hidden representation O of the expected external intervention amount and the directly affected blood glucose state is received. 1:d and the remaining state vector s d-b+1:m Output the vector of results for all blood glucose state variables after being subjected to the expected external intervention (i.e., at time t+1). During the reverse reasoning process, the hidden state representation O is received. 1:d and state variable s1:g-b Then, output the ideal external intervention amount. To achieve Figure 2 The reverse causation process is shown.

[0120] The blood glucose monitoring and prediction method of the present invention performs linear transformation through affine coupling layers in symmetric and asymmetric basic blocks, thereby achieving the purpose of introspection by obtaining reference external intervention quantities through reverse reasoning of affine coupling layers.

[0121] In this embodiment of the invention, adjusting the expected external intervention amount based on the reference external intervention amount includes:

[0122] Based on the reference external intervention amount, the expected external intervention amount is adjusted through the reinforcement learning model;

[0123] The expected external intervention amount is used as an action in the agent policy of the reinforcement learning model. The curiosity reward of the expected external intervention amount in the reinforcement learning model is obtained through the expected blood glucose state and the blood glucose state at the next moment.

[0124] The reinforcement learning model and the reversible neural network are trained based on the difference between the curiosity reward, the blood glucose state at the next moment, and the expected blood glucose state.

[0125] When the reversible neural network and the reinforcement learning model have been trained, the expected external intervention amount output by the trained reinforcement learning model is used as the adjusted expected external intervention amount.

[0126] In this embodiment, the expected external intervention is taken as the action from the agent policy of the reinforcement learning model. The prediction error of the agent policy's causal knowledge of its environment—that is, the difference between the expected blood glucose state and the blood glucose state at the next moment—is obtained by comparing the expected blood glucose state with the expected blood glucose state at the next moment. This prediction error is used to obtain the curiosity reward of the reinforcement learning model for the expected external intervention, and a curiosity-driven intrinsic reward signal is output. The curiosity reward is then used to jointly train the reversible neural network and the reinforcement learning model.

[0127] The acquisition of curiosity reward is achieved by stimulating the agent strategy's curiosity about the deviation between the expected blood glucose state and the blood glucose state at the next moment, thereby obtaining a curiosity reward. Based on the above idea, the curiosity reward can be formally represented as:

[0128]

[0129] Here, η>0 is a weighting factor. State transition function This aligns perfectly with the state predictions of invertible neural networks; therefore, combining... Figure 2 As shown, the proposed introspection module outputs an intrinsic curiosity reward that is obtained through forward reasoning using a causal reversible neural network.

[0130] The blood glucose monitoring and prediction method of this invention proposes a causal reinforcement learning approach, which performs counterfactual thinking before each action is given by the policy network, in order to reduce the model's blind trial and error.

[0131] In this embodiment of the invention, training the reinforcement learning model and the reversible neural network based on the difference between the curiosity reward, the blood glucose state at the next moment, and the expected blood glucose state includes:

[0132] Based on the difference between the blood glucose state at the next moment and the expected blood glucose state, the inverse loss function and the forward loss function of the reversible neural network are obtained.

[0133] Based on the inverse loss function, the forward loss function, and the curiosity reward, a comprehensive loss function is obtained;

[0134] The invertible neural network and the reinforcement learning model are trained based on the comprehensive loss function, the weight values ​​of the invertible neural network, and the weighting values ​​of the reinforcement learning model, wherein the weight values ​​are any values ​​in the range of 0 to 1, and the weighting values ​​are greater than 0.

[0135] When the comprehensive loss function reaches its minimum value, it is determined that the training of the invertible neural network and the reinforcement learning model is complete.

[0136] In this embodiment, the inverse loss function and the forward loss function of the reversible neural network are obtained based on the difference between the blood glucose state at the next time step and the expected blood glucose state. The agent strategy continuously interacts with the environment, and the parameters W of the reversible neural network are minimized by the loss function L. F After optimization, we can obtain:

[0137]

[0138] Given the current blood glucose level s t and the next moment's blood sugar status By reasoning in advance about the optimal action to achieve the desired goal, it's worth noting that the introspection of the surrogate strategy can be achieved using reverse abduction in a causal reversible neural network, i.e., the reverse reasoning process in the reversible neural network. (The reverse output of the reversible neural network is also mentioned.) It is the action a output by the policy network. t =π(s) t The prediction of θ) is therefore, Is performing action a t Prior to this, the ultimate training goal is to achieve unity between thought and action.

[0139] Regarding model training, in forward inference, external intervention a t and state s at time t t The hidden states of all causal effects are fed into a reversible neural network, and are used to predict the state s at time t+1. t+1 Conversely, using the same network structure and parameters, the state s at time t... t and expected state at time t+1 The input is fed into a reversible neural network to counterfactually infer the amount of external intervention 'a' required to reach the expected state. t Therefore, the loss function consists of two parts: forward prediction loss and reverse counterfactual loss, and is defined as a weighted sum of the forward and reverse losses:

[0140]

[0141] Where λ is the weight of the balancing loss. The loss is calculated as the shuffled mini-batch average, and the derivative of each parameter can be calculated via bidirectional backpropagation.

[0142] The blood glucose monitoring and prediction method of the present invention achieves joint training of reinforcement learning model and reversible neural network through introspection-driven exploration, and makes the function reach the minimum value by minimizing the loss function, thereby judging the training is complete and improving the ability to control blood glucose.

[0143] In this embodiment of the invention, obtaining the comprehensive loss function based on the inverse loss function, the forward loss function, and the curiosity reward includes:

[0144] The total reward is obtained by summing the curiosity reward and the external reward of the reinforcement learning model;

[0145] The comprehensive loss function is obtained based on the total reward, the inverse loss function, and the forward loss function;

[0146] The comprehensive loss function is:

[0147]

[0148] Where β is the weight value, λ is the weighted value, and r t Let be the sum of the rewards at time t. Let be the gradient loss of the proxy strategy.

[0149] In this embodiment, in addition to curiosity rewards, the agent strategy can also choose to receive some extrinsic rewards from the environment. Let the intrinsic curiosity reward generated by the agent strategy at time t be... External rewards are Intrinsic curiosity rewards will help enhance the agent's exploration ability when extrinsic rewards are sparse. The strategy is trained to maximize the sum of these two rewards.

[0150] For joint training of invertible neural networks and reinforcement learning models, the loss function is minimized:

[0151]

[0152] Among them, L I This is the inverse loss function, used to measure the difference between predicted and actual behavior. The overall optimization problem to be solved in learning the agent policy is a combination of the above loss functions, which can be expressed as:

[0153]

[0154] Where 0≤β≤1 is used to balance the inverse model loss and the forward model loss, and λ>0 is used to weight the importance of the policy gradient loss and the learning introspection-driven loss, where L I For the inverse loss function, L F For the positive loss function, r t It is the sum of curiosity rewards and external rewards, i.e., the total reward.

[0155] The blood glucose monitoring and prediction method of the present invention, by jointly training a reversible neural network and a reinforcement learning model, enables the reinforcement learning model to introspect before the agent policy executes an action and to perform counterfactual thinking before each action is given by the policy network, so as to reduce the model's blind trial and error.

[0156] In this embodiment of the invention, the method further includes: constructing the reversible neural network, wherein constructing the reversible neural network includes:

[0157] Based on the causal graph of external intervention amount and blood glucose status and the Floyd algorithm, the topological order of the causal graph is obtained;

[0158] Based on the topological order, a hierarchical structure of the initial reversible neural network is obtained, wherein the hierarchical structure includes multiple layers, and the network nodes of each layer are no less than the network nodes of the previous layer;

[0159] Based on the input and output dimensions of the i-th layer of the initial reversible neural network, a preset dimension alternation process is performed to obtain the reversible neural network.

[0160] In this embodiment, for the causal graph, the present invention can employ a generation algorithm based on a reversible neural network structure. For example, in a spatiotemporal causal graph with two external intervention variables and six observation variables, if the corresponding variable order is: a1, a2, s1, s2, s3, s6, s5, s4, since s1 and s2 are only directly affected by a1 and a2, while s3 and s6 are affected by the combined effects of a1, a2, s1, and s2, and s5 and s4 are affected indirectly by a1 and a2, the first basic block only reflects the direct causal effect of a1 and a2 on s1 and s2. Alternatively, s4 can be separated from the second basic block and set as a separate third basic block, but considering model complexity, the number of basic blocks is usually minimized as much as possible.

[0161] In a preferred embodiment of the present invention, as shown in Table 1, the algorithm for generating the reversible neural network structure uses the Floyd algorithm to solve for the topological order corresponding to the directed acyclic causal graph, takes the preceding variables as the upper half input based on the affine coupling layer, and applies causal effects to the following variables in the lower half.

[0162] Table 1

[0163]

[0164]

[0165] The blood glucose monitoring and prediction method of the present invention utilizes basic blocks to form a reversible neural network structure, which enables the reversible neural network structure to intuitively reflect the causal effects between variables.

[0166] The present invention also provides a blood glucose prediction and control system, including a computer-readable storage medium storing a computer program and a processor, wherein the computer program is read and executed by the processor to implement any of the blood glucose prediction and control methods described above.

[0167] The blood glucose monitoring and prediction method and system of this invention obtains the blood glucose status of a target individual at the current moment and the expected external intervention amount through a reinforcement learning model. The external intervention amount can be external factors such as carbohydrate intake or insulin injection volume. The expected external intervention amount is used to predict the target individual's blood glucose status at the next moment, thus achieving blood glucose prediction. Simultaneously, to ensure the target individual's blood glucose status reaches the desired level, a reversible neural network performs causal inference based on the current blood glucose status and the desired blood glucose status to obtain the specific external intervention amount required for the process from the current blood glucose status to the desired blood glucose status, i.e., the reference external intervention amount. The expected external intervention amount is then adjusted using the reference external intervention amount. This allows for self-reflection on the expected external intervention amount before each iteration of the reinforcement learning network, reducing blind trial and error of the model. The expected blood glucose status is used to adjust the expected external intervention amount through the reversible neural network. By utilizing counterfactual thinking, the machine learning's ability to generalize to frequently changing external intervention amounts and other fresh data is improved.

[0168] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0169] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0170] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for predicting and controlling blood glucose, characterized in that, include: Based on the reinforcement learning model, the current blood glucose status of the target person and the expected amount of external intervention are obtained; The current blood glucose status and the expected external intervention amount are positively input into the reversible neural network to obtain the blood glucose status at the next moment. The current blood glucose state and the desired blood glucose state are inversely input into the reversible neural network to obtain the reference external intervention amount required to change the current blood glucose state to the desired blood glucose state; The projected external intervention amount is adjusted based on the reference external intervention amount; When the blood glucose level at the next moment reaches the desired blood glucose level, the adjusted expected external intervention amount will be used as the final external intervention amount.

2. The blood glucose prediction and control method according to claim 1, characterized in that, The reversible neural network comprises multiple basic blocks, each of which has two complementary affine coupling layers; The step of positively inputting the current blood glucose status and the expected external intervention amount into a reversible neural network to obtain the blood glucose status at the next moment includes: The expected external intervention amount is divided into two parts of latent intervention states using the orthogonal weight standardization method. The intervention hidden state is positively input into the affine coupling layer of the basic block, and the blood glucose state at the next moment is obtained through the positive reasoning of the affine coupling layer.

3. The blood glucose prediction and control method according to claim 2, characterized in that, The step of inversely inputting the current blood glucose state and the desired blood glucose state into the reversible neural network to obtain the reference external intervention amount required to change the blood glucose state from the current blood glucose state to the desired blood glucose state includes: The current blood glucose state is divided into two latent blood glucose states using the orthogonal weight standardization method. The latent blood glucose state is input in reverse to the affine coupling layer of the basic block, and the reference external intervention quantity is obtained through the reverse reasoning of the affine coupling layer.

4. The blood glucose prediction and control method according to claim 3, characterized in that, The basic block includes symmetric and asymmetric basic blocks. The affine coupling layer of the basic block forward inputs the intervention latent state, and the blood glucose state at the next time step is obtained through forward inference of the affine coupling layer, including: The intervention hidden state is positively input into the affine coupling layer of the basic block; The output vector of the affine coupling layer is obtained by performing forward inference using the symmetric basic blocks, the asymmetric basic blocks, and the forward inference expression. The forward inference expression is: ； ； ； in, and These are the two parts describing the intervention hidden states. These refer to the two parts describing the latent blood glucose state. , The projected amount of external intervention, The blood glucose variable is the blood glucose state at the current moment after being affected by the expected amount of external intervention; The blood glucose state at the next moment is obtained based on the blood glucose variable after the expected external intervention amount has been applied to the current blood glucose state.

5. The blood glucose prediction and control method according to claim 4, characterized in that, The process of inputting the latent blood glucose state back into the affine coupling layer of the basic block, and obtaining the reference external intervention quantity through back inference from the affine coupling layer, includes: The hidden blood glucose state is input in reverse to the affine coupling layer of the basic block; The output vector of the affine coupling layer is obtained by performing reverse reasoning using the symmetric basic block, the asymmetric basic block, and the reverse reasoning expression. The reverse reasoning expression is: ； ； ； in, and These are the two parts describing the intervention hidden states. These refer to the two parts describing the latent blood glucose state. , The blood glucose variable is the result of the predicted external intervention amount affecting the current blood glucose status. The reference external intervention amount.

6. The blood glucose prediction and control method according to claim 1, characterized in that, The step of adjusting the expected external intervention amount based on the reference external intervention amount includes: Based on the reference external intervention amount, the expected external intervention amount is adjusted through the reinforcement learning model; The expected external intervention amount is used as an action in the agent policy of the reinforcement learning model. The curiosity reward of the expected external intervention amount in the reinforcement learning model is obtained through the expected blood glucose state and the blood glucose state at the next moment. The reinforcement learning model and the reversible neural network are trained based on the difference between the curiosity reward, the blood glucose state at the next moment, and the expected blood glucose state. When the reversible neural network and the reinforcement learning model have been trained, the expected external intervention amount output by the trained reinforcement learning model is used as the adjusted expected external intervention amount.

7. The blood glucose prediction and control method according to claim 6, characterized in that, The step of training the reinforcement learning model and the reversible neural network based on the difference between the curiosity reward, the blood glucose state at the next moment, and the expected blood glucose state includes: Based on the difference between the blood glucose state at the next moment and the expected blood glucose state, the inverse loss function and the forward loss function of the reversible neural network are obtained. Based on the inverse loss function, the forward loss function, and the curiosity reward, a comprehensive loss function is obtained; The invertible neural network and the reinforcement learning model are trained based on the comprehensive loss function, the weight values ​​of the invertible neural network, and the weighting values ​​of the reinforcement learning model, wherein the weight values ​​are any values ​​in the range of 0 to 1, and the weighting values ​​are greater than 0. When the comprehensive loss function reaches its minimum value, it is determined that the training of the invertible neural network and the reinforcement learning model is complete.

8. The blood glucose prediction and control method according to claim 7, characterized in that, The step of obtaining a comprehensive loss function based on the inverse loss function, the forward loss function, and the curiosity reward includes: The total reward is obtained by summing the curiosity reward and the external reward of the reinforcement learning model; The comprehensive loss function is obtained based on the total reward, the inverse loss function, and the forward loss function; The comprehensive loss function is: ； in, The weight value is... The weighted value, Let be the sum of the rewards at time t. The gradient loss of the proxy strategy is given. For the inverse loss function, This is the positive loss function.

9. The blood glucose prediction and control method according to claim 1, characterized in that, Also includes: Constructing the reversible neural network includes: Based on the causal graph of external intervention amount and blood glucose status and the Floyd algorithm, the topological order of the causal graph is obtained; Based on the topological order, a hierarchical structure of the initial reversible neural network is obtained, wherein the hierarchical structure includes multiple layers, and the network nodes of each layer are no less than the network nodes of the previous layer; Based on the input and output dimensions of the i-th layer of the initial reversible neural network, a preset dimension alternation process is performed to obtain the reversible neural network.

10. A blood glucose prediction and control system, characterized in that, The method includes a computer-readable storage medium storing a computer program, which is read and executed by the processor to implement the blood glucose prediction and control method as described in any one of claims 1-9.

Images