A zero-shot blood glucose prediction method based on a two-scale framework

Abstract

This invention discloses a zero-shot blood glucose prediction method based on a dual-scale Transformer framework, belonging to the field of diabetes prediction technology. The specific steps are as follows: Step 1: Preprocess the input univariate blood glucose time series to obtain local and global blood glucose time series; Step 2: The prediction model extracts high-resolution features from the local blood glucose time series and low-resolution features from the global blood glucose time series; Step 3: The prediction model fuses the high-resolution features from the local blood glucose time series and the low-resolution features from the global blood glucose time series. The prediction model introduces a Fishr regularization term and a weighted focusing loss value for joint optimization training until training converges. By matching the loss landscape between domains and within batches with the inter-domain gradient variance, the zero-shot generalization ability is improved. Relying solely on univariate CGMS sequences, the cost of clinical application is reduced.

Description

Technical Field

[0001] This invention relates to the field of diabetes prediction technology, and specifically to a zero-sample blood glucose prediction method based on a dual-scale Transformer framework. Background Technology

[0002] Diabetes mellitus is a metabolic disease characterized by chronic hyperglycemia and has become a major global public health challenge. Type 1 diabetes mellitus (T1DM) is caused by the destruction of pancreatic beta cells, leading to an absolute lack of insulin, requiring lifelong exogenous insulin therapy. Type 2 diabetes mellitus (T2DM), on the other hand, is characterized by insulin resistance and relative insulin deficiency, often accompanied by obesity and lifestyle factors. Both types of diabetes require strict glycemic management to reduce the risk of hypoglycemic and hyperglycemic events, as well as related cardiovascular, renal, and neurological complications.

[0003] Continuous glucose monitoring systems (CGMS) provide high-frequency, real-time blood glucose data, significantly improving the accuracy of patient blood glucose management. Accurate multi-step blood glucose prediction can provide early warning of abnormal fluctuations, offering a window for proactive intervention and further reducing the risk of complications, thus possessing significant clinical value. Existing blood glucose prediction methods face two main challenges: first, significant individual patient heterogeneity means existing prediction models are prone to overfitting patient-specific characteristics, resulting in insufficient generalization ability in zero-sample, cross-patient scenarios, hindering large-scale deployment; second, existing single-scale prediction model architectures cannot simultaneously account for local fine fluctuations (such as postprandial spikes) and global long-term trends (such as diurnal rhythms), making them susceptible to sensor noise interference or missing key acute events.

[0004] The development of blood glucose prediction algorithms has evolved from traditional statistical prediction models to machine learning. Early methods mostly used linear time series prediction models such as ARIMA, but these models are based on the assumption of stationary sequences, making them sensitive to non-stationarity of blood glucose and noise, resulting in limited prediction accuracy. Subsequently, classic machine learning algorithms such as Support Vector Machines (SVM) and Random Forests were applied to blood glucose prediction, but they rely on manual feature engineering and have insufficient generalization ability on long-term time series. In recent years, deep learning has significantly improved the performance of blood glucose prediction. Recurrent Neural Networks (RNNs) and Long Short-Term Memory Networks (LSTMs) adapt to nonlinear physiological dynamics by capturing temporal dependencies, but they are limited by gradient vanishing and computational efficiency issues. The Transformer architecture, due to its powerful ability to capture long-range dependencies and parallel computing efficiency, has been widely used in natural language processing and time series prediction tasks. Existing research has applied Transformers to blood glucose sequence modeling, such as GlucoseTransformer (a deep learning model designed for prediction of continuous blood glucose monitoring time-series data), Gluformer (GlucoseTransformer, a Transformer model specifically for blood glucose prediction), and Gpformer (Glucose PredictionTransformer, a Transformer model specifically for blood glucose prediction). On the one hand, Transformers, based on a self-attention mechanism, excel at processing sequential data and can perform parallel computation to capture long-term dependencies; on the other hand, their positional encoding mechanism helps to process data sequentially, compensating for the shortcomings of RNN (Recurrent Neural Network) prediction models.

[0005] Despite these advancements, existing blood glucose prediction algorithms still face two core challenges. First, the dilemma of capturing multi-scale temporal patterns: Existing Transformer methods typically employ a single attention scale, either over-focusing on local short-term fluctuations, easily overfitting to CGM (continuous glucose monitoring) sensor noise and measurement artifacts, making predictions sensitive to transient disturbances; or prioritizing global long-term dependencies, thus ignoring clinically critical local acute events. This trade-off between local accuracy and global stability makes the prediction trajectory unstable in different physiological scenarios such as postprandial, exercise, and sleep, with significant fluctuations in the time to residence (TIR) ​​of the target blood glucose range. Second, insufficient generalization across patient zero-shot scenarios: Most prediction models severely overfit patient-specific physiological characteristics during training, resulting in a sharp decline in performance in zero-shot scenarios with new patients. This domain shift problem is particularly prominent when transferring across datasets, diabetes types, or life scenarios, severely hindering the large-scale clinical application of the algorithms.

[0006] Based on this, the present invention designs a zero-sample blood glucose prediction method based on a dual-scale Transformer framework to solve the above problems. Summary of the Invention

[0007] To address the aforementioned shortcomings of existing technologies, this invention provides a zero-sample blood glucose prediction method based on a dual-scale Transformer framework.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A zero-shot blood glucose prediction method based on a dual-scale Transformer framework, the specific steps of which are as follows:

[0010] Step 1: Process the input univariate blood glucose time series Preprocessing was performed to obtain local blood glucose time series. and global blood glucose time series ;

[0011] Step 2: Prediction model extracts local time series of blood glucose. High-resolution features in and global blood glucose time series Low-resolution features in ;

[0012] Step 3: Predictive model for local time series of blood glucose High-resolution features in and global blood glucose time series Low-resolution features in Feature fusion is performed, and the prediction model is jointly optimized and trained by introducing Fishr (gradient variance invariant method) regularization term and weighted focusing loss value until training converges;

[0013] Step 4: Perform the same preprocessing operation as in Step 1 on the univariate blood glucose time series of the patients to be predicted, and input the processed data into the prediction model to obtain the prediction sequence.

[0014] Furthermore, the specific steps for step 1 are as follows:

[0015] Step 11: Use linear interpolation to analyze univariate blood glucose time series. The potential fill portion at the end is filled;

[0016] Step 12: Use two sets of sliding window parameters to fill in the univariate blood glucose time series. Segmentation generation and A series of consecutive time blocks, Combining consecutive time blocks yields local blood glucose time series. ,Will The global blood glucose time series is obtained by combining consecutive time blocks. .

[0017] Furthermore, step 2 is performed as follows:

[0018] Step 21: Local time series of blood glucose and global blood glucose time series The blood glucose local temporal embedding data and blood glucose global temporal embedding data are obtained by passing through the embedding layer in the prediction model, respectively, performing value projection, temporal feature embedding, location encoding and patient type embedding;

[0019] Step 22: Local temporal embedding data of blood glucose is fed into a 2-layer Transformer encoder stack to extract high-resolution features. The global temporal embedding data of blood glucose is input into a 4-layer Transformer encoder stack to extract low-resolution features. .

[0020] Furthermore, step 21 is performed as follows:

[0021] Step 211: Local time series of blood glucose and global blood glucose time series After passing through the embedding layer, the timestamp encoding first processes the local time sequence of blood glucose. and global blood glucose time series Average pooling is performed on minute, hour, and weekday features within the sequence block to obtain local time series of blood glucose. and global blood glucose time series The 3D temporal feature vector of the sequence block in the sequence;

[0022] Step 212: By a linear projection to Dimension, and add to the sequence block;

[0023] Step 213: Local time series analysis of blood glucose and global blood glucose time series The sequence block embeddings in the data use a learnable embedding layer encoding for diabetes subtypes, including local and global blood glucose temporal embeddings.

[0024] Furthermore, the specific steps in step 22 are as follows:

[0025] Step 221: In the self-attention mechanism, for the input representation The query, key, and value matrices are generated through three linear projections.

[0026] Step 222: Calculate the attention score:

[0027] Step 223: Apply the above operations in parallel according to the multi-head mechanism. Each output is a separate header, then the outputs are concatenated and projected back through another linear layer. Dimensions, to obtain local time series of blood glucose High-resolution features in and global blood glucose time series Low-resolution features in .

[0028] Furthermore, the specific steps for step 3 are as follows:

[0029] Step 31: Local time series of blood glucose High-resolution features in and global blood glucose time series Low-resolution features in The input is fed into a prediction model, where a learnable router dynamically fuses local blood glucose time series data. High-resolution features in and global blood glucose time series Low-resolution features in This yields the final predicted sequence;

[0030] Step 32: Calculate the weighted focusing loss value of the final predicted sequence;

[0031] Step 33: Calculate the Fishr loss value of the Fishr regularization term for the final predicted sequence;

[0032] Step 34: Calculate the total loss value using the weighted focusing loss value and the Fishr loss value;

[0033] Step 35: Perform backpropagation on the total loss, calculate the gradient, and update all learnable parameters in the prediction model using the optimizer;

[0034] Step 36: Repeat steps 31 to 35 to traverse the high-resolution features. and low-resolution features After multiple rounds, the change in total loss after each round is less than Alternatively, when the preset maximum number of training rounds is reached, training is stopped, the parameters of the best-performing prediction model are saved, and the prediction model is obtained.

[0035] Furthermore, the specific steps in step 31 are as follows:

[0036] Step 311: Transfer high-resolution features and low-resolution features By concatenating along the feature dimension, the concatenated features are obtained;

[0037] Step 312: Input the spliced ​​features into the linear layer. The linear layer projects the spliced ​​features into a 2D space and then uses the Softmax function to generate the fusion weights of each position for the two branches.

[0038] Step 313: Adjust the high-resolution features according to their weights and low-resolution features By performing fusion, fusion characteristics are obtained. ;

[0039] Step 314: Merge features The data is normalized by a normalization layer, then flattened, and the flattened vector is input into a linear layer and projected onto the length of a future time step to obtain the final prediction sequence.

[0040] Furthermore, the specific steps in step 32 are as follows:

[0041] Step 321: Calculate the fundamental root mean square error of the final predicted sequence. ;

[0042] Step 322: Divide continuous blood glucose values ​​into three categories according to clinical thresholds, and define the corresponding BIN value as the class-specific weighting coefficient α in subsequent focus loss;

[0043] Step 323: Calculate the confidence level of the probability that the predicted sequence falls within its true blood glucose range. ;

[0044] Step 324: Based on the root mean square error and probability confidence Calculate the weighted focusing loss value .

[0045] Furthermore, the specific steps in step 33 are as follows:

[0046] Step 331: Within each training batch, patients are sampled in equal batches;

[0047] Step 332: For the sampling data of each patient within the batch, calculate its specific task loss. The task loss is calculated through backpropagation. For all parameters of the prediction model gradient and gradient Flatten and concatenate them into a gradient vector;

[0048] Step 333: Calculate the mean of the gradient vectors for all patients. According to gradient and the mean of the gradient vectors of all patients Calculate the Fishr loss value.

[0049] Compared with the prior art, the beneficial effects of this invention are as follows: by using a dual-branch multi-resolution architecture and a layer-by-layer learnable router, multi-scale features are dynamically fused to balance the capture of local and global information; patient-level Fishr regularization is introduced, treating each patient as an independent domain, and loss landscapes are aligned by matching intra-batch gradient variance between domains, thereby improving zero-sample generalization ability; and relying solely on univariate CGMS sequences, thereby reducing the cost of clinical application. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0051] Figure 1 This is a schematic diagram of the overall process of a zero-sample blood glucose prediction method based on a dual-scale Transformer framework according to the present invention.

[0052] Figure 2 This is the RMSE graph of blood glucose prediction under different historical time series in this invention;

[0053] Figure 3 This is a graph showing the accuracy of blood glucose prediction under different historical time series according to the present invention;

[0054] Figure 4 This is a Clark error grid analysis diagram of the present invention on the Shanghai type 2 diabetes dataset;

[0055] Figure 5 This is a Clark error grid analysis diagram on the Ohio University Type 1 Diabetes Dataset, which is the basis of this invention.

[0056] Figure 6 This is a Clark error grid analysis diagram of the blood glucose replacement test dataset based on the present invention;

[0057] Figure 7 This is a box plot of the ablation experiment of this invention on the Shanghai type 2 diabetes dataset;

[0058] Figure 8 This is a box plot of the ablation experiment of the present invention on the type 1 diabetes dataset from Ohio University, USA.

[0059] Figure 9 This is a box plot of the ablation experiment of the present invention on the blood glucose replacement test dataset;

[0060] Figure 10This is a time-series prediction graph of the present invention under different ablation index models;

[0061] Figure 11 Box plot showing the comparison of this invention with other methods on the Shanghai type 2 diabetes dataset;

[0062] Figure 12 Box plot showing the comparison of this invention with other methods on the Ohio University Type 1 Diabetes Dataset;

[0063] Figure 13 This is a box plot comparing the present invention with other methods on a blood glucose replacement test dataset;

[0064] Figure 14 This is a comparison chart of the time series prediction values ​​of this invention and other methods. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0066] Example 1: In some embodiments, please refer to the accompanying drawings. Figure 1 A zero-shot blood glucose prediction method based on a two-scale Transformer framework is described below.

[0067] Step 1: Step 1: Input univariate blood glucose time series Preprocessing was performed to obtain local blood glucose time series. and global blood glucose time series ;

[0068] Step 1 is performed as follows:

[0069] Step 11: Use linear interpolation to analyze univariate blood glucose time series. The potential fill portion at the end is filled;

[0070]

[0071] Step 12: Use two sets of sliding window parameters to fill in the univariate blood glucose time series. Segmentation generation and A series of consecutive time blocks, Combining consecutive time blocks yields local blood glucose time series. ,Will The global blood glucose time series is obtained by combining consecutive time blocks. ;

[0072] and The calculation method is as follows:

[0073]

[0074]

[0075] Univariate blood glucose time series Length; Univariate blood glucose time series The number of data points added at the end; Local time series of blood glucose The length of the sliding window during generation; Local time series of blood glucose The step size of the sliding window during generation; Global blood glucose time series The length of the sliding window during generation; Global blood glucose time series The step size of the sliding window during generation. ;

[0076]

[0077]

[0078] N represents the number of patches in the local time series of blood glucose. N' represents the number of patches in the global blood glucose time series. ; Local time series of blood glucose The feature dimension of each patch (i.e.) (number of columns), global blood glucose time series The feature dimension of each patch (i.e.) (number of columns);

[0079] Compared to traditional transformers, this application naturally preserves local temporal semantics in each temporal block embedding, avoiding information loss caused by point-by-point processing of individual time steps. Furthermore, within a fixed length... Below, the computation and memory overhead of the attention map decreases from O( The squared value decreased to O( ),in ;

[0080] Step 2: Prediction model extracts local time series of blood glucose. High-resolution features in and global blood glucose time series Low-resolution features in ;

[0081] Step 2 is performed as follows:

[0082] Step 21: Local time series of blood glucose and global blood glucose time series The blood glucose local temporal embedding data and blood glucose global temporal embedding data are obtained by passing through the embedding layer in the prediction model, respectively, performing value projection, temporal feature embedding, location encoding and patient type embedding;

[0083] Step 21 is performed as follows:

[0084] Step 211: Local time series of blood glucose and global blood glucose time series After passing through the embedding layer, the timestamp encoding first processes the local time sequence of blood glucose. and global blood glucose time series Average pooling is performed on minute, hour, and weekday features within the sequence block to obtain local time series of blood glucose. and global blood glucose time series The 3D temporal feature vector of the sequence block in the sequence;

[0085] Step 212: By a linear projection to Dimension, and add to the sequence block;

[0086]

[0087]

[0088] The dimension of the embedded vector;

[0089] This is the index of the sequence block within the sequence. For the even or odd index of the embedding dimension;

[0090] This encoding is added to the sequence block representation of each branch to preserve relative positional relationships and periodicity;

[0091] Step 213: Local time series analysis of blood glucose and global blood glucose time series The sequence block embeddings in the data use a learnable embedding layer encoding for diabetes subtypes, including local and global blood glucose temporal embeddings.

[0092] T1DM is marked as learnable embedding layer code 0, and T2DM is marked as learnable embedding layer code 1;

[0093] Ensure the predictive model is sensitive to physiological differences among different patients;

[0094] Step 22: Local temporal embedding data of blood glucose is fed into a 2-layer Transformer encoder stack to extract high-resolution features. The global temporal embedding data of blood glucose is input into a 4-layer Transformer encoder stack to extract low-resolution features. .

[0095] The Transformer layer adopts a standard encoder structure, with each layer including a multi-head self-attention network and a feedforward network, supplemented by residual connections and layer normalization.

[0096] The specific steps for step 22 are as follows:

[0097] Step 221: In the self-attention mechanism, for the input representation The query (Q), key (K), and value (V) matrices are generated through three linear projections.

[0098] for or , , ;

[0099] Generate query (Q), key (K), and value (V) matrices using three linear projections:

[0100]

[0101] in The weight matrix is ​​a learnable weight matrix;

[0102]

[0103] For the number of attention heads;

[0104] Step 222: Calculate the attention score:

[0105] Attention score is calculated as follows:

[0106]

[0107] Step 223: Apply the above operations in parallel according to the multi-head mechanism. Each output is a separate header, then the outputs are concatenated and projected back through another linear layer. Dimensions, to obtain local time series of blood glucose High-resolution features in and global blood glucose time series Low-resolution features in ;

[0108] A 2-layer Transformer encoder stack is used to maintain sensitivity to local details; a 4-layer Transformer encoder stack is used to enhance the capture of long-range dependencies, ensuring that each branch optimizes feature extraction at its respective scale.

[0109] Different sequence block lengths and step sizes are used to achieve both fine-grained local modeling and long-range global modeling of blood glucose dynamics. Local temporal blood glucose data. Focusing on capturing rapid fluctuations and localized patterns in the short term, such as postprandial blood glucose spikes or immediate responses after insulin injections; global blood glucose time series. Aimed at extracting broader global trends, it can effectively alleviate the trade-off between short-term noise interference and long-term dependence faced by single-scale Transformers when processing blood glucose sequences, thereby improving the robustness of prediction models in capturing complex physiological dynamics.

[0110] Step 3: Predictive model for local time series of blood glucose High-resolution features in and global blood glucose time series Low-resolution features in Feature fusion is performed, and the prediction model is jointly optimized and trained by introducing Fishr regularization term and weighted focusing loss value until training converges.

[0111] Step 3 is detailed below:

[0112] Step 31: Local time series of blood glucose High-resolution features in and global blood glucose time series Low-resolution features in The input is fed into a prediction model, where a learnable router dynamically fuses local blood glucose time series data. High-resolution features in and global blood glucose time series Low-resolution features in This yields the final predicted sequence;

[0113] The specific steps for step 31 are as follows:

[0114] Step 311: Transfer high-resolution features and low-resolution features By concatenating along the feature dimension, the concatenated features are obtained;

[0115] The splicing characteristics are: ;

[0116] Step 312: Input the spliced ​​features into the linear layer. The linear layer projects the spliced ​​features into a 2D space and then uses the Softmax function to generate the fusion weights of each position for the two branches.

[0117]

[0118] in: and These are learnable parameters;

[0119] Step 313: Adjust the high-resolution features according to their weights and low-resolution features By performing fusion, fusion characteristics are obtained. ;

[0120] The fusion feature is calculated as follows:

[0121]

[0122] For high-resolution branches The weight, Low-resolution features The weights;

[0123] Gating mechanisms allow predictive models to adaptively balance local and global information based on the specific context of the input sequence and patient characteristics, avoiding the introduction of noise or information loss;

[0124] Step 314: Merge features The data is normalized by a normalization layer, then flattened, and the flattened vector is input into a linear layer and projected onto the length of a future time step to obtain the final prediction sequence.

[0125] Step 32: Calculate the weighted focusing loss value of the final predicted sequence;

[0126] The specific steps for step 32 are as follows:

[0127] Step 321: Calculate the fundamental root mean square error of the final predicted sequence. ;

[0128] Root mean square error The calculation is as follows:

[0129]

[0130] in and These are the actual blood glucose value and the predicted value from the prediction model at the m-th prediction time point, respectively. To predict the total number of time points in the time domain.

[0131] Root mean square error Directly quantifying the numerical bias of predictions is suitable for continuous blood glucose regression tasks, but its uniform weighting mechanism cannot cope with the imbalance problem.

[0132] Step 322: Blood glucose value discretization: Continuous blood glucose values ​​are divided into three categories according to clinical thresholds and the corresponding BIN values ​​are defined as class-specific weighting coefficients α in the subsequent focus loss.

[0133] The three categories are: hypoglycemia (<70mg / dL, BIN=10), normal blood glucose (70-180mg / dL, BIN=1), and hyperglycemia (>180mg / dL, BIN=2).

[0134] Step 323: Calculate the confidence level of the probability that the predicted sequence falls within its true blood glucose range. ;

[0135] The probability confidence level when the predicted sequence is highly confident and correct A probability confidence level close to 1 when predicting a sequence is difficult or incorrect. Approaching 0;

[0136] Step 324: Based on the root mean square error and probability confidence Calculate the weighted focusing loss value ;

[0137]

[0138] For category weighting coefficients, ;

[0139] Step 33: Calculate the Fishr loss value of the Fishr regularization term for the final predicted sequence;

[0140] The specific steps for step 33 are as follows:

[0141] Step 331: Within each training batch, patients are sampled in equal batches;

[0142] Ensure that samples are included from different patients.

[0143] For example, from a total of 100 patients, 8 patients are randomly selected in each batch, and data from 4 time windows are extracted from each patient.

[0144] Step 332: For the sampled data of each patient within a batch (considered as an independent "domain"), calculate its specific task loss. The task loss is calculated through backpropagation. For all parameters of the prediction model gradient and gradient Flatten and concatenate them into a gradient vector;

[0145]

[0146]

[0147] Represents a specific training domain A subset of samples within;

[0148] Step 333: Calculate the mean of the gradient vectors for all patients. According to gradient and the mean of the gradient vectors of all patients Calculate the Fishr loss value;

[0149]

[0150] This indicates the number of unique training domains within the current batch, used for normalization of the regularization term;

[0151] The goal is to narrow down the gradient distribution among different patients, forcing the model to learn features that are universal to all patients.

[0152] Step 34: Calculate the total loss value using the weighted focusing loss value and the Fishr loss value;

[0153] The total loss is calculated as follows:

[0154]

[0155] It is the regularization intensity coefficient;

[0156] Step 35: Perform backpropagation on the total loss, calculate the gradient, and update all learnable parameters in the prediction model using the optimizer;

[0157] Learnable parameters include embedding layer parameters, Transformer encoder parameters, learnable router parameters, and prediction head parameters;

[0158] Step 36: Repeat steps 31 to 35 to traverse the high-resolution features. and low-resolution features After multiple rounds, the change in total loss after each round is less than Alternatively, when the preset maximum number of training rounds is reached, training is stopped, the parameters of the best-performing prediction model are saved, and the prediction model is obtained.

[0159] Step 4: Perform the same preprocessing operation as in Step 1 on the univariate blood glucose time series of the patients to be predicted, and input the processed data into the prediction model to obtain the prediction sequence.

[0160] By employing a dual-branch multi-resolution architecture and a layer-by-layer learnable router, multi-scale features are dynamically fused to balance the capture of local and global information. Patient-level Fishr regularization is introduced, treating each patient as an independent domain. By matching the loss landscape between domains and within batch gradient variance, zero-shot generalization ability is improved. Relying solely on univariate CGMS sequences, the cost of clinical application is reduced.

[0161] The prediction model includes a dual-scale temporal preprocessing layer, a dual-branch Transformer feature extractor, an adaptive feature fusion module, a joint optimization trainer, and a zero-shot inference engine.

[0162] 1. The dual-scale temporal preprocessing layer is used to perform step 1; the dual-scale temporal preprocessing layer implements the following functions;

[0163] 1.1 Completion: Linear interpolation to fill in missing values ​​at the end of blood glucose sequences;

[0164] 1.2 Splitting: The sequence after splitting and filling using a double sliding window;

[0165] 1.3 Output: Local time series of blood glucose levels and global blood glucose time series ;

[0166] 2. The dual-branch Transformer feature extractor executes step 2, predicting the core neural network of the model;

[0167] The two-branch Transformer feature extractor includes the following structure:

[0168] 2.1 Embedded layer (encoding logic shared by the two branches);

[0169] First, the time-series data is transformed into features that can be recognized by the Transformer;

[0170] Value projection: Blood glucose value → High-dimensional feature vector;

[0171] Timestamp encoding: Time information → Temporal characteristics;

[0172] Position encoding: adding positional information to a sequence;

[0173] Patient type embedding: a key foundation for adapting to different patients with zero samples;

[0174] 2.2 Dual Depth Encoder Stack;

[0175] Local branch (high resolution): 2-layer Transformer encoder → extract high resolution features;

[0176] Global branch (low resolution): 4-layer Transformer encoder → extract low-resolution features;

[0177] Output: Two sets of complementary high-resolution features and low-resolution features;

[0178] 3. The adaptive feature fusion module and the joint optimization trainer jointly execute step 3 to realize the prediction model, which is also the core of the prediction model training.

[0179] 3.1 Learnable Router (Predictive Head)

[0180] High-resolution and low-resolution features are fused into the final predicted sequence:

[0181] Stitching: High-resolution features + low-resolution features;

[0182] Dynamic weights: Linear projection + Softmax to generate adaptive weights;

[0183] Fusion Output: After weighted fusion, a blood glucose prediction sequence is generated through a linear layer;

[0184] 3.2 Joint Loss Optimization Mechanism (Prediction Model Training Rules)

[0185] Update the prediction model parameters using backpropagation:

[0186] Main loss: Weighted focusing loss;

[0187] Regularized loss: Fishr loss;

[0188] After convergence: the optimal prediction model is obtained.

[0189] IV. The zero-shot inference engine is used to execute step 4;

[0190] After the model is trained, the prediction logic for new patients is as follows:

[0191] New patient blood glucose timeline → Perform the exact same pretreatment;

[0192] Input the trained model → forward propagation;

[0193] Direct output: Future blood glucose prediction sequence.

[0194] Specific cases are as follows:

[0195] A. Dataset and Preprocessing

[0196] Three real-world continuous glucose monitoring (CGM) datasets were used: the Shanghai Type 2 Diabetes dataset collected by Tongji University (100 Type 2 diabetes patients, monitoring duration 3–14 days), the Ohio University Type 1 Diabetes 2018 and 2020 datasets (12 Type 1 diabetes patients, 8 weeks of long-term monitoring each), and the publicly available glucose replacement trial dataset (226 Type 1 diabetes patients, monitoring duration exceeding 26 weeks). The demographic characteristics of the subjects in these clinical datasets are shown in Table 1. Due to the inevitable presence of missing and outlier values ​​caused by transient sensor detachment, calibration errors, or signal interference in clinical CGM devices, we first imputed the missing glucose values ​​in the training set using linear interpolation; subsequently, we used an outlier detection and cleaning method based on interquartile range (IQR).

[0197]

[0198]

[0199]

[0200] in and These represent the 25th and 75th percentiles of the patient's blood glucose sequence, respectively. Any blood glucose value below the lower bound or above the upper bound was considered an extreme outlier and was pruned to the corresponding boundary value. To ensure all samples had a completely consistent input length, and to align the monitoring durations of different patients, only the first 72 consecutive hours after the start of monitoring for each patient were retained as the final modeling sequence. Ultimately, the patient sequences were strictly divided into training, validation, and test sets in a 7:1:2 ratio, and no time windows of the same patient appeared across sets. This constructed a rigorous patient-level zero-sample domain generalization experimental setting, realistically simulating the core requirement of accurate and reliable blood glucose prediction for completely unseen patients in clinical deployment scenarios.

[0201] Table 1: Demographic Characteristics of the Dataset

[0202]

[0203] B. Selection of History Length

[0204] The performance of blood glucose prediction is highly dependent on the length of historical information observed by the prediction model: too short a historical window cannot capture complete physiological dynamics, while too long a window introduces a large amount of redundant information, increases noise, and significantly increases the computational burden. To determine the optimal historical sequence length, four historical window lengths—60, 120, 240, and 480 minutes—were systematically evaluated under a fixed 5-minute sampling frequency. For each historical length, four prediction window lengths of 30, 60, 90, and 120 minutes were tested. All experiments were conducted under identical zero-sample patient-level partitioning, prediction model structure, and hyperparameter settings to ensure comparability of results.

[0205] Evaluation metrics included both clinical and statistical criteria: root mean square error (RMSE, mg / dL), mean absolute error (MAE, mg / dL), the proportion of region A in Clark error grid analysis (region A, within the clinically acceptable range of ±20% or ±20 mg / dL), and the accuracy of glycemic event detection (ACC, %), where hypoglycemia was defined as <70 mg / dL and hyperglycemia as >180 mg / dL. Experimental results are as follows: Figures 2-6 As shown in Table 2. Figures 4-6 Clark's error grid divides the relative error between predicted and reference blood glucose values ​​into five regions: Region A is the clinically accurate region, where the deviation from the reference value is within 20% or within 20 mg / dL when the value is below 70 mg / dL; Region B is the clinically acceptable region, where the deviation exceeds 20% but does not lead to incorrect treatment; Region C is the clinically incorrect region, where the deviation may lead to unnecessary treatment; Region D is the clinically dangerous region, where serious blood glucose abnormalities are not detected; and Region E is the clinically critical region, where the deviation will lead to completely opposite treatment decisions. It can be seen that a 120-minute history length achieves the best balance across almost all prediction time domains and indicators, and 120 minutes was ultimately selected as the history sequence length. The Clark error grid analysis results for the three datasets with 120-minute history windows and 30-minute prediction windows are shown below. Figures 2-6 As shown in Table 2, the Clark error grid for all test samples shows that most of the predicted data is in region A, which fully demonstrates the reliability of the 120-minute historical window prediction.

[0206] Table 2: Performance of windows with different historical sequence lengths

[0207]

[0208] Table 2 continued:

[0209]

[0210] The curves showing the changes in RMSE and accuracy over the prediction time domain at different sequence lengths (60, 120, 240, and 480 minutes) indicate that 120 minutes achieves the best balance. Figures 4-6 As shown, Clark error grid plots for each patient group on three test sets (Shanghai Type 2 Diabetes, Ohio University Type 1 Diabetes, and Glucose Replacement Trial datasets) show that almost all points fall within the clinically acceptable region A, demonstrating strong zero-sample generalization ability.

[0211] C. Ablation test

[0212] To systematically evaluate the contribution of each key component of the prediction model to zero-sample blood glucose prediction performance, we conducted ablation experiments on three test sets, successively removing timestamp encoding, weighted Focal loss, Fishr regularization, low-resolution branch, and high-resolution branch. We reported RMSE, MAE, and event detection accuracy (ACC) at four prediction time domains of 30, 60, 90, and 120 minutes. All ablation variants maintained consistent hyperparameters and training procedures to ensure comparability of results. The results are summarized in Table 3 and analyzed using... Figure 7-10 The box plots and time series curves are used to supplement the visualization.

[0213] Figure 7-9 Consistent with the results in Table 3, the complete prediction model achieved the best performance across all datasets and prediction time domains, demonstrating the indispensable synergy of its components. Removing any single component led to performance degradation, indicating that each module significantly contributes to the overall robustness and accuracy of the prediction model. The most severe performance degradation occurred when the low-resolution branch was removed, reflecting its largest contribution. This is fundamentally because the long-term global dependence of blood glucose sequences heavily relies on the slow trends captured by large-scale patches. Without this branch, the prediction model relies solely on high-resolution local patterns, failing to effectively integrate long-term physiological context, resulting in a significant collapse in generalization ability in zero-sample scenarios with high patient physiological heterogeneity. The removal of the low-resolution branch directly disrupts the core architecture of multi-scale information fusion, making it a key bottleneck in maintaining long-term accuracy and cross-patient robustness.

[0214] Figures 7-9 To illustrate patient-level RMSE box plots for different prediction time domains on three test sets (Shanghai Type 2 Diabetes, Ohio University Type 1 Diabetes, and Glucose Replacement Trial datasets), the rectangular boxes represent the interquartile range (IQR), which highly reflects the dispersion of the middle 50% of the data values; the line segments extending from the top and bottom of the boxes represent the normal fluctuation range of the data; and the black dots represent outliers, which are statistically extreme observations. On all three datasets, the complete model's boxes are generally lower and have no extreme observations, indicating that the complete prediction model has the lowest and most compact error distribution. Figure 10 For the comparison of long-term prediction curves for a typical patient, the complete prediction model (black) most closely follows the true trajectory (red), while the ablation variants show varying degrees of bias and lag.

[0215] Table 3: Ablation Experiment Results

[0216]

[0217] Table 3 (continued)

[0218]

[0219] D. Comparison with SOTA (State-Of-The-Art) methods

[0220] To evaluate the superiority of the proposed method in zero-sample blood glucose prediction, we compared it with state-of-the-art (SOTA) methods, including Transformer-based N-BEATS (Neural Basis Expansion Analysis for Interpretable Time Series Forecasting), Bi-LSTM (Bidirectional Long Short-Term Memory), and ARIMA (Autoregressive Integrated Moving Average). Experiments were conducted with the same patient-level zero-sample partitioning, evaluating RMSE, MAE, and ACC over the 30–120 minute prediction time domain. The results are shown in Table 4. Figure 11-14 Box plots and time-series curves were visualized. Results showed that the complete prediction model achieved best performance across all datasets and time domains, particularly excelling in the long time domain. This is attributed to the fine dynamics captured by multi-resolution dual-experts and the patient-wide invariance enhanced by Fishr. N-BEATS showed acceptable short-term prediction, but its lack of domain-invariant mechanisms resulted in weak generalization. Bi-LSTM sequence modeling was limited and highly susceptible to noise interference. ARIMA, as a linear prediction model, could not handle glucose non-stationarity, leading to the largest error. In summary, these results confirm that the proposed framework is comprehensively superior in both numerical and clinical robustness.

[0221] Comparison with State-of-the-Art (SOTA) methods: Figures 11-13 On three test sets (Shanghai type 2 diabetes, Ohio University type 1 diabetes and glucose replacement trial datasets), patient-level RMSE box plots in different prediction time domains show that the proposed method consistently achieves the lowest and most compact error distribution. Figure 14A comparison of long-term prediction curves for a typical patient shows that the proposed method (black curve) most closely follows the actual blood glucose trajectory (red curve), while competing methods (N-BEATS, Bi-LSTM, and ARIMA) exhibit increasingly significant bias and lag in the longer prediction time domain. Specific data are shown in Table 4.

[0222] In the 30-120 minute prediction time domain, it reduces RMSE by 10-30% and improves the accuracy of hypoglycemic / hyperglycemic event detection by 2-5%, significantly surpassing existing state-of-the-art methods. Its core innovation lies in its dual-branch multi-resolution Transformer architecture, which captures local fine dynamics and global long-range trends through parallel high-resolution and low-resolution experts respectively, and introduces a layer-by-layer learnable router to achieve dynamic fusion, effectively solving the trade-off between local noise overfitting and global detail neglect in traditional single-scale Transformers. Simultaneously, the framework introduces patient-level Fishr domain-invariant regularization, treating each patient as an independent domain. By matching the gradient variance of patients within a batch to align the loss landscape, it significantly suppresses individual-specific overfitting, thereby greatly improving zero-shot generalization ability across patients and diabetes types. While maintaining its lightweight and easy-to-deploy characteristics, this framework performs excellently in real-world scenarios with high physiological heterogeneity.

[0223] Table 4: Comparison with the SOTA method

[0224]

[0225] Table 4 (continued)

[0226]

[0227] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A zero-shot blood glucose prediction method based on a two-scale Transformer framework, characterized in that, The specific steps are as follows: Step 1: Preprocessing the input univariate blood glucose time series to obtain blood glucose local time series and blood glucose global time series ; Step 2: Prediction model extracts local time series of blood glucose. High-resolution features in and global blood glucose time series Low-resolution features in ; Step 3: Predictive model for local time series of blood glucose High-resolution features in and global blood glucose time series Low-resolution features in Feature fusion is performed, and the prediction model is jointly optimized and trained by introducing Fishr regularization term and weighted focusing loss value until training converges. Step 4: Perform the same preprocessing operation as in Step 1 on the univariate blood glucose time series of the patients to be predicted, and input the processed data into the prediction model to obtain the prediction sequence; The specific steps for step 3 are as follows: Step 31: Local time series of blood glucose High-resolution features in and global blood glucose time series Low-resolution features in The input is fed into a prediction model, where a learnable router dynamically fuses local blood glucose time series data. High-resolution features in and global blood glucose time series Low-resolution features in This yields the final predicted sequence; Step 32: Calculate the weighted focusing loss value of the final predicted sequence; Step 33: Calculate the Fishr loss value of the Fishr regularization term for the final predicted sequence; Step 34: Calculate the total loss value using the weighted focusing loss value and the Fishr loss value; Step 35: Perform backpropagation on the total loss, calculate the gradient, and update all learnable parameters in the prediction model using the optimizer; Step 36: Repeat steps 31 to 35 to traverse the high-resolution features. and low-resolution features After multiple rounds, the change in total loss after each round is less than Alternatively, when the preset maximum number of training rounds is reached, training is stopped, the parameters of the best-performing prediction model are saved, and the prediction model is obtained. The specific steps for step 32 are as follows: Step 321 : Calculate the final prediction sequence's basis root mean square error ; Step 322: Divide continuous blood glucose values ​​into three categories according to clinical thresholds, and define the corresponding BIN values ​​as the weighting coefficient α in subsequent focus loss; Step 323: Calculate the confidence level of the probability that the predicted sequence falls within its true blood glucose range. ; Step 324: Compute the weighted focus loss value based on the base root mean square error and the probability confidence Step 325: Compute the weighted focus loss value based on the base root mean square error ; The specific steps for step 33 are as follows: Step 331: Within each training batch, patients are sampled in equal batches; Step 332: For the sampling data of each patient within the batch, calculate its specific task loss. The task loss is calculated through backpropagation. For all parameters of the prediction model gradient and gradient Flatten and concatenate them into a gradient vector; Step 333: Calculate the mean of all patient gradient vectors , according to the gradient and the mean of all patient gradient vectors Calculate the Fishr loss value.

2. The method of claim 1, wherein the method is based on a two-scale Transformer framework. Step 1 is performed as follows: Step 11: Linear interpolation is used to fill in the end of the monovariant blood glucose time series the end of the potential filling part is filled in; Step 12: Use two sets of sliding window parameters to fill in the univariate blood glucose time series. Segmentation generation and A series of consecutive time blocks, Combining consecutive time blocks yields local blood glucose time series. ,Will The global blood glucose time series is obtained by combining consecutive time blocks. .

3. The dual-scale Transformer framework based zero-shot blood glucose prediction method of claim 1, wherein, Step 2 is performed as follows: Step 21: Local time series of blood glucose and global blood glucose time series The blood glucose local temporal embedding data and blood glucose global temporal embedding data are obtained by passing through the embedding layer in the prediction model, respectively, performing value projection, temporal feature embedding, location encoding and patient type embedding; Step 22: Blood glucose local temporal embedding data is fed into a stack of 2-layer Transformer encoders to extract high-resolution features Blood glucose global temporal embedding data is fed into a stack of 4-layer Transformer encoders to extract low-resolution features .

4. The dual-scale Transformer framework based zero-shot blood glucose prediction method of claim 3, wherein, Step 21 is performed as follows: Step 211: Local time series of blood glucose and global blood glucose time series After passing through the embedding layer, the timestamp encoding first processes the local time sequence of blood glucose. and global blood glucose time series Average pooling is performed on minute, hour, and weekday features within the sequence block to obtain local time series of blood glucose. and global blood glucose time series The 3D temporal feature vector of the sequence block in the sequence; Step 212: Projecting by a line to dimension, and adding to the sequence block; Step 213: Local time series analysis of blood glucose and global blood glucose time series The sequence block embeddings in the data are encoded using a learnable embedding layer for diabetes subtypes, resulting in local and global temporal blood glucose embeddings.

5. The method of claim 4, wherein the method is based on a two-scale Transformer framework. The specific steps for step 22 are as follows: Step 221 : in the self-attention mechanism, for the input representation query, key and value matrices are generated by three linear projections; Step 222: Calculate the attention score: Step 223: Apply the above operations in parallel according to the multi-head mechanism. Each output is a separate header, then the outputs are concatenated and projected back through another linear layer. Dimensions, to obtain local time series of blood glucose High-resolution features in and global blood glucose time series Low-resolution features in .

6. The method of claim 1, wherein the method is based on a two-scale Transformer framework. The specific steps for step 31 are as follows: Step 311: splice the high-resolution features and the low-resolution features along the feature dimension to obtain spliced features; Step 312: Input the spliced ​​features into the linear layer. The linear layer projects the spliced ​​features into a 2D space and then uses the Softmax function to generate the fusion weights of each position for the two branches. Step 313: Adjust the high-resolution features according to their weights and low-resolution features By performing fusion, fusion characteristics are obtained. ; Step 314: Merge features The data is normalized by a normalization layer, then flattened, and the flattened vector is input into a linear layer and projected onto the length of a future time step to obtain the final prediction sequence.

Images