A PPG-based cross-modal cross-attention fusion network for blood glucose detection

Abstract

This invention relates to the fields of biomedical signal processing and artificial intelligence, and discloses a blood glucose detection method based on a PPG-based cross-modal attention fusion network. The method includes acquiring raw PPG signals from the human body and preprocessing the raw PPG signals to obtain effective PPG signal segments. Based on the same PPG signal segment, at least two different modal feature representations are constructed, including at least: a first modal feature, which is a temporal feature constructed based on the PPG signal segment, including the original temporal signal and its derivative features and frequency domain features; and an image feature obtained by mapping the PPG signal segment to a two-dimensional temporal image. This invention, by constructing temporal and image dual-modal features from the same PPG signal segment, can simultaneously mine the temporal dynamics, frequency domain characteristics, and two-dimensional spatial morphological information of the signal, overcoming the limited expressive power of traditional single temporal features, and achieving a more comprehensive and deeper representation of blood glucose-related physiological information.

Description

Technical Field

[0001] This invention relates to the fields of biomedical signal processing and artificial intelligence, and in particular to a blood glucose detection method based on a PPG-based cross-modal cross-attention fusion network. Background Technology

[0002] Currently, blood glucose testing mainly relies on invasive finger-prick blood sampling methods, which are highly accurate but invasive and cannot provide continuous monitoring. Among non-invasive methods, techniques based on single physiological signals such as PPG usually only utilize single temporal features, making it difficult to fully characterize the complex nonlinear relationship between blood glucose changes and cardiovascular dynamics.

[0003] Existing deep learning solutions mostly focus on single modal inputs or simple feature concatenation, lacking effective modeling of the relationships between features of different modalities. This results in limited model generalization ability and stability, and insufficient prediction accuracy and robustness in scenarios with large individual differences. Summary of the Invention

[0004] This invention provides a blood glucose detection method based on PPG cross-modal cross-attention fusion network to solve existing technical problems, thereby addressing the issue of limited expressive power of traditional single temporal features.

[0005] To address the aforementioned technical problems, according to one aspect of the present invention, more specifically, a blood glucose detection method based on a PPG-based cross-modal cross-attention fusion network includes the following steps: S1. Acquire the raw PPG signal of the human body and preprocess the raw PPG signal to obtain an effective PPG signal segment. S2. Based on the same PPG signal segment, construct feature representations of at least two different modes, wherein at least: The first modal feature is a time-series feature constructed based on the PPG signal segment, including the original time-series signal and its derivative features and frequency domain features; The second modal feature is the image feature obtained by mapping the PPG signal segment into a two-dimensional time-series image; S3. Perform feature encoding on the first modal feature and the second modal feature respectively to obtain temporal feature representation and image feature representation; S4. Based on the cross-modal cross-attention mechanism, the temporal feature representation and image feature representation are subjected to bidirectional information interaction and fusion to obtain cross-modal fused features; S5. Jointly model the cross-modal fusion features with individual structured features and output the corresponding blood glucose prediction value.

[0006] Furthermore, the preprocessing steps include: filtering, denoising, segmenting, and normalizing the raw PPG signal.

[0007] Furthermore, the time-series characteristics also include at least one of the following: time-domain statistical characteristics, time-frequency domain characteristics, nonlinear and periodic characteristics.

[0008] Furthermore, the method of mapping PPG signal segments into two-dimensional time-series images includes: converting one-dimensional time-series signals into two-dimensional grayscale images, spectrum diagrams, time-frequency diagrams, or Gram angle field diagrams.

[0009] Furthermore, the cross-modal cross-attention mechanism includes: The temporal feature representation is used as the query vector, and the image feature representation is used as the key vector for the first attention calculation. The image feature representation is used as the query vector, and the temporal feature representation is used as the key vector for second attention calculation. The features from the two attention outputs are weighted and fused to obtain cross-modal fusion features.

[0010] Furthermore, the individual structured characteristics include at least one of the following: age, sex, body mass index (BMI), history of diabetes, blood pressure, history of heart disease, and medication use.

[0011] Furthermore, the feature encoding step includes: Temporal features are encoded using a temporal encoder, which includes a one-dimensional convolutional neural network and a long short-term memory network. Image features are encoded using an image encoder, which includes a two-dimensional convolutional neural network and a residual network.

[0012] Furthermore, the image encoder employs a multi-scale feature extraction structure, including multiple convolutional layers or attention layers of different scales, for extracting local and global features from the image.

[0013] The blood glucose detection method based on PPG cross-modal cross-attention fusion network provided by this invention achieves the following advantages compared with existing technologies: 1. This invention constructs time-series and image dual-modal features from the same PPG signal segment, which can simultaneously mine the temporal dynamics, frequency domain characteristics and two-dimensional spatial morphological information of the signal, overcoming the limitation of the traditional single time-series feature's limited expressive ability, and realizing a more comprehensive and deeper characterization of blood glucose-related physiological information.

[0014] 2. This invention introduces a cross-modal cross-attention mechanism to achieve bidirectional and adaptive interaction and fusion between temporal features and image features, rather than simple feature splicing. This explicitly models the intrinsic correlation and complementary relationship between features of different modalities, significantly improving the effectiveness and accuracy of multi-source information fusion and enhancing the model's ability to characterize complex nonlinear relationships.

[0015] 3. This invention combines cross-modal fusion features with individual structured features (such as age, BMI, medical history, etc.) for joint modeling, enabling the algorithm to adaptively learn and adapt to the physiological characteristics and health conditions of different users. This effectively alleviates the fluctuations in model performance caused by individual differences, thereby achieving higher prediction stability and robustness in a large population.

[0016] 4. This invention is based on non-invasive PPG signal processing, which does not require blood sampling. It is suitable for wearable devices to achieve continuous and non-invasive blood glucose monitoring, which greatly improves user experience and long-term monitoring compliance, and provides a practical technical means for daily management and early warning of chronic diseases such as diabetes. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of the present invention; Figure 2 This is a flowchart of the present invention. Detailed Implementation

[0018] To make the technical solution of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] like Figure 1 , 2 As shown in the figure, this invention provides a blood glucose detection method based on a PPG-based cross-modal cross-attention fusion network. The core of this method lies in addressing the problems of limited feature representation, simple modal fusion, and poor robustness to individual differences in existing non-invasive blood glucose detection technologies based on photoplethysmography (PPG). A novel multimodal feature construction and deep fusion architecture is proposed. This method extracts and constructs two heterogeneous modal features—temporal and image-based—from the same PPG signal segment, and utilizes a cross-modal cross-attention mechanism for deep interaction and fusion. Finally, it combines individual structured features to achieve high-precision and high-stability blood glucose level prediction. The invention will be described in detail below with reference to the accompanying drawings and specific steps.

[0020] Step 1: Signal Acquisition and Preprocessing First, raw PPG signals from the human body are acquired using wearable devices (such as smart bracelets or smartwatches) or dedicated photoelectric sensors. These raw signals typically contain motion artifacts, baseline drift, and high-frequency noise. Therefore, preprocessing is required to obtain clean signal segments suitable for analysis. The preprocessing steps specifically include: S101. Use a bandpass filter (e.g., 0.5 Hz - 5 Hz) to filter out low-frequency baseline drift and high-frequency noise in the signal that are not related to the pulse.

[0021] S102. Wavelet transform, adaptive filtering and other algorithms can be used to further suppress residual motion artifacts.

[0022] S103. Segment the continuous PPG signal according to heart rate or a fixed time window (e.g., 8-10 seconds per segment) to obtain multiple independent PPG signal segments. , where T is the segment length.

[0023] S104. Perform max-min normalization or Z-score standardization on each signal segment to eliminate amplitude scale differences and accelerate model convergence.

[0024] After the above preprocessing, a valid PPG signal fragment is obtained for subsequent analysis. .

[0025] Step 2: Multimodal Feature Construction One of the key innovations of this invention is to construct two different modal feature representations from the same PPG signal segment in order to fully mine the blood glucose-related information contained in the signal.

[0026] First modal feature (time series feature): directly constructed based on the PPG signal segment. This time series feature not only includes the original one-dimensional time series waveform data of the PPG, but also extracts deeper descriptors through feature engineering or shallow networks, aiming to characterize the time domain, frequency domain, and nonlinear dynamics of the signal. Specifically, it may include: 1. Basic waveform: Normalized PPG signal .

[0027] 2. Characteristics of the derivative: First Derivative: It reflects the speed of waveform change; Second Derivative: It reflects the waveform acceleration and extreme point information; 3. Frequency domain characteristics: For Perform a Fast Fourier Transform (FFT) and obtain its logarithmic magnitude spectrum: ,in To prevent numerical underflow at the minimum value, frequency domain energy distribution characteristics are explicitly introduced.

[0028] 4. Other advanced features: These may include time-domain statistical features (such as mean, variance, skewness, and kurtosis), time-frequency domain features (such as those obtained through wavelet transform), nonlinear features (such as approximate entropy and sample entropy), and periodic features. These features collectively constitute a rich representation of the PPG signal in the time dimension, denoted as the time-series feature vector. .

[0029] Second modal features (image features): Transforming one-dimensional time-series signals... The image is mapped to a two-dimensional image to extract its spatial structure information. This invention preferably uses two methods—Gramian Angular Field (GAF) and Recurrence Plot (RP)—to generate multi-channel images: Summation Gram Square (GASF): This method sums the normalized signal... Map the angle to polar coordinates using the inverse cosine function, and then calculate the sum of the cosines: , ; ; The resulting GASF matrix is ​​a T×T symmetric matrix that encodes the temporal correlation between signal point pairs.

[0030] Differential Gram Field (GADF): Calculate the sine difference of the angle. ; Recurrence graphs (RP): used to visualize the periodicity, chaos, and other nonlinear dynamic characteristics of time-series data. Their elements... Defined as: ; in, and It is the state vector of the signal in phase space (which can be constructed by time delay embedding method); It is a distance threshold; It is the Hervested step function (1 if the distance is less than the value, otherwise 0).

[0031] Finally, the three single-channel images GASF, GADF, and RP are stacked along the channel dimension to form a multi-channel tensor of the form [2, H, W]. As input to the image modality, this method generates heterogeneous modalities rich in complementary information from a single signal source, overcoming the complexity of Comparative Document 1, which relies on multiple hardware sensors (BVP, HR, EDA), and the limitation of Comparative Document 2, which only performs time-frequency domain analysis.

[0032] Step 3: Feature Encoding To transform the original features constructed above into a higher-level abstract representation that is more suitable for deep learning models, the features of the two modalities need to be encoded separately.

[0033] Temporal feature encoding: using a temporal encoder to encode temporal features Processing is performed. The encoder can be: One-dimensional convolutional neural network (1D-CNN): Extracts local temporal patterns through convolutional kernels.

[0034] Long Short-Term Memory (LSTM) networks or Bidirectional LSTM (Bi-LSTM): capture long-range dependencies. For an LSTM unit, its computation can be represented as: ; ; ; ; ; ; in, , , These are the forget gate, input gate, and output gate, respectively. In cellular state; It is in a hidden state; This is the sigmoid function.

[0035] Transformer encoder: Utilizes a self-attention mechanism to model the global temporal context. The encoded output is a fixed-dimensional temporal feature representation. .

[0036] Image feature encoding: Using an image encoder to encode second-modality features (multi-channel images) The encoder can be: Two-dimensional convolutional neural networks (2D-CNN): such as ResNet, extract local texture and shape features of images through convolutional layers.

[0037] Visual Transformer (VIT): The image is divided into blocks and input into a Transformer structure. The encoded output is a fixed-dimensional image feature representation. .

[0038] Step 4: Cross-modal cross-attention fusion This is the core innovation of the present invention, aiming to break through the limitations of traditional simple concatenation fusion methods (such as directly concatenating features of different signals in comparison file 1), and achieve deep, bidirectional information interaction between temporal and image modal features. Specifically, it is achieved through a cross-modal cross-attention mechanism, the mathematical expression of which is as follows: 1. Time-as-Query Image: Time series features Projection to query vector: .

[0039] Image features The projection is into a key vector and a value vector: , .

[0040] Calculate attention weights and output: ; in, This is the dimension of the key-to-key pair. This step generates a temporal context vector modulated and supplemented by image information. .

[0041] 2. Image query timing (image-as-Query): Symmetrically, image features Projected onto the query vector: .

[0042] Time series features Projection as key and value: , .

[0043] Calculate injection force output: ; This step generates an image context vector that is constrained and supplemented by temporal information. .

[0044] 3. Feature fusion: The two context vectors calculated in the above two steps are then fused. For example, after concatenation, they are passed through a fully connected layer (FFN) for dimensionality reduction and information integration. ; Ultimately, we obtained cross-state composite features that contain bidirectional interactive information. This feature retains the inherent information of each modality while deeply integrating complementary information from another modality, significantly enhancing its representational capabilities.

[0045] Step 5: Joint Modeling and Glucose Prediction To further improve the model's generalization ability and personalized prediction accuracy in real-world populations, the cross-modal fusion features obtained in the previous step will be used... Interacting with individual structured characteristics (e.g., age, gender, BM, etc.) are used for joint modeling.

[0046] 1. Feature concatenation: Concatenate the two features into a joint feature vector. .

[0047] 2. Regression Prediction: The input is fed into a regression prediction network (usually consisting of several fully connected layers).

[0048] 3. Output: The final output layer of this regression network consists of a single neuron, which directly outputs a continuous scalar value, namely the predicted blood glucose concentration. : ; By incorporating individual characteristics, the model can adaptively correct baseline shifts caused by physiological differences, effectively mitigating model performance fluctuations due to individual differences, thereby achieving higher predictive stability in a large population.

[0049] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A blood glucose detection method based on PPG cross-modal cross-attention fusion network, characterized in that, Includes the following steps: S1. Acquire the raw PPG signal of the human body and preprocess the raw PPG signal to obtain an effective PPG signal segment. S2. Based on the same PPG signal segment, construct feature representations of at least two different modes, wherein at least: The first modal feature is a time-series feature constructed based on the PPG signal segment, including the original time-series signal and its derivative features and frequency domain features; The second modal feature is the image feature obtained by mapping the PPG signal segment into a two-dimensional time-series image; S3. Perform feature encoding on the first modal feature and the second modal feature respectively to obtain temporal feature representation and image feature representation; S4. Based on the cross-modal cross-attention mechanism, the temporal feature representation and image feature representation are subjected to bidirectional information interaction and fusion to obtain cross-modal fused features; S5. Jointly model the cross-modal fusion features with individual structured features and output the corresponding blood glucose prediction value.

2. The blood glucose detection method based on PPG cross-modal cross-attention fusion network according to claim 1, characterized in that: The preprocessing steps include: filtering, denoising, segmenting, and normalizing the raw PPG signal.

3. The blood glucose detection method based on PPG cross-modal cross-attention fusion network according to claim 1, characterized in that: The time-series features also include at least one of the following: time-domain statistical features, time-frequency domain features, nonlinear features, and periodic features.

4. The blood glucose detection method based on PPG cross-modal cross-attention fusion network according to claim 1, characterized in that: The method of mapping PPG signal segments into two-dimensional time-series images includes: converting one-dimensional time-series signals into two-dimensional grayscale images, spectrum images, time-frequency images, or Gram angle field images.

5. The blood glucose detection method based on PPG cross-modal cross-attention fusion network according to claim 1, characterized in that: The cross-modal cross-attention mechanism includes: The temporal feature representation is used as the query vector, and the image feature representation is used as the key vector for the first attention calculation. The image feature representation is used as the query vector, and the temporal feature representation is used as the key vector for second attention calculation. The features from the two attention outputs are weighted and fused to obtain cross-modal fusion features.

6. The blood glucose detection method based on PPG cross-modal cross-attention fusion network according to claim 1, characterized in that: The individual structured characteristics include at least one of the following: age, sex, body mass index, history of diabetes, blood pressure, history of heart disease, and medication use.

7. The blood glucose detection method based on PPG cross-modal cross-attention fusion network according to claim 1, characterized in that: The feature encoding step includes: Temporal features are encoded using a temporal encoder, which includes a one-dimensional convolutional neural network and a long short-term memory network. Image features are encoded using an image encoder, which includes a two-dimensional convolutional neural network and a residual network.

8. The blood glucose detection method based on PPG cross-modal cross-attention fusion network according to claim 7, characterized in that: The image encoder employs a multi-scale feature extraction structure, including multiple convolutional layers or attention layers of different scales, to extract local and global features from the image.

Images