Non-invasive blood glucose prediction method and device based on multi-modal sensor data fusion

Abstract

The present application relates to a kind of non-invasive blood glucose prediction method and equipment based on multi-modal sensor data fusion, method includes: obtaining radio frequency reflectogram as main modal data, and obtaining including photoelectric plethysmogram signal, inertial measurement unit signal, temperature signal, humidity signal and pressure signal as auxiliary modal data;Radio frequency spectrum feature and auxiliary modal feature are extracted and fused;The fusion feature is input into modal correction amount calculation module containing gate branch, residual correction branch and learnable scaling parameter, generates radio frequency spectrum feature correction amount;Original radio frequency spectrum feature is added with the correction amount, to obtain the radio frequency spectrum feature after calibration;Final output continuous blood glucose concentration prediction value.The present application is by using auxiliary modal for the dynamic calibration and compensation of radio frequency spectrum feature instead of directly participating in blood glucose prediction, maintains the specificity of radio frequency reflectogram to blood glucose, improves the robustness of model to movement, environmental temperature and humidity and the change of contact state and blood glucose prediction accuracy.

Description

Technical Field

[0001] This invention relates to the field of blood glucose monitoring technology, and in particular to a non-invasive blood glucose prediction method and device based on multimodal sensor data fusion. Background Technology

[0002] Non-invasive blood glucose monitoring is crucial for diabetes management. Radiofrequency microwave reflectance spectroscopy-based blood glucose detection technology achieves non-invasive sensing by measuring changes in the reflectance coefficient caused by variations in tissue dielectric properties with blood glucose concentration. Acquiring multi-point reflectance intensity parameters in the 4 GHz to 6 GHz frequency band has been extensively studied due to its sensitivity to blood glucose changes and suitable tissue penetration.

[0003] However, radio frequency (RF) reflected signals are highly susceptible to interference from non-glucose factors such as motion artifacts, sensor-skin contact pressure, ambient temperature and humidity, and skin blood perfusion status. A single RF mode cannot guarantee prediction accuracy under different application scenarios. To enhance anti-interference capabilities, existing solutions have introduced auxiliary modes such as photoplethysmography (PPG) sensors, inertial measurement units, temperature sensors, humidity sensors, and pressure sensors, attempting to compensate for the aforementioned disturbances through multi-sensor information fusion.

[0004] The common multimodal fusion approach currently involves directly concatenating radio frequency reflectance spectrum features with data from various auxiliary sensors into a single feature vector, which is then input into a deep learning regression model. The model then learns the mapping from the concatenated features to blood glucose values. This direct fusion method has significant drawbacks: the auxiliary modalities contain numerous perturbation patterns unrelated to blood glucose. When the feature dimension is too high and the modal contributions are unbalanced, the model tends to rely on interference-related features from the auxiliary modalities for inference, thus weakening the dominant role of the radio frequency reflectance spectrum in its intrinsic sensitivity to blood glucose. This results in poor robustness to motion, contact pressure, and environmental changes, leading to drift in predicted blood glucose values ​​and failing to meet the accuracy requirements of non-invasive continuous blood glucose monitoring. Furthermore, the concatenation fusion method lacks explicit decoupling and compensation mechanisms for interference factors, failing to guarantee that the radio frequency features retain their specificity for blood glucose after fusion.

[0005] In summary, there is an urgent need for a multimodal fusion method that can utilize auxiliary sensor information to perform targeted dynamic calibration and compensation of radio frequency reflectance spectrum characteristics while maintaining the sensitivity of the radio frequency master mode to blood glucose, so as to improve the accuracy and environmental robustness of non-invasive blood glucose prediction. Summary of the Invention

[0006] The purpose of this invention is to overcome the defects of the prior art by providing a non-invasive blood glucose prediction method and device based on multimodal sensor data fusion, so as to solve or partially solve the problem that the existing direct splicing fusion method easily leads to the weakening of the specificity of radio frequency reflection spectrum for blood glucose by non-blood glucose perturbation factors in the auxiliary mode, resulting in poor model robustness and prediction drift.

[0007] The objective of this invention can be achieved through the following technical solutions: One aspect of the present invention provides a non-invasive blood glucose prediction method based on multimodal sensor data fusion, comprising the following steps: The radio frequency resonant reflection spectrum is acquired as the main mode data, which includes radio frequency reflection intensity parameters at multiple frequency points collected in the frequency range of 4 GHz to 6 GHz; Acquire auxiliary modal data, which includes photoplethysmography (PPG) signals, inertial measurement unit (IMU) signals, temperature signals, humidity signals, and pressure signals; Radio frequency (RF) spectral features are extracted from the radio frequency resonant reflection spectrum, auxiliary mode features are extracted from the auxiliary mode data, and the RF spectral features and the auxiliary mode features are fused to obtain fused features; The fused features are input into the modal correction calculation module to generate a correction amount for the radio spectrum features. The modal correction calculation module includes a gated branch for outputting the correction intensity gate value of each feature dimension, a residual correction branch for outputting the correction direction, and a learnable scaling parameter for adjusting the correction amplitude. The correction amount is determined by the learnable scaling parameter, the output of the gated branch, and the output of the residual correction branch. The original RF characteristics are added to the correction amount to obtain the calibrated RF characteristics; Based on the calibrated spectral characteristics, the blood glucose numerical regression module outputs continuous blood glucose concentration prediction values.

[0008] As a preferred technical solution, the extraction process of the auxiliary modal features includes the following steps: Photoplethysmography (PPG) features were extracted from the PPG signal window to obtain the first feature characterizing blood perfusion status and motion artifacts. Inertial measurement unit features are extracted from the acceleration and / or angular velocity data windows to obtain a second feature characterizing motion intensity and attitude change; Environmental and contact state features were extracted from temperature, humidity, and pressure data to obtain a third feature characterizing the environmental state and the degree of sensor contact. The first feature, the second feature, and the third feature are spliced ​​or fused to obtain the auxiliary modal feature.

[0009] As a preferred technical solution, the gated branch includes a multilayer perceptron and a sigmoid activation function to output a gated vector with a value range of (0,1), and the residual correction branch includes a multilayer perceptron and a tanh activation function to output a residual correction vector with a value range of (-1,1). The learnable scaling parameter is a scalar or a vector, and the correction amount is obtained by element-wise multiplication of the learnable scaling parameter, the gated vector, and the residual correction vector.

[0010] As a preferred technical solution, during the training phase, the loss function includes a regression loss term for measuring the error between the predicted continuous blood glucose concentration and the reference value of the continuous blood glucose monitoring device, and a correction regularization term for constraining the magnitude of the correction.

[0011] As a preferred technical solution, when a certain auxiliary modality data is missing, the missing modality is processed by zero-filling, mean-filling, setting a missing flag, or modality dropout.

[0012] As a preferred technical solution, the obtained radio frequency reflection intensity parameters are preprocessed, including outlier removal, smoothing, baseline correction and normalization.

[0013] As a preferred technical solution, the extraction of radio frequency spectrum features is achieved through a one-dimensional convolutional neural network. The one-dimensional convolutional neural network is used to learn local peak and valley changes, resonant frequency shift, spectral slope, reflection intensity attenuation mode, local frequency band response differences, or overall trend of change across the entire frequency band in the radio frequency reflection spectrum.

[0014] As a preferred technical solution, when acquiring the auxiliary mode data, the auxiliary sensor data within the corresponding time window is extracted according to the acquisition time or acquisition time interval of the radio frequency resonant reflection spectrum, and the temporal change information of the auxiliary sensor data is retained.

[0015] As a preferred technical solution, the radio frequency reflection intensity parameter is the reflection amplitude, reflection loss, reflection coefficient magnitude, logarithmic reflection intensity, or a reflection intensity value after normalization.

[0016] In another aspect, an electronic device is provided, including one or more processors, a memory, and one or more programs stored in the memory, said one or more programs including instructions for executing the aforementioned non-invasive blood glucose prediction method based on multimodal sensor data fusion.

[0017] Compared with the prior art, the present invention has at least one of the following beneficial effects: (1) Improve the accuracy of blood glucose prediction: In the existing direct splicing and fusion method, a large number of perturbation modes in the auxiliary mode that are not related to blood glucose easily weaken the specificity of radio frequency reflection spectrum for blood glucose, resulting in poor model robustness and easy drift of predicted values. This invention adopts a mode correction calculation module composed of gated branches, residual correction branches and learnable scaling parameters to perform gated residual calibration on the radio frequency spectrum features, strictly limiting the contribution of the auxiliary mode to the dynamic calibration and compensation of the radio frequency spectrum features, thereby maintaining the core position of radio frequency reflection spectrum as the blood glucose-sensitive master mode and significantly improving the accuracy of blood glucose prediction.

[0018] (2) Compensation for disturbance factors: In view of the problem that existing methods lack explicit decoupling and compensation mechanisms for non-glucose interference factors such as motion artifacts, contact pressure, and ambient temperature and humidity, and cannot guarantee the specificity of radio frequency features, this invention uses gated branches to determine the dimensions that need to be corrected in the radio frequency features, uses residual correction branches to estimate the correction direction, and adjusts the correction amplitude through learnable scaling parameters. The combination of these three factors generates a targeted feature correction amount, realizing explicit modeling and fine compensation for non-glucose disturbance factors. This enables the model to adaptively eliminate interference according to the current sensor state, ensuring that the calibrated radio frequency features can reflect changes in blood glucose concentration more stably and specifically.

[0019] (3) Enhanced model training stability and generalization ability: In the direct splicing multimodal fusion structure, when the auxiliary modality dimension is too high and the contribution is unbalanced, the model is prone to rely on auxiliary features for inference and ignore the main modality information. To address this issue, this invention introduces a regularization loss term for the magnitude of the correction during training to constrain the correction amplitude of the auxiliary modality to the radio frequency feature, prevent the model from generating excessive radio frequency feature correction, and ensure that blood glucose prediction is always based on radio frequency information, with auxiliary information only making reasonable compensation, thereby enhancing the stability and generalization ability of model training. Attached Figure Description

[0020] Figure 1 This is a flowchart of the non-invasive blood glucose prediction method based on multimodal sensor data fusion in the embodiment; Figure 2 This is a schematic diagram of a non-invasive blood glucose prediction system based on multimodal sensor data fusion in the embodiment. Figure 3 This is a schematic diagram of the Clarke error network analysis results in the example; Figure 4 This is a schematic diagram of the electronic device in the embodiment. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] Example 1 To address the problems existing in the aforementioned prior art, this embodiment provides a non-invasive blood glucose prediction method based on multimodal sensor data fusion. This method uses radio frequency resonant reflection spectrum as the main mode of blood glucose sensing, and one or more of photoplethysmography (PPG), inertial measurement unit (IMU), temperature sensor, humidity sensor, and pressure sensor as auxiliary modes. The radio frequency reflection intensity parameters are dynamically calibrated and compensated through the auxiliary modes to obtain calibrated RF spectrum characteristics, and blood glucose concentration prediction values ​​are output based on these calibrated characteristics.

[0023] See Figure 1 The method includes the following steps: Step S1: Obtain the radio frequency resonant reflection spectrum as the main mode data.

[0024] The radio frequency reflectance spectrum ranges from 4 GHz to 6 GHz, and radio frequency reflectance intensity parameters at 1001 frequency points within this band are collected. The reference blood glucose label is derived from a continuous glucose monitoring device (CGM), with blood glucose units expressed in mmol / L. The model's final output is a continuous blood glucose concentration value, which falls under the regression prediction task.

[0025] For a single blood glucose prediction process, the radio frequency sensing unit acquires the radio frequency reflection spectrum in the frequency range of 4 GHz to 6 GHz. Let the... The radio spectrum obtained from the second acquisition for: in, Indicates the first The radio frequency reflection intensity parameters corresponding to each frequency point. The radio frequency reflection intensity parameters can be reflection amplitude, reflection loss, reflection coefficient magnitude, logarithmic reflection intensity, or reflection intensity values ​​after normalization.

[0026] Radio frequency points can be represented as: Thus, each RF measurement generates a one-dimensional spectral input vector of length 1001. If the device simultaneously obtains phase, complex reflection coefficient, or other RF parameters, it can also be extended to a multi-channel spectral input. In this embodiment, the RF reflection intensity parameter is preferably used as the primary input.

[0027] Step S2: Obtain auxiliary modal data.

[0028] The auxiliary modes include PPG signals, IMU motion signals, temperature signals, humidity signals, and pressure signals. Since the selected PPG, IMU, temperature, humidity, and pressure sensors can be common commercial sensors, their sampling rates may vary depending on the sensor model, driving method, or system configuration. Therefore, this solution does not limit the fixed sampling rate of the auxiliary sensors.

[0029] For a single RF spectrum acquisition, instead of interpolating the auxiliary sensor signal to a single time point value, a corresponding auxiliary sensor time window is extracted based on the RF spectrum acquisition time or time interval. Let the... The acquisition time interval for the secondary emission spectrum is: in , These are the data collection start point and the data collection stop point, respectively.

[0030] Then the corresponding auxiliary modal data for: in, This represents the raw waveform data of PPG within this time interval. This represents the acceleration, angular velocity, or motion state data within that time interval. Representing temperature data, Represents humidity data. This indicates pressure or contact status data.

[0031] In another implementation, the time of the radio spectrum acquisition center can also be used. Based on this, the following time window is extracted: in, and It is a configurable time parameter used to cover changes in auxiliary sensors related to the measurement state before and after RF acquisition.

[0032] This design preserves the temporal variation information of the auxiliary sensors, enabling the model to identify motion artifacts, pressure fluctuations, temperature and humidity drift, and changes in PPG perfusion status, rather than using only static auxiliary sensor readings at a single point in time.

[0033] Step S3: Extract RF spectrum features from the RF resonant reflection spectrum, extract auxiliary mode features from the auxiliary mode data, and fuse the RF spectrum features with the auxiliary mode features to obtain fused features.

[0034] The radio frequency spectrum feature extraction module is used to extract glucose-related features from the radio frequency reflection intensity sequence of 1001 frequency points from 4GHz to 6GHz. This module can employ a one-dimensional convolutional neural network, with the following input: After adding the channel dimension, the input can also be represented as: in, This represents the number of spectral channels. When using only reflection intensity... .

[0035] The RF feature extraction module can sequentially include a one-dimensional convolutional layer, a batch normalization layer, a ReLU activation layer, a pooling layer, several convolutional feature extraction blocks, a global average pooling layer, and a fully connected layer. Its output is an RF feature vector. : in, This represents a radio frequency feature extraction network. , denoted as the dimension of the spectral eigenvector.

[0036] The radio frequency (RF) spectral feature extraction module is used to learn local peak and valley variations, resonant frequency shifts, spectral slope, reflection intensity attenuation modes, local frequency band response differences, and overall trends across the entire frequency band in the RF reflection spectrum. These features are correlated with the dielectric properties, water content changes, tissue state changes, and blood glucose concentration changes of the tested tissue.

[0037] The auxiliary modal feature extraction module includes a PPG feature extraction submodule, an IMU feature extraction submodule, and an environment and contact state feature extraction submodule.

[0038] PPG feature extraction receives the raw PPG waveform window and extracts PPG features using a one-dimensional convolutional layer, batch normalization layer, ReLU activation layer, Dropout layer, and pooling layer. : in, This indicates the PPG feature extraction submodule.

[0039] This feature is used to characterize factors such as blood perfusion status, pulse waveform changes, skin contact status, and the intensity of motion artifacts.

[0040] The IMU feature extraction submodule receives data from the accelerometer and / or gyroscope windows and extracts features such as motion intensity, jitter state, attitude change, and degree of stillness. : in, This indicates the IMU feature extraction submodule.

[0041] Temperature, humidity, and pressure data can be jointly input into the environmental and contact state feature extraction submodule. This submodule can first calculate the mean, variance, maximum, minimum, rate of change, slope, and stability indices within a time window, and then map them to environmental state features through a multilayer perceptron; alternatively, it can directly input these data as time series data into a one-dimensional convolutional network. Its output is: in, This module represents the extraction of environmental and contact state features. Temperature and humidity are used to characterize changes in the external environment and skin surface conditions, while pressure is used to characterize the degree of adhesion between the sensor and human skin, contact stability, and measurement consistency.

[0042] Then, the auxiliary modal features are spliced ​​or fused to obtain unified auxiliary modal features. : in, This represents the comprehensive characterization of the current measurement state by the auxiliary modes. It indicates splicing or merging.

[0043] Step S4: Input the fused features into the modal correction calculation module to generate a correction amount for the RF spectrum features.

[0044] The modal correction calculation module includes a gated branch for outputting the correction intensity gate value for each feature dimension, a residual correction branch for outputting the correction direction, and a learnable scaling parameter for adjusting the correction magnitude. The correction amount is determined by the learnable scaling parameter, the output of the gated branch, and the output of the residual correction branch.

[0045] The multimodal fusion module fuses RF features and auxiliary modal features. Specifically, it fuses RF features... With auxiliary modal features By splicing the data, the fused features are obtained. : in, This indicates a fusion feature that includes glucose-related radio frequency information and non-glucose perturbation state information. Indicates splicing.

[0046] In this embodiment, the auxiliary sensor data does not directly replace the radio frequency spectrum for blood glucose prediction. Instead, it is used to estimate the interference caused by the current measurement state to the radio frequency reflection intensity parameter and further generate correction values ​​for the radio frequency spectrum characteristics. Therefore, the radio frequency spectrum remains the primary mode for blood glucose prediction, while the auxiliary sensor serves as the calibration and compensation mode.

[0047] Step S5: Add the original RF characteristics to the correction amount to obtain the calibrated RF characteristics.

[0048] The modal correction calculation module is used to calculate the modal correction based on the fusion features. Calculate the calibration compensation for the spectral characteristics. This module includes a gated branch, a residual correction branch, and a learnable scaling parameter.

[0049] Gated branches are used to determine which dimensions of the spectral features need to be corrected. Gated branches may include multilayer perceptrons. LayerNorm and Sigmoid activation function Its output is: in, , representing the correction strength gating value for each RF feature dimension. If the current measurement is affected by significant motion, pressure instability, temperature and humidity changes, or abnormal PPG infusion, the gating branch can increase the correction weight of the relevant dimension. When the measurement state is stable, the gating branch suppresses unnecessary corrections.

[0050] The residual correction branch is used to estimate the correction direction. This branch may include the multilayer perceptron. LayerNorm and tanh activation function Its output is: in, This is used to indicate whether each dimension of the spectral characteristics should be positively or negatively compensated.

[0051] Learnable scaling parameters are used to limit or adjust the magnitude of the correction. They can be scalars or vectors. Finally, the modal correction is calculated as follows: in, This indicates element-wise multiplication.

[0052] This structure enables the model to learn three things simultaneously: whether calibration is needed, which RF feature dimensions to calibrate, and the magnitude of the calibration.

[0053] The gated residual calibration module adds the original RF characteristics to the modal correction to obtain the calibrated RF characteristics: in, This indicates the spectral characteristics after calibration with an auxiliary sensor.

[0054] The function of this module is to use auxiliary information such as PPG, IMU, temperature, humidity and pressure to compensate for non-glucose interference components in the radio frequency reflection intensity parameter, so that the calibrated feature more stably reflects changes in blood glucose concentration.

[0055] Step S6: Based on the calibrated spectral characteristics, output the continuous blood glucose concentration prediction value through the blood glucose numerical regression module.

[0056] The blood glucose value regression module receives the calibrated radio spectrum characteristics. Through multilayer perceptron Output continuous blood glucose concentration prediction values: in, The unit for predicting blood glucose levels is mmol / L.

[0057] The regression module may include a linear layer, a LayerNorm layer, a ReLU activation layer, a Dropout layer, and a final linear output layer. Since the final task in this embodiment is numerical regression, the model output layer is a single neuron output used to predict continuous blood glucose concentration.

[0058] The training phase process will be explained below: (1) Collect multimodal sensor data and CGM reference blood glucose label.

[0059] For the Secondary spectral sampling to construct training samples: in, It is a sequence of radio frequency reflection intensities containing 1001 frequency points in the range of 4GHz to 6GHz; Reference blood glucose values ​​provided for CGM, in units of .

[0060] Since CGM (Continuous Glucose Monitoring) data is continuous, its temporal resolution is typically lower than that of some auxiliary sensors. Therefore, during training sample construction, the spectral samples can be matched with the CGM blood glucose values ​​at the closest time point or corresponding time period based on the timestamp. In an optional implementation, a configurable time offset can be introduced based on the physiological lag between glucose in human tissue fluid and blood glucose to further optimize the label alignment.

[0061] (2) Radio spectrum preprocessing.

[0062] 1. Unify the radio spectrum in the range of 4GHz to 6GHz into 1001 frequency points to ensure that different samples have the same input length.

[0063] 2. Perform outlier removal and smoothing on the RF reflection intensity parameters to reduce occasional measurement noise.

[0064] 3. Perform baseline correction and normalization on the irradiated spectrum. The normalization method can be in-sample normalization, global normalization, minimum-maximum normalization, logarithmic transformation, or differential processing relative to the reference spectrum.

[0065] The preprocessed radio spectrum is represented as follows: This vector serves as the input to the radio spectrum feature extraction module.

[0066] (3) Auxiliary sensor preprocessing.

[0067] For PPG, IMU, temperature, humidity and pressure sensors, since their sampling rate is not limited, a time window is first extracted based on the timestamp.

[0068] PPG signal preprocessing may include detrending, filtering, outlier removal, amplitude normalization, and fixed-length resampling. The processed PPG time window is then input into the PPG feature extraction submodule.

[0069] IMU signal preprocessing may include outlier removal, triaxial acceleration and triaxial angular velocity normalization, motion intensity calculation, modulus calculation, and fixed-length resampling. The processed IMU time window is used to reflect the motion state during the current measurement process.

[0070] Preprocessing of temperature, humidity, and pressure data can include outlier removal, missing value imputation, normalization, and calculation of statistical features over time windows. Pressure data can also be used to determine whether the sensor is in stable contact with the skin.

[0071] When an auxiliary mode is missing, zero-filling, mean-filling, missing flags, or mode dropout can be used to handle the situation, enabling the model to adapt to different commercial sensor configurations and situations where some sensors fail.

[0072] (4) Feature extraction and fusion training.

[0073] Input the preprocessed RF spectrum into the RF feature extraction module: Input the preprocessed auxiliary modality data into the corresponding feature extraction module: The fusion yields auxiliary modal features: Then, the radio frequency features and auxiliary modal features are concatenated: (5) Calibration and compensation training.

[0074] Fusion features Input the gated branch and the residual correction branch, and you get: Further calculation of the spectral characteristic correction: Then the correction is superimposed on the original spectral features: This process enables the model to automatically compensate for disturbances in the RF reflection intensity parameters caused by changes in motion, temperature, humidity, pressure, and PPG perfusion status, based on the status of auxiliary sensors. For example, when IMU data shows jitter during measurement, the model can increase the correction weight of motion-related disturbance dimensions through gating branches; when pressure data indicates unstable sensor adhesion, the model can perform residual correction on RF spectral features that are significantly affected by contact conditions; and when temperature or humidity changes, the model can compensate for RF reflection intensity drift based on environmental conditions.

[0075] (6) Blood glucose regression training.

[0076] Input the calibrated spectral characteristics into the blood glucose value regression module: in, This represents the blood glucose concentration predicted by the model, in units of... .

[0077] The primary goal of model training is to minimize the error between the predicted blood glucose value and the CGM reference blood glucose value. The loss function can be the mean squared error loss, the mean absolute error loss, or the Huber loss.

[0078] For example, when using mean squared error loss: in, For batch size, This is the reference blood glucose value for CGM.

[0079] To avoid the model generating excessively large spectral corrections, a correction regularization term can be added: The final training loss can be expressed as: in, This is a regularization weight for the correction. This regularization term limits the excessive modification of the RF spectrum characteristics by the auxiliary modes, ensuring the model maintains its structural characteristics of being primarily based on the RF spectrum with auxiliary sensors providing compensation.

[0080] (7) Training process.

[0081] The training process in this embodiment may include the following steps: 1. Collect radio frequency reflection intensity data at 1001 frequency points in the range of 4GHz to 6GHz, and simultaneously collect PPG, IMU, temperature, humidity and pressure data, and obtain CGM blood glucose tags.

[0082] 2. Construct training samples based on timestamps, and match each radio spectrum data with the auxiliary sensor data and CGM blood glucose label within the corresponding time window.

[0083] 3. Perform outlier removal, smoothing, baseline correction, and normalization on the radio frequency reflection intensity data.

[0084] 4. Perform time window truncation, outlier handling, normalization, and fixed-length conversion on PPG, IMU, temperature, humidity, and pressure data.

[0085] 5. Input the RF spectrum into the RF feature extraction module to obtain the RF features.

[0086] 6. Input the auxiliary sensor data into the auxiliary modal feature extraction module to obtain the auxiliary modal features.

[0087] 7. The spectral features and auxiliary modal features are concatenated to obtain the fused features.

[0088] 8. Calculate the gating vector and residual correction vector based on the fusion features to obtain the RF spectrum feature correction amount.

[0089] 9. Add the RF feature correction amount to the original RF feature to obtain the calibrated RF feature.

[0090] 10. Input the calibrated spectral characteristics into the blood glucose value regression module, and output the predicted blood glucose value in mmol / L.

[0091] 11. Update the model parameters based on the error between the predicted blood glucose value and the CGM label until the model converges.

[0092] In practical use, the radio frequency reflection spectrum within the current 4GHz to 6GHz range is first collected to form 1001 frequency points of radio frequency reflection intensity input. At the same time, PPG, IMU, temperature, humidity and pressure data within the current measurement time window are collected.

[0093] The model first extracts spectral features and auxiliary modal features. Then, based on the auxiliary modal features, it calculates the gated residual correction of the spectral features to obtain the calibrated spectral features. Finally, the model inputs the calibrated spectral features into the regression module and outputs the current blood glucose concentration prediction value in mmol / L.

[0094] The reasoning process does not require manual setting of temperature compensation coefficients, humidity compensation coefficients, pressure compensation coefficients, or motion compensation coefficients. Instead, the trained neural network automatically completes the radio spectrum calibration based on the current state of the multimodal sensor.

[0095] To verify the effectiveness of the method, comparative experiments were conducted on a test set. The test dataset covered various ambient temperatures ranging from 15°C to 35°C, humidity levels ranging from 30% to 80%, everyday motion interference such as stillness, walking, and arm waving, and complex multimodal scenarios including varying skin contact pressures. The mean relative absolute difference (MARD) results for each scheme in blood glucose monitoring are as follows: Traditional pure radio frequency scheme 1 or scheme 2 with manually set compensation coefficients exhibited severe signal distortion when faced with motion artifacts and drastic temperature and humidity changes, leading to a significant spike in MARD. The average MARD for schemes 1 and 2 across all scenarios were 27.4% and 16.8%, respectively. In contrast, the scheme corresponding to the proposed method achieved a comprehensive MARD of 9.1% across all scenarios without requiring any manually set compensation coefficients, demonstrating robustness under interference environments. Figure 3 The results of Clarke error network analysis are shown, where the horizontal axis represents the reference glucose value and the vertical axis represents the predicted glucose value, both in mmol / L. In the figure, (a) Scheme 1 is the uncompensated scheme, (b) Scheme 2 is the traditional artificial compensation scheme, and (c) Scheme 3 is the scheme of this application. As can be seen from (c), the data point distribution of this application in the Clarke error grid is as follows: Area A, i.e., the clinical accuracy area, accounts for 89.5%, which is an improvement of 18.2% compared with the traditional artificial compensation scheme (Scheme 2) and an improvement of 21.7% compared with the uncompensated scheme (Scheme 1).

[0096] Example 2 Based on Example 1, this example provides a non-invasive blood glucose prediction system based on multimodal sensor data fusion to implement the non-invasive blood glucose prediction method of Example 1. (See also...) Figure 2 The system includes: 1. Sensor module, used to collect main mode data and auxiliary mode data. The main mode data is the radio frequency resonant reflection spectrum in the frequency range of 4GHz to 6GHz. The auxiliary mode data includes photoplethysmography pulse wave signal, inertial measurement unit signal, temperature signal, humidity signal and pressure signal.

[0097] 2. Radio frequency spectrum feature extraction module, used to extract radio frequency spectrum features from radio frequency resonant reflection spectrum.

[0098] 3. Auxiliary modal feature extraction module, used to extract auxiliary modal features from auxiliary modal data.

[0099] 4. A multimodal fusion module is used to splice the radio spectrum features with the auxiliary modal features to obtain fused features.

[0100] 5. Modal correction calculation module, used to generate correction amounts for the radio spectrum features based on the fusion features. The modal correction calculation module includes a gated branch for outputting the correction intensity gate value of each feature dimension, a residual correction branch for outputting the correction direction, and a learnable scaling parameter for adjusting the correction amplitude. The correction amount is determined by the element-wise product of the learnable scaling parameter, the output of the gated branch, and the output of the residual correction branch.

[0101] 6. A gated residual calibration module is used to add the original RF characteristics to the correction amount to obtain the calibrated RF characteristics.

[0102] 7. Blood glucose numerical regression module, used to output continuous blood glucose concentration prediction values ​​based on calibrated radio spectrum characteristics.

[0103] This invention incorporates auxiliary sensor data such as PPG, IMU, temperature, humidity, and pressure into the radio frequency characteristic calibration process, enabling the model to identify and compensate for non-glucose disturbance factors in the radio frequency reflection intensity parameter.

[0104] Compared to methods that directly concatenate RF and auxiliary sensor data for blood glucose regression, this approach employs a structure with gated branches, residual correction branches, and learnable scaling parameters. This allows the auxiliary mode to primarily modify RF features rather than directly dominating blood glucose prediction. Therefore, this method maintains the RF spectrum as the core of the blood glucose-sensitive master mode while improving the model's robustness to changes in motion, contact pressure, ambient temperature and humidity, and skin perfusion status.

[0105] Using radio frequency reflectance spectra at 1001 frequency points from 4 GHz to 6 GHz as input and CGM blood glucose values ​​as supervisory labels, the model can output continuous blood glucose prediction values ​​in mmol / L, thereby achieving non-invasive blood glucose measurement.

[0106] Example 3 Based on the foregoing embodiments, this embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, the one or more programs including instructions for executing the non-invasive blood glucose prediction method based on multimodal sensor data fusion as described in Embodiment 1.

[0107] like Figure 4 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The method described herein. Of course, in addition to software implementation, this invention does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0108] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0109] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0110] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A non-invasive blood glucose prediction method based on multimodal sensor data fusion, characterized in that, Includes the following steps: The radio frequency resonant reflection spectrum is acquired as the main mode data, which includes radio frequency reflection intensity parameters at multiple frequency points collected in the frequency range of 4 GHz to 6 GHz; Acquire auxiliary modal data, which includes photoplethysmography (PPG) signals, inertial measurement unit (IMU) signals, temperature signals, humidity signals, and pressure signals; Radio frequency (RF) spectral features are extracted from the radio frequency resonant reflection spectrum, auxiliary mode features are extracted from the auxiliary mode data, and the RF spectral features and the auxiliary mode features are fused to obtain fused features; The fused features are input into the modal correction calculation module to generate a correction amount for the radio spectrum features. The modal correction calculation module includes a gated branch for outputting the correction intensity gate value of each feature dimension, a residual correction branch for outputting the correction direction, and a learnable scaling parameter for adjusting the correction amplitude. The correction amount is determined by the learnable scaling parameter, the output of the gated branch, and the output of the residual correction branch. The original RF characteristics are added to the correction amount to obtain the calibrated RF characteristics; Based on the calibrated spectral characteristics, the blood glucose numerical regression module outputs continuous blood glucose concentration prediction values.

2. The non-invasive blood glucose prediction method based on multimodal sensor data fusion according to claim 1, characterized in that, The extraction process of the auxiliary modal features includes the following steps: Photoplethysmography (PPG) features were extracted from the PPG signal window to obtain the first feature characterizing blood perfusion status and motion artifacts. Inertial measurement unit features are extracted from the acceleration and / or angular velocity data windows to obtain a second feature characterizing motion intensity and attitude change; Environmental and contact state features were extracted from temperature, humidity, and pressure data to obtain a third feature characterizing the environmental state and the degree of sensor contact. The first feature, the second feature, and the third feature are spliced ​​or fused to obtain the auxiliary modal feature.

3. The non-invasive blood glucose prediction method based on multimodal sensor data fusion according to claim 1, characterized in that, The gated branch includes a multilayer perceptron and a sigmoid activation function, used to output a gated vector with a value range of (0,1). The residual correction branch includes a multilayer perceptron and a tanh activation function, used to output a residual correction vector with a value range of (-1,1). The learnable scaling parameter is a scalar or a vector. The correction amount is obtained by element-wise multiplication of the learnable scaling parameter, the gated vector, and the residual correction vector.

4. The non-invasive blood glucose prediction method based on multimodal sensor data fusion according to claim 1, characterized in that, During the training phase, the loss function includes a regression loss term that measures the error between the predicted continuous blood glucose concentration and the reference value of the continuous blood glucose monitoring device, and a correction regularization term that constrains the magnitude of the correction.

5. The non-invasive blood glucose prediction method based on multimodal sensor data fusion according to claim 1, characterized in that, When a certain auxiliary modality data is missing, the missing modality is handled by zero-filling, mean-filling, setting a missing flag, or modality dropout.

6. The non-invasive blood glucose prediction method based on multimodal sensor data fusion according to claim 1, characterized in that, The acquired radio frequency reflection intensity parameters are preprocessed, including outlier removal, smoothing, baseline correction, and normalization.

7. The non-invasive blood glucose prediction method based on multimodal sensor data fusion according to claim 1, characterized in that, The extraction of radio frequency spectral features is achieved through a one-dimensional convolutional neural network, which is used to learn local peak and valley variations, resonant frequency shifts, spectral slopes, reflection intensity attenuation modes, local frequency band response differences, or overall trend of changes across the entire frequency band in the radio frequency reflection spectrum.

8. The non-invasive blood glucose prediction method based on multimodal sensor data fusion according to claim 1, characterized in that, When acquiring the auxiliary mode data, the auxiliary sensor data within the corresponding time window is extracted according to the acquisition time or acquisition time interval of the radio frequency resonant reflection spectrum, and the temporal change information of the auxiliary sensor data is retained.

9. A non-invasive blood glucose prediction method based on multimodal sensor data fusion according to claim 1, characterized in that, The radio frequency reflection intensity parameters are reflection amplitude, reflection loss, reflection coefficient modulus, logarithmic reflection intensity, or reflection intensity value after normalization.

10. An electronic device, characterized in that, It includes one or more processors, memory, and one or more programs stored in the memory, said one or more programs including instructions for executing the non-invasive blood glucose prediction method based on multimodal sensor data fusion as described in any one of claims 1-9.

Images