Noninvasive blood glucose monitoring method and device based on multi-modal information

Abstract

The invention discloses a non-invasive blood glucose monitoring method and device based on multi-modal information. The method comprises the following steps: acquiring multi-modal physiological information of a target, capturing direct influence and instantaneous representation of blood glucose on the physiological information by utilizing a main model, and acquiring a first blood glucose predicted value and a corresponding first confidence interval; acquiring behavior information of the target, capturing long-term influence of different behaviors on blood glucose by using the auxiliary model, and acquiring a second blood glucose predicted value and a corresponding second confidence interval; the first blood glucose prediction value and the second blood glucose prediction value are subjected to weighted fusion, a fused blood glucose prediction value is obtained, and the weighted weight is determined according to the first confidence interval and the second confidence interval. According to the non-invasive blood glucose monitoring method, the causes and the fruits of the blood glucose change are modeled at the same time, and the accuracy of non-invasive blood glucose monitoring is improved.

Description

A non-invasive blood glucose monitoring method and device based on multimodal information Technical Field

[0001] This invention relates to the field of medical and health technology, and more specifically, to a non-invasive blood glucose monitoring method and device based on multimodal information. Background Technology

[0002] Diabetes can cause multi-system damage, leading to serious complications such as cardiovascular disease, kidney disease, neuropathy, and retinopathy, placing a heavy burden on individuals' quality of life and the social healthcare system. Active blood glucose monitoring is a fundamental aspect of diabetes prevention and precise management. While traditional finger-prick blood sampling is accurate, its invasiveness and inconvenience result in poor patient compliance. Implantable continuous glucose monitoring (ICM) allows for dynamic tracking, but it still requires consumable replacements, is costly, and poses risks such as skin allergies. Therefore, developing safe, convenient, and continuously measurable non-invasive blood glucose monitoring technologies has become a cutting-edge focus and strategic direction for diabetes prevention and control.

[0003] Currently, research on non-invasive blood glucose monitoring mainly focuses on optics, electronics, radio frequency, acoustics, and physiological signal sensing. Due to the complexity of human tissue structures and the diversity of monitoring environments, non-invasive blood glucose monitoring technologies based on single-modal information are susceptible to interference and have low accuracy. Therefore, employing multimodal information can more comprehensively capture the physiological response characteristics of blood glucose fluctuations and reduce interference from factors such as human physiological state and environmental changes, representing a key breakthrough for non-invasive monitoring technology to move from "theoretical verification" to "clinical applicability." However, existing multimodal information fusion methods typically use simple splicing or voting methods, failing to fully utilize the complementarity between modal information, thus limiting the accuracy of non-invasive blood glucose monitoring.

[0004] Non-invasive blood glucose monitoring technology based on multimodal information is a novel blood glucose detection method that integrates multi-source physiological data such as radio frequency signals, electrocardiogram signals, and photoplethysmography (PPG). This technology captures features related to blood glucose changes in different modal signals and combines them with auxiliary modeling of behavioral factors such as diet, exercise, medication, and sleep to construct a multimodal fusion model that dynamically reflects blood glucose fluctuations, thereby achieving high-precision continuous blood glucose monitoring without blood sampling. For example, in the prior art, patent application CN2025104877773 discloses a calibration method for non-invasive blood glucose monitors based on joint optimization, which can make the measurement data of non-invasive blood glucose monitors more accurate. Patent application CN2022107588856 discloses an optical non-invasive blood glucose concentration detection method based on time-spectrum-spatial multi-parameter fusion, realizing the establishment of a non-specific non-invasive general blood glucose prediction model and improving the accuracy of blood glucose concentration prediction through multi-parameter feature fusion.

[0005] Analysis reveals that existing non-invasive blood glucose monitoring technologies based on multimodal information rarely model both the "cause" (i.e., behaviors such as diet, exercise, medication, and sleep that cause changes in blood glucose) and the "effect" (i.e., changes in blood glucose that cause changes in physiological information such as radio frequency signals, electrocardiogram signals, and photoplethysmography pulse waves). Furthermore, the multimodal fusion process typically employs simple feature splicing or voting, failing to fully utilize the complementarity of different modal information and thus limiting the accuracy of non-invasive blood glucose monitoring. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a non-invasive blood glucose monitoring method and device based on multimodal information.

[0007] According to a first aspect of the present invention, a non-invasive blood glucose monitoring method based on multimodal information is provided. The method includes the following steps: acquiring multimodal physiological information of a target, and using a master model to capture the direct impact and instantaneous representation of blood glucose on the physiological information, obtaining a first predicted blood glucose value and a corresponding first confidence interval; acquiring behavioral information of the target, and using an auxiliary model to capture the long-term impact of different behaviors on blood glucose, obtaining a second predicted blood glucose value and a corresponding second confidence interval; and performing a weighted fusion of the first predicted blood glucose value and the second predicted blood glucose value to obtain a fused predicted blood glucose value, wherein the weighting weights are determined based on the first confidence interval and the second confidence interval.

[0008] According to a second aspect of the present invention, a non-invasive blood glucose monitoring device based on multimodal information is provided. The device includes: a first prediction unit for acquiring multimodal physiological information of a target and using a master model to capture the direct impact and instantaneous representation of blood glucose on physiological information, thereby obtaining a first predicted blood glucose value and a corresponding first confidence interval; a second prediction unit for acquiring behavioral information of the target and using an auxiliary model to capture the long-term impact of different behaviors on blood glucose, thereby obtaining a second predicted blood glucose value and a corresponding second confidence interval; and a fusion prediction unit for weighted fusion of the first and second predicted blood glucose values ​​to obtain a fused predicted blood glucose value, wherein the weighting weights are determined based on the first and second confidence intervals.

[0009] Compared with existing technologies, the advantages of this invention are that the non-invasive blood glucose monitoring method based on multimodal information is a joint modeling method based on physiological modal information such as radio frequency, electrocardiogram, and photoplethysmography (PPG) and behavioral modal information such as diet, medication, exercise, and sleep. By simultaneously modeling the "cause" (i.e., behaviors such as diet, exercise, medication, and sleep cause changes in blood glucose) and the "effect" (i.e., changes in blood glucose cause changes in physiological information such as radio frequency signals, electrocardiogram signals, and PPG), the accuracy of non-invasive blood glucose monitoring is improved.

[0010] Other features and advantages of the invention will become clear from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the invention and, together with their description, serve to explain the principles of the invention.

[0012] Figure 1 is a flowchart of a non-invasive blood glucose monitoring method based on multimodal information according to an embodiment of the present invention; Figure 2 is a schematic diagram of the process of a non-invasive blood glucose monitoring method based on multimodal information according to an embodiment of the present invention. Detailed Implementation

[0013] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0014] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0015] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0016] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0017] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0018] As shown in Figures 1 and 2, the proposed non-invasive blood glucose monitoring method based on multimodal information includes the following steps: Step S1, for the target's multimodal physiological information, the master model is used to capture the direct impact and instantaneous characterization of blood glucose on physiological information, and to obtain the predicted blood glucose value and the corresponding confidence interval.

[0019] Multimodal physiological information can include radio frequency signals, electrocardiogram (ECG) signals, and photoplethysmography (PPG) waves. The master model, for example, primarily uses physiological information such as radio frequency signals, single-lead or multi-lead ECGs, and single-wavelength or multi-wavelength PPG waves as input to directly estimate the current blood glucose concentration. The master model mainly focuses on capturing the direct impact and instantaneous representation of blood glucose on physiological signals.

[0020] In one embodiment, the modeling process of the master model includes the following steps: Step S11: Collect multimodal physiological information and perform time segmentation.

[0021] For example, multimodal physiological information can be collected using wearable or desktop devices, including the amplitude and phase of radio frequency signals at different frequencies, single-lead or multi-lead electrocardiogram waveforms, and photoplethysmography (PPG) waveforms of a single wavelength or multiple wavelengths. Radio frequency signals can reflect subtle changes in tissue dielectric properties and water content with varying blood glucose levels. Electrocardiogram signals can reflect cardiac electrical activity and the state of the autonomic nervous system caused by changes in blood glucose levels. PPG waves can reflect changes in peripheral blood flow and vascular elasticity at different blood glucose concentrations. During data acquisition, the device's hardware clock or network time protocol is used to annotate the above modal information with a unified timestamp, with an error of <5 ms, achieving time alignment between different modalities. Each modal information is recorded at a fixed sampling rate and segmented using a sliding window method, for example, every 60 seconds as an analysis window, to ensure time synchronization.

[0022] Step S12: Construct a joint signal quality evaluation vector.

[0023] A single-mode Signal Quality Index (SQI) vector only reflects the independent quality of its own signal, while different modes often have coupling and complementary relationships. Therefore, in one embodiment, a joint SQI vector is constructed to evaluate the information of different modes. In the construction process, the following two types of class information are considered comprehensively: single-mode quality score and cross-modal consistency feature score.

[0024] For radio frequency (RF) signals, the combined SQI score is calculated as follows: 1) Single-mode quality score of RF signals: including the spectral signal-to-noise ratio, resonant peak clarity, phase continuity, amplitude stability, etc. of RF signals; 2) Cross-modal consistency score of RF signals: including the consistency between the fluctuation of RF signals and the heart rate changes of ECG signals, and whether the fluctuation of RF signals and the amplitude changes of photoplethysmography (PPG) waves are consistent in direction, etc.

[0025] For ECG signals, the combined SQI score is calculated as follows: 1) Single-mode quality score of ECG signals: including the success rate of R-peak detection of ECG signals, the similarity between QRS waveform and historical templates, baseline drift amplitude, power frequency interference ratio, etc.; 2) Cross-modal consistency feature score of ECG signals: including whether the delay of ECG signals and photoplethysmography pulse waves is stable, and the consistency between the heart rate changes of ECG signals and the fluctuations of radio frequency signals.

[0026] For photoplethysmography (PPG) signals, the joint SQI score is calculated as follows: 1) Single-mode quality score of PPG: including pulse peak detection rate, waveform similarity, spectral concentration, and multi-wavelength signal consistency; 2) Cross-modal consistency feature score of PPG: including whether the delay between PPG and ECG signals is stable, and whether the amplitude change of PPG and the fluctuation of RF signals are consistent in direction. Based on the above method, signal quality indices for RF signals, ECG signals, and PPG signals are calculated respectively. After normalization (0~1), each index is weighted and averaged to obtain the joint SQI score for each mode. For example, in the joint SQI score, 1 represents excellent signal quality, and 0 represents severe interference. The joint SQI scores for each mode are concatenated to construct the joint SQI vector, i.e.:

[0027] Step S13: Extract multimodal features.

[0028] When each element in the joint SQI vector is ≥0.5, it indicates that the acquired multimodal information initially meets the quality requirements, and feature extraction can be performed on each modality. Radio frequency (RF) signals are processed using a frequency-domain convolutional network to extract features such as resonant frequency, phase change, and dielectric response, which are then stored as RF feature vectors. Electrocardiogram (ECG) signals are processed using one-dimensional convolution or a temporal Transformer to extract features such as heart rate variability, QRS complex, and QTc, which are then stored as ECG feature vectors. Photoplethysmography (PPG) pulse wave signals are processed using convolutional networks of different scales to extract features such as pulse morphology and hemodynamics, which are then stored as photoelectric feature vectors. After feature extraction, these feature vectors are used as input to the main model. Simultaneously, the main model retains the joint SQI vector to dynamically adjust its weights during the fusion phase.

[0029] Step S14: Execute modal gating mechanism based on joint SQI vector.

[0030] The purpose of step S14 is to perform preliminary "credibility screening" of each modality's features based on the joint SQI vector before formally calculating cross-modal attention, ensuring that low-quality signals do not excessively interfere with the fusion result. For example, this can be achieved by using the joint SQI vector to generate gating coefficients for each modality through a lightweight neural network (such as two fully connected layers). A gating coefficient closer to 1 indicates a reliable modality, while a coefficient closer to 0 indicates an unreliable modality. These gating coefficients are applied directly to each modality's features before cross-modal calculation to scale the features, automatically reducing the weight of low-quality modalities during fusion; they can also be added as bias terms to the attention weight calculation to adjust the information flow ratio between different modalities. In this way, the main model already possesses "quality perception capability" before starting to calculate the cross-modal attention mechanism.

[0031] Step S15: Perform cross-modal attention fusion based on the joint SQI vector.

[0032] The main purpose of step S15 is to enable the interaction and fusion of various modal features in the cross-modal attention mechanism, and to dynamically adjust the attention weights by combining the joint SQI vector to achieve the optimal combination of different modal information.

[0033] For example, the specific operation process is as follows: The feature vector of each modality is linearly mapped to obtain: query vector, key vector, and value vector. The query vector represents "what information I want to obtain from other modalities", the key vector represents "what information I can provide", and the value vector represents "the actual information content I convey". When calculating the cross-modal attention weights, the gating coefficients generated in step S14 are applied to the key vectors, and the query vector is matched with the gated key vectors to calculate the attention weights. The value vectors are then optimized based on the attention weights to obtain the fused cross-modal feature representation. This cross-modal feature represents the most reliable and comprehensive blood glucose-related features at the current moment. The advantage of this cross-modal attention fusion based on joint SQI vectors is that even if the signal quality of a certain modality deteriorates at a specific moment (e.g., motion interference causes abnormalities in photoplethysmography pulse waves), the main model can automatically reduce its weight, thereby maintaining the overall prediction stability and accuracy.

[0034] Step S16: Estimate blood glucose and perform time smoothing.

[0035] The cross-modal features obtained in step S15 are input into the blood glucose regression unit (e.g., a lightweight Transformer or a temporal convolutional network) to output the blood glucose estimate at the current moment. To reduce the impact of short-term jitter, the main model can smooth the output over continuous time periods (e.g., exponential moving average) and output the predicted blood glucose value and confidence interval.

[0036] Step S2: For the target's behavioral information, use an auxiliary model to capture the long-term impact of different behaviors on blood glucose, and obtain the predicted blood glucose value and the corresponding confidence interval.

[0037] The auxiliary model primarily uses behavioral information such as diet, exercise, medication, and sleep as input, and maps these inputs to "medium- to long-term trends in blood glucose over time." Therefore, the auxiliary model mainly focuses on analyzing the long-term effects of different behaviors on blood glucose.

[0038] In one embodiment, modeling the auxiliary model includes the following steps: Step S21: Determine the input and output of the auxiliary model.

[0039] The auxiliary model takes into account behavioral information such as diet, exercise, medication, and sleep. Specifically, this includes: Diet: meal times, meal type estimation (high / medium / low glycemic index), carbohydrate / fat / protein intake per meal, etc.; Exercise: type, intensity, start / end time, etc.; Medications: medication type (rapid-acting insulin / long-acting / oral), dosage, administration time, etc.; Sleep: nighttime sleep start / end, nap start / end, sleep quality score, etc. Optional inputs include: historical reference blood glucose levels, and basic individual information (age, weight, diabetes type, insulin use history), etc. The auxiliary model outputs a blood glucose level estimate (a single numerical value) at any given time and a confidence interval.

[0040] Step S22: Preprocess the input events of the auxiliary model.

[0041] The purpose of step S22 is to standardize the raw events into a format usable by the model, achieving a unified format. For example, for each event such as diet, exercise, medication, and sleep, fixed fields are generated and their values ​​are standardized, such as carbohydrates / fat / protein in grams, exercise intensity in a unified unit, and sleep quality in a 0-1 scale. If only food images or text descriptions are available, carbohydrate content can be estimated using a food recognition or estimation module.

[0042] Step S23: Represent and encode the input events of the auxiliary model.

[0043] Step S23 primarily transforms each event into a "processable" vector and time-encoded representation, enabling events of different types and magnitudes to be converted into a consistent input. Specifically, different categories of events undergo One-Hot encoding, such as One-Hot encoding for events like "breakfast / lunch / dinner / snack / football / basketball / insulin / midday nap / nighttime sleep." The numerical attributes of each event category (such as carbohydrates, dosage, exercise duration, intensity, and sleep score) are subjected to logarithmic / normalization transformation to compress long tails. Furthermore, "time difference" information is added to each event category, calculating "how long ago" (in minutes / hours) for each event during blood glucose prediction. If an event of a certain category occurred more than 24 hours ago, that event is no longer represented or encoded. Finally, for each event at any given time, a vector record is output, including the event type, the event's numerical attributes, and the time difference.

[0044] Step S24: Design blood glucose-time response templates for different events.

[0045] The purpose of this step is to define a glucose-time response template for each event type, representing the typical shape of glucose changes over time after the event occurs, such as a post-lunch glucose rise-fall curve or a sustained glucose decline curve after insulin intake. Each glucose-time response template is controlled by a set of learnable parameters (such as rise rate, peak time, decay rate, and delay). These learnable parameters can be initialized empirically, and the residuals are learned using a small neural network or basis functions to improve the fit. The glucose-time response template should support a time range of 0-24 hours, with a focus on fitting 0-4 hours.

[0046] Step S25: Map the event intensity.

[0047] Step S25 primarily maps metrics such as "50g of carbohydrates" or "5U of insulin" to "effective intensity of blood glucose," and then multiplies it by a blood glucose-time response template. Mapping calculation methods include linear mapping and small-scale nonlinear mapping, with the output being the "intensity s" of each event, representing the relative contribution of that event to blood glucose levels.

[0048] Step S26: Superimpose the impact of the events to obtain the event-driven curve.

[0049] The event-driven curve R(t) is obtained by summing the contributions of all historical events at the current moment. For example, the implementation process is as follows: A fixed historical window (the past 24 hours) is used; only events within this window are accumulated, ignoring events occurring more than 24 hours ago to avoid infinite accumulation. For each event within the window, its "time difference" is calculated, and the response shape value at that time point is generated using the template from step S24 (the template value is taken based on the time difference). The intensity s from step S25 is multiplied by this template value to obtain the contribution of that event at the current moment. The contributions of all events are summed to obtain the event-driven curve R(t).

[0050] Step S27: Perform baseline modeling of long-term changes in blood glucose to obtain a blood glucose baseline.

[0051] The purpose of step S27 is to capture slow time-varying trends (such as circadian rhythms and long-term metabolic changes) that are not explained by short-term events. This can be achieved by using historical blood glucose values ​​from the past few days as a low-pass filter to obtain the blood glucose baseline B(t).

[0052] Step S28: Merge the event-driven curve and the blood glucose baseline to obtain the final trend.

[0053] By fusing the event-driven curve R(t) and the blood glucose baseline B(t), the system outputs a predicted blood glucose value and confidence interval based on behavioral information such as diet, exercise, medication, and sleep.

[0054] Step S3: Weighted fusion of the prediction results of the main model and the auxiliary model to obtain the fused blood glucose prediction value.

[0055] The main function of step S3 is to fuse the prediction results of the main model and the auxiliary model to improve the accuracy of non-invasive blood glucose monitoring.

[0056] In one embodiment, weighted fusion of the prediction results of the main model and the auxiliary model includes the following steps: Step S31: Prepare input.

[0057] The predicted blood glucose levels and their corresponding confidence intervals are obtained from the main model, and also from the auxiliary model. A smaller confidence interval indicates a more reliable prediction, while a larger confidence interval indicates a less reliable prediction.

[0058] Step S32: Convert the confidence interval into weights.

[0059] The weights of each model are defined based on the size of the confidence interval. The weights are inversely proportional to the confidence interval; the smaller the confidence interval, the larger the weight. The weights are then normalized so that the sum of the two weights is 1. This gives us the relative reliability of the main model and the auxiliary model at the current time point.

[0060] Step S33: Integrate blood glucose prediction values.

[0061] Based on the weights obtained in step S32, the predicted values ​​of the main model and the auxiliary model are weighted and averaged to obtain the fused blood glucose prediction value.

[0062] Accordingly, the present invention also provides a non-invasive blood glucose monitoring device based on multimodal information, used to implement one or more aspects of the above method. For example, the device includes: a first prediction unit: used to acquire multimodal physiological information of the target, and use a master model to capture the direct impact and instantaneous representation of blood glucose on physiological information, obtaining a first predicted blood glucose value and a corresponding first confidence interval; a second prediction unit: used to acquire behavioral information of the target, and use an auxiliary model to capture the long-term impact of different behaviors on blood glucose, obtaining a second predicted blood glucose value and a corresponding second confidence interval; and a fusion prediction unit: used to perform weighted fusion of the first and second predicted blood glucose values ​​to obtain a fused predicted blood glucose value, wherein the weighting weights are determined based on the first and second confidence intervals. Each unit can be implemented using dedicated hardware or general-purpose hardware combined with software.

[0063] In summary, the present invention has the following advantages: 1) The present invention proposes a dual-branch (main model + auxiliary model) combined non-invasive blood glucose monitoring method, which realizes the simultaneous modeling of the "cause" (i.e., behaviors such as diet, exercise, drugs, and sleep will cause changes in blood glucose) and the "effect" (i.e. changes in blood glucose will cause changes in physiological information such as radio frequency signals, electrocardiogram signals, and photoplethysmography pulse waves), thereby improving the accuracy of non-invasive blood glucose monitoring.

[0064] 2) This invention designs a master model for modeling, and during the modeling process, it constructs a joint signal quality assessment (SQI) vector and cross-modal attention fusion based on the joint SQI vector, which improves the feature capture capability of multimodal physiological information.

[0065] 3) This invention designs an auxiliary model for modeling, especially by designing event representation and encoding, and designing blood glucose-time response templates for different events, which improves the ability to capture behavioral features.

[0066] 4) Verification has shown that this invention improves the accuracy of non-invasive blood glucose monitoring and its applicability to target individuals by modeling the main model and the auxiliary model and fusing the prediction results of the main model and the auxiliary model.

[0067] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0068] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0069] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0070] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, Python, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0071] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0072] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0073] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0074] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0075] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A non-invasive blood glucose monitoring method based on multimodal information, comprising the following steps: The target's multimodal physiological information is acquired, and the master model is used to capture the direct impact and instantaneous representation of blood glucose on physiological information, thereby obtaining the first blood glucose prediction value and the corresponding first confidence interval. The system acquires behavioral information of the target and uses an auxiliary model to capture the long-term effects of different behaviors on blood glucose, thereby obtaining a second blood glucose prediction value and the corresponding second confidence interval. The first and second blood glucose prediction values ​​are weighted and fused to obtain the fused blood glucose prediction value, wherein the weighting weights are determined based on the first and second confidence intervals.

2. The method according to claim 1, characterized in that, The first predicted blood glucose value and the corresponding first confidence interval are obtained according to the following steps: Collecting the multimodal physiological information, which includes the amplitude and phase of radio frequency signals at different frequencies, single-lead or multi-lead electrocardiogram signals, and single-wavelength or multi-wavelength photoplethysmography (PPG) signals; for the multimodal physiological information, constructing a joint signal quality assessment vector based on single-modal quality scores and cross-modal consistency feature scores; for the multimodal physiological information that meets the quality requirements, extracting the corresponding multimodal features as input to the main model, which include radio frequency features, electrocardiogram (ECG) features, and other data. The system employs three single-modal features: eigenvalues, pulse wave features, and takt wave features. Based on the joint signal quality assessment vector, gating coefficients for each modality in the multimodal physiological information are determined. These gating coefficients are used to scale different single-modal features or to adjust the information flow ratio between different modalities of physiological information. Cross-modal attention fusion is performed based on the joint signal quality assessment vector to obtain fused cross-modal features. The gating coefficients are applied to the key vector, and the query vector is matched with the gated key vector to calculate attention weights. The value vector is then optimized based on the attention weights to obtain the fused cross-modal features. The cross-modal features are input into the blood glucose regression unit, which outputs the first predicted blood glucose value and the corresponding first confidence interval at the current time.

3. The method according to claim 2, characterized in that, The blood glucose regression unit is a lightweight Transformer or temporal convolutional network.

4. The method according to claim 2, characterized in that, The joint signal quality evaluation vector is represented as follows: Wherein, the joint SQI vector represents the joint signal quality assessment vector. The joint SQI score of the radio frequency signal includes the single-mode quality score and the cross-modal consistency feature score of the radio frequency signal. The joint SQI score of the electrocardiogram (ECG) signal includes the single-mode quality score and the cross-modal consistency feature score of the ECG signal. The joint SQI score of the photoplethysmography (PPG) signal includes the single-mode quality score and the cross-modal consistency feature score of the PPG.

5. The method according to claim 1, characterized in that, The second predicted blood glucose value and the corresponding second confidence interval are obtained according to the following steps: acquiring multi-category behavioral information of the target as raw events; converting the raw events into standardized multi-category events through a unified format and numerical standardization; converting the multi-category events into vector and time codes, wherein each event is represented by a vector record, which includes the event type, the event's numerical attributes, and the time difference; defining a corresponding blood glucose-time response template for each category of event, representing the shape of blood glucose changes over time after the event occurs, and each blood glucose-time response template is controlled by a set of learnable parameters; performing event intensity mapping for each category of event to characterize the relative contribution of the event to blood glucose; summing the contributions of multiple historical events at the current moment to obtain an event-driven curve; obtaining a blood glucose baseline by low-pass filtering the historical blood glucose values ​​of several days; and fusing the event-driven curve and the blood glucose baseline to obtain the second predicted blood glucose value and the corresponding second confidence interval.

6. The method according to claim 2, characterized in that, The radio frequency features are obtained by extracting radio frequency signal features through a frequency domain convolutional network. The electrocardiogram (ECG) features are obtained by extracting ECG signal features through a one-dimensional convolutional or temporal Transformer network. The pulse wave features are obtained by extracting photoplethysmography (PPG) pulse wave signal features through convolutional networks of different scales.

7. The method according to claim 1, characterized in that, The behavioral information includes eating behavior, exercise behavior, medication behavior, and sleep behavior.

8. The method according to claim 1, characterized in that, The weighted fusion of the first and second blood glucose prediction values ​​includes: defining the weights of the main model and the auxiliary model based on the size of the first and second confidence intervals, wherein the weights are inversely proportional to the confidence intervals; normalizing the weights of the main model and the auxiliary model to obtain normalized weights; and performing weighted fusion of the first and second blood glucose prediction values ​​based on the normalized weights.

9. A non-invasive blood glucose monitoring device based on multimodal information, comprising: First prediction unit: used to acquire multimodal physiological information of the target, and use the master model to capture the direct impact and instantaneous representation of blood glucose on physiological information, and obtain the first blood glucose prediction value and the corresponding first confidence interval; The second prediction unit is used to acquire behavioral information of the target and use an auxiliary model to capture the long-term effects of different behaviors on blood glucose, thereby obtaining the second blood glucose prediction value and the corresponding second confidence interval. Fusion prediction unit: used to perform weighted fusion of the first blood glucose prediction value and the second blood glucose prediction value to obtain the fused blood glucose prediction value, wherein the weighting weight is determined according to the first confidence interval and the second confidence interval.

10. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images