Noninvasive blood glucose monitoring method and system based on multi-modal fusion

Abstract

The invention relates to a non-invasive blood glucose monitoring technology based on multi-modal fusion, and aims to solve the problem that the accuracy of non-invasive blood glucose monitoring is limited due to the fact that a simple splicing or voting method is often adopted in an existing multi-modal information fusion method and complementarity among different modal signals is not fully utilized. According to the scheme, data-level, feature-level and decision-level fusion is carried out on the electrocardio signal, the electroencephalogram signal and the photoelectric volume pulse wave signal, the correlation among different modal signals is fully mined, and the accuracy of noninvasive blood glucose monitoring is improved.

Description

Technical Field

[0001] This disclosure relates to intelligent medical devices, and more particularly to a non-invasive blood glucose monitoring method and system based on multimodal fusion. Background Technology

[0002] Diabetes mellitus is a chronic metabolic disease that seriously endangers human health, causing severe complications such as cardiovascular disease, kidney disease, neuropathy, and retinopathy, placing a heavy burden on individual quality of life and the social healthcare system. Active blood glucose monitoring is a fundamental aspect of diabetes prevention and precision management. Non-invasive blood glucose monitoring technology is of great significance for the long-term management of diabetes and the prevention of its complications. It enables continuous, convenient, and painless blood glucose assessment, and is expected to significantly improve patient compliance and promote the development of personalized metabolic health monitoring. However, non-invasive blood glucose monitoring technologies based on single-modal signals (such as electrocardiogram, photoplethysmography, etc.) often face problems such as insufficient sensitivity, susceptibility to individual differences, significant environmental interference, and limited correlation between the signal and blood glucose changes, making it difficult to meet the accuracy and stability requirements of clinical applications. Employing multiple modal signals can more comprehensively capture the physiological response characteristics of blood glucose fluctuations and reduce interference from factors such as human physiological state and environmental changes, representing a key breakthrough for non-invasive monitoring technology from "theoretical verification" to "clinical usability." However, existing multimodal information fusion methods often employ simple splicing or voting methods, failing to fully utilize the complementarity between different modal signals, thus limiting the accuracy of non-invasive blood glucose monitoring. Summary of the Invention

[0003] This invention proposes a non-invasive blood glucose monitoring method and system based on multimodal fusion. Specifically, it fuses three signals—electrocardiogram (ECG), electroencephalogram (EEG), and photoplethysmography (PPG)—at the data, feature, and decision levels to fully explore the correlation between different modal signals, thus providing support for improving the accuracy of non-invasive blood glucose monitoring.

[0004] Firstly, this disclosure proposes a non-invasive blood glucose monitoring method based on multimodal fusion, comprising: converting three-channel electrocardiogram (ECG), three-channel electroencephalogram (EEG), and three-wavelength photoplethysmography (PPG) signals of equal length within the same time period into two-dimensional images; then stacking the three two-dimensional images corresponding to the same signals to obtain three-dimensional images corresponding to ECG, EEG, and PPG signals, respectively; performing feature mining on the three-dimensional images corresponding to ECG, EEG, and PPG signals to obtain one-dimensional features corresponding to ECG, EEG, and PPG signals, respectively; performing cross-product operations on each of the three features to obtain three two-dimensional features; stacking these features to obtain a three-dimensional feature; and obtaining the blood glucose value based on the obtained three-dimensional feature.

[0005] In one embodiment of the above technical solution, the electrocardiogram (ECG) signal is acquired using a three-channel ECG acquisition device, wherein: the three electrodes are located 1-2 finger widths below the right clavicle, 1-2 finger widths below the left clavicle, and within a preset range on the left lower anterior chest wall, respectively, and the ground electrode is located within a preset range on the right lower anterior chest wall; the electroencephalogram (EEG) signal is acquired using a three-channel EEG acquisition device: the three electrodes are located at the midline of the forehead, the central midline, and the midline of the parietal lobe, respectively, the reference electrode is located at the left earlobe, and the ground electrode is located at the center of the forehead; the photoplethysmography (PPG) signal is obtained by incidenting three different wavelengths of light onto the fingers.

[0006] In one embodiment of the above technical solution, the Markov transition field method is used to convert three-channel electrocardiogram (ECG), three-channel electroencephalogram (EEG), and three-wavelength photoplethysmography (PPG) signals of equal length within the same time period into two-dimensional images. The steps include: normalizing the amplitude of the three-channel ECG, three-channel EEG, and three-wavelength PPG signals of equal length within the same time period, and discretizing the continuous signals into Q state sequences; constructing a Q×Q dimension Markov transition matrix for each state sequence corresponding to the ECG, EEG, and PPG signals by statistically analyzing the transition relationships between any adjacent time points. Matrix elements This represents the probability of transitioning from state i to state j; Extending the sequence chronologically to the length of the original sequence, and mapping the combination of each time t with any time k to the corresponding transition probability, an L×L Markov transition field image is generated, thus obtaining a two-dimensional image.

[0007] In one embodiment of the above technical solution, the Grami angle field method is used instead of the Markov transition field method.

[0008] In one embodiment of the above technical solution, the blood glucose value is obtained based on the obtained three-dimensional features. The steps include: for the three-dimensional features at the current time t, inputting them into a deep convolutional neural network to output the predicted blood glucose value. Using the predicted blood glucose values ​​at times t-5, t-4, t-3, t-2, t-1, and t, a fuzzy integral algorithm is used for regression processing. Based on preset weights, a new blood glucose value is obtained and used as the current blood glucose value at time t. The closer the blood glucose value is to the current time, the greater its weight.

[0009] In one embodiment of the above technical solution, the fuzzy integral algorithm is the Volkovich fuzzy integral or the Sugeno fuzzy integral.

[0010] In one embodiment of the above technical solution, the sampling rates of electrocardiogram (ECG), electroencephalogram (EEG), and photoplethysmography (PPG) signals are the same. If the sampling rates are different, an interpolation method is used to process the signals so that the signal lengths are the same within the same time period.

[0011] In one embodiment of the above technical solution, a deep residual network is used to achieve feature mining.

[0012] Secondly, in the implementation of the above method, a non-invasive blood glucose monitoring system based on multimodal fusion is obtained. The system includes a data-level fusion module, a feature-level fusion module, and a decision-level fusion module. Specifically: the data-level fusion module is configured to convert three-channel electrocardiogram (ECG), three-channel electroencephalogram (EEG), and three-wavelength photoplethysmography (PPG) signals of equal length within the same time period into two-dimensional images. Then, the three two-dimensional images corresponding to the same signals are stacked to obtain three-dimensional images corresponding to ECG, EEG, and PPG signals, respectively. The feature-level fusion module is configured to perform feature mining on the three-dimensional images corresponding to ECG, EEG, and PPG signals, respectively, to obtain one-dimensional features corresponding to ECG, EEG, and PPG signals. These features are then multiplied pairwise to obtain three two-dimensional features, which are then stacked to obtain a three-dimensional feature. The decision-level fusion module is configured to obtain blood glucose values ​​based on the obtained three-dimensional features.

[0013] The beneficial technical effects of this invention are as follows: The non-invasive blood glucose monitoring technology based on multimodal information in this application is a novel blood glucose monitoring method that integrates multi-source physiological data. This technology captures features related to blood glucose changes in different modal signals, constructs a multimodal fusion model that dynamically reflects blood glucose fluctuations, and enables high-precision continuous blood glucose monitoring without blood sampling. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the framework of a non-invasive blood glucose monitoring system based on multimodal fusion in one embodiment.

[0016] Figure 2 This is a schematic diagram of data-level fusion of three-channel ECG signals in one implementation.

[0017] Figure 3 This is a schematic diagram of feature-level fusion of three-channel ECG signals in one implementation. Detailed Implementation

[0018] Existing non-invasive blood glucose monitoring technologies based on multimodal signals often employ simple feature splicing or voting during multimodal fusion, failing to fully utilize the complementarity of different modal information and thus limiting the accuracy of non-invasive blood glucose monitoring.

[0019] The main objective of this invention is to improve the accuracy of non-invasive blood glucose monitoring by fully exploring the correlation between different modal signals through data-level, feature-level, and decision-level fusion of three signals: ECG, EEG, and PPG.

[0020] The following description, in conjunction with the accompanying drawings, clearly and completely describes how the technical solution of this case is implemented. Obviously, the described embodiments are only a part of the embodiments of this case, and not all of them. Based on the embodiments in this case, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0021] The non-invasive blood glucose monitoring method in this application is a multimodal fusion method based on data level, feature level and decision level, used for the fusion of ECG, EEG and PPG signals, thereby achieving high-accuracy non-invasive blood glucose monitoring.

[0022] Figure 1 The framework of a non-invasive blood glucose monitoring system based on multimodal fusion is illustrated. It mainly consists of four modules: a multimodal signal acquisition module, a data-level fusion module, a feature-level fusion module, and a decision-level fusion module. The modules are described below.

[0023] (a) Multimodal signal acquisition module The multimodal signal acquisition module is configured to acquire multimodal signals within a preset time period. If the preset time period is 30 seconds, the module will acquire a multimodal signal every 30 seconds.

[0024] In this application, the multimodal signal consists of ECG, EEG, and PPG signals. In terms of hardware implementation, the ECG signal is acquired using a three-channel ECG acquisition device. This device has three electrodes and one ground electrode. When acquiring ECG signals, electrode 1 is positioned 1-2 finger widths below the right clavicle, electrode 2 is positioned 1-2 finger widths below the left clavicle, electrode 3 is positioned within a preset range on the lower left anterior chest wall, and the ground electrode is positioned within a preset range on the lower right anterior chest wall. The preset range is the area from which a valid signal can be acquired. The EEG signal is acquired using a three-channel EEG acquisition device. This device has three electrodes, one reference electrode, and one ground electrode. When acquiring ECG signals, electrode 1 is positioned on the midline of the forehead, electrode 2 is positioned on the central midline, electrode 3 is positioned on the midline of the parietal lobe, the reference electrode is positioned at the left earlobe, and the ground electrode is positioned at the center of the forehead. For the PPG signal, three different wavelengths of light are incident on the fingers to obtain three wavelengths of PPG signals. That is... Figure 1 The first module shown in the diagram performs three-channel ECG signal acquisition, three-channel EEG signal acquisition, and three-wavelength PPG signal acquisition. The acquired three-channel ECG signals are denoted as... , , The acquired three-channel EEG signals are denoted as follows: , , The three wavelength PPG signals are denoted as follows: , , .

[0025] For ease of analysis, the sampling rates of ECG, EEG, and PPG signals should be kept consistent; otherwise, interpolation methods should be used to process the signals so that their lengths are the same within the same time frame.

[0026] (ii) Data-level fusion module The data-level fusion module is configured to convert the three channels of ECG signals separately. , and Three-channel EEG signal , and Three-wavelength PPG signal , and Data-level fusion is performed separately to obtain separate 3D images. , and .

[0027] Figure 2 Given , and A method for implementing data-level fusion of signals.

[0028] Step 1: For , and It is a one-dimensional signal. Assuming the ECG sampling rate is fs (Hz), when the sampling time is τ (s), the data length is L = fs × τ. Therefore, for , and It can be represented as , , ,in This indicates that the signal is a one-dimensional signal with a length of L.

[0029] Step 2: Use the Markov transition field method to analyze the one-dimensional signal. Transform it into a two-dimensional image. The transformation process is as follows: First, for The signal amplitude is normalized, and the continuous signal is discretized into Q state sequences using equal-probability binning or K-means quantization. Then, a Q×Q Markov transition matrix is ​​constructed by statistically analyzing the transition relationships between any adjacent time points. , where matrix elements This represents the probability of transitioning from state i to state j. Then, the Markov transition matrix... Extending the sequence chronologically to the length of the original sequence, and mapping each time step t with any time step k to the corresponding transition probability, an L×L Markov transition field image, i.e., a two-dimensional image, is generated. In this application, other methods, such as the Grammy angle field method, can also be used to convert a one-dimensional signal into a two-dimensional image.

[0030] Step 3: For and Similarly, using the method in step 2, it is converted into a Markov transition field image, i.e., a two-dimensional image. and .

[0031] Step 4: Transform the 2D image , and By fusing the images, a three-dimensional image can be obtained. That is, the three-dimensional image is composed of three two-dimensional images stacked together.

[0032] Following steps 2 to 4 above, three-dimensional images can be obtained respectively. and That is to say... Figure 1 As shown in the second module, the data-level fusion module acquires 3D images of ECG, EEG, and PPG.

[0033] Based on the above data-level fusion method for EEG, ECG, and PPG, a data-level fusion module can be obtained by implementing it through a program or a combination of program and hardware.

[0034] (III) Feature-level fusion module The feature-level fusion module is configured to perform feature mining algorithms on 3D images. , and Feature mining and fusion.

[0035] Figure 3 The three-dimensional image is given , and Feature-level fusion methods.

[0036] Step 1: Separate 3D images , and The data is used as input to a feature mining algorithm. This algorithm processes the 3D image using methods such as deep residual networks. At the end of the algorithm, the features are flattened, and the mined features are output. , and Therefore, the output feature is one-dimensional. The number of representative features.

[0037] Step 2: and Perform the outer product operation to obtain the features. It should be noted that when performing the outer product operation, one of the features needs to be converted to... In the form of. and Perform the outer product operation to obtain the features. .Will and Perform the outer product operation to obtain the features. .

[0038] Step 3: Convert the two-dimensional features obtained in Step 2 , and By fusing the data, a three-dimensional feature map is obtained. ( Figure 1 The third module shown is a three-dimensional feature composed of three stacked two-dimensional features.

[0039] Based on the aforementioned feature-level fusion methods for EEG, ECG, and PPG, a feature-level fusion module can be obtained by implementing it through a program or a combination of program and hardware.

[0040] (iv) Decision-level integration module The main function of the decision-level fusion module is to output blood glucose values ​​based on the obtained three-dimensional features using deep learning algorithms.

[0041] The specific steps for implementing decision-level fusion are as follows.

[0042] Step 1: For the current time t, the three-dimensional features output by the feature-level fusion module of the multimodal signal are: Based on this three-dimensional feature, a deep convolutional neural network algorithm is used to process the feature and output a predicted blood glucose value. .

[0043] Step 2: Calculate the three-dimensional features output by the feature-level fusion module of the multimodal signal at the previous time step (t-1). Based on this three-dimensional feature, a deep convolutional neural network algorithm is used to process the feature and output a predicted blood glucose value. Similarly, the blood glucose values ​​at times t-2, t-3, t-4, and t-5 are obtained respectively. , , , , .

[0044] Step 3: Set the current blood glucose prediction value And the predicted blood glucose values ​​at the first 5 time points. , , , , Using the same data as input to the algorithm, a fuzzy integral algorithm (such as Volkovich fuzzy integral or Sugeno fuzzy integral) is used to regress the current blood glucose prediction value and the blood glucose prediction values ​​of the previous 5 time points, assigning different weights, with the weight increasing closer to the current time point. After regression processing, a new blood glucose value is obtained as the output. Figure 1 The fourth module in the system, this blood glucose value is the final predicted blood glucose value at time t.

[0045] Based on the decision-level fusion methods for EEG, ECG, and PPG described above, a special decision-level fusion module can be obtained by implementing it through a program or a combination of program and hardware.

[0046] In summary, the non-invasive blood glucose monitoring method based on multimodal fusion at the data, feature, and decision levels proposed in this invention can fully mine the features of ECG, EEG, and PPG signals, thereby improving the accuracy of non-invasive blood glucose monitoring. The method includes: converting three-channel ECG signals, three-channel EEG signals, and three wavelengths of photoplethysmography (PPG) signals of equal length within the same time frame into two-dimensional images; then stacking the three two-dimensional images corresponding to the same signals to obtain three-dimensional images corresponding to ECG, EEG, and PPG signals, respectively; performing feature mining on the three-dimensional images corresponding to ECG, EEG, and PPG signals to obtain one-dimensional features corresponding to ECG, EEG, and PPG signals, respectively; performing pairwise cross products on these three images to obtain three two-dimensional features; and stacking these features to obtain a single three-dimensional feature; and finally, obtaining the blood glucose value based on the obtained three-dimensional feature.

[0047] The present invention can also be a system and / or a computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present invention.

[0048] For example, a non-invasive blood glucose monitoring system based on multimodal fusion includes a data-level fusion module, a feature-level fusion module, and a decision-level fusion module. The data-level fusion module is configured to convert three-channel electrocardiogram (ECG), three-channel electroencephalogram (EEG), and three-wavelength photoplethysmography (PPG) signals of equal length into two-dimensional images within the same time frame. The three two-dimensional images corresponding to the same signals are then stacked to obtain three-dimensional images corresponding to the ECG, EEG, and PPG signals, respectively. The feature-level fusion module is configured to perform feature mining on the three-dimensional images corresponding to the ECG, EEG, and PPG signals, respectively, to obtain one-dimensional features corresponding to the ECG, EEG, and PPG signals. These features are then multiplied pairwise to obtain three two-dimensional features, which are then stacked to obtain a single three-dimensional feature. The decision-level fusion module is configured to obtain blood glucose values ​​based on the obtained three-dimensional features.

[0049] This application can also be a non-invasive blood glucose monitoring device based on multimodal fusion. The device includes an EEG signal acquisition unit, an ECG signal acquisition unit, a PPG signal acquisition unit, and a data processing unit. Specifically: the EEG signal acquisition unit uses a three-channel ECG acquisition device, with three electrodes located 1-2 finger widths below the right clavicle, 1-2 finger widths below the left clavicle, and within a preset range on the lower left anterior chest wall; the ground electrode is located within a preset range on the lower right anterior chest wall. The ECG signal acquisition unit uses a three-channel EEG acquisition device, with three electrodes located at the midline of the forehead, the central midline, and the midline of the parietal lobe; a reference electrode is located at the left earlobe; and the ground electrode is located at the center of the forehead. The PPG signal acquisition unit uses three different wavelengths of light incident on the fingers. The data processing unit is configured to obtain blood glucose values ​​based on the signals obtained from the EEG signal acquisition unit, ECG signal acquisition unit, PPG signal acquisition unit, and data processing unit, using the aforementioned non-invasive blood glucose monitoring method based on data-level, feature-level, and decision-level multimodal fusion.

[0050] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0051] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0052] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, Python, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0053] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0054] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0055] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0056] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be well known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0057] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A non-invasive blood glucose monitoring method based on multimodal fusion, characterized in that, include: Three-channel electrocardiogram (ECG), three-channel electroencephalogram (EEG), and three-wavelength photoplethysmography (PPG) signals of the same length are converted into two-dimensional images within the same time period. Then, the three two-dimensional images corresponding to the same signals are stacked to obtain three-dimensional images corresponding to ECG, EEG, and PPG signals, respectively. Feature mining is performed on the three-dimensional images corresponding to ECG, EEG, and PEP pulse wave signals to obtain one-dimensional features corresponding to ECG, EEG, and PEP pulse wave signals respectively. The three are then multiplied in pairs to obtain three two-dimensional features, which are then stacked to obtain a three-dimensional feature. Based on the obtained three-dimensional features, the blood glucose value is obtained.

2. The method according to claim 1, characterized in that: The electrocardiogram signal was acquired using a three-channel ECG acquisition device, with the three electrodes located 1-2 finger widths below the right clavicle, 1-2 finger widths below the left clavicle, and within a preset range on the left lower anterior chest wall, respectively, and the ground electrode located within a preset range on the right lower anterior chest wall. The EEG signals were acquired using a three-channel EEG acquisition device: three electrodes were located at the midline of the forehead, the central midline, and the midline of the parietal lobe, respectively; the reference electrode was located at the left earlobe; and the ground electrode was located at the center of the forehead. The photoplethysmography (PPG) signal is obtained by shining three different wavelengths of light onto the finger.

3. The method according to claim 1, characterized in that, Using the Markov transition field method, three-channel electrocardiogram (ECG), three-channel electroencephalogram (EEG), and three-wavelength photoplethysmography (PPG) signals of equal length within the same time interval are converted into two-dimensional images. The steps include: Amplitude normalization was performed on three-channel electrocardiogram signals, three-channel electroencephalogram signals, and photoplethysmography signals of three wavelengths with the same signal length within the same time period, and the continuous signals were discretized into Q state sequences. For the state sequences corresponding to electrocardiogram (ECG), electroencephalogram (EEG), and photoplethysmography (PPG) signals, a Q×Q dimension Markov transition matrix is ​​constructed by statistically analyzing the transition relationships between any two adjacent time points. Matrix elements This represents the probability of transitioning from state i to state j; Will Extending the sequence chronologically to the length of the original sequence, and mapping the combination of each time t with any time k to the corresponding transition probability, an L×L Markov transition field image is generated, thus obtaining a two-dimensional image.

4. The method according to claim 3, characterized in that, The Grami angle field method is used instead of the Markov transition field method.

5. The method according to claim 1, characterized in that, Based on the obtained three-dimensional features, the blood glucose value is obtained. The steps include: For the three-dimensional features at time t, input them into a deep convolutional neural network to output the predicted blood glucose value. ; Using the predicted blood glucose values ​​at times t-5, t-4, t-3, t-2, t-1, and t, a fuzzy integral algorithm is used for regression processing. Based on preset weights, a new blood glucose value is obtained and used as the current blood glucose value at time t. The closer the blood glucose value is to the current time, the greater its weight.

6. The method according to claim 5, characterized in that, The fuzzy integral algorithm is either the Volkovich fuzzy integral or the Sugeno fuzzy integral.

7. The method according to claim 1, characterized in that, The sampling rates of electrocardiogram (ECG), electroencephalogram (EEG), and photoplethysmography (PPG) signals are the same. If the sampling rates are different, an interpolation method is used to process the signals so that the signal length is the same within the same time period.

8. The method according to claim 1, characterized in that, Feature mining is achieved using deep residual networks.

9. A non-invasive blood glucose monitoring system based on multimodal fusion, characterized in that, The system includes a data-level fusion module, a feature-level fusion module, and a decision-level fusion module; wherein: The data-level fusion module is configured to convert three-channel electrocardiogram (ECG) signals, three-channel electroencephalogram (EEG) signals, and three-wavelength photoplethysmography (PPG) signals of the same length into two-dimensional images within the same time period. Then, the three two-dimensional images corresponding to the same signals are stacked to obtain three-dimensional images corresponding to ECG, EEG, and PPG signals, respectively. The feature-level fusion module is configured to perform feature mining on the three-dimensional image corresponding to the electrocardiogram, the three-dimensional image corresponding to the electroencephalogram, and the three-dimensional image corresponding to the photoplethysmography (PPG) signal, respectively, to obtain one-dimensional features corresponding to the electrocardiogram, the one-dimensional features corresponding to the electroencephalogram, and the one-dimensional features corresponding to the PPG signal. The three are then multiplied in pairs to obtain three two-dimensional features, which are then stacked to obtain a three-dimensional feature. The decision-level fusion module is configured to obtain blood glucose values ​​based on the obtained three-dimensional features.

10. The system according to claim 9, characterized in that, The system includes an EEG signal acquisition unit, an ECG signal acquisition unit, and a PPG signal acquisition unit; wherein: The EEG signal acquisition unit uses a three-channel ECG acquisition device, wherein the three electrodes are located 1-2 finger widths below the right clavicle, 1-2 finger widths below the left clavicle, and within a preset range on the lower left anterior chest wall, and the ground electrode is located within a preset range on the lower right anterior chest wall. The ECG signal acquisition unit uses a three-channel EEG acquisition device: three electrodes are located on the midline of the forehead, the central midline, and the midline of the parietal lobe, respectively; the reference electrode is located at the left earlobe; and the ground electrode is located in the center of the forehead. The PPG signal acquisition unit uses three different wavelengths of light incident on the finger to obtain the signal.

Images