Blood pressure collection method and collection system based on multi-modal signal fusion

Abstract

The invention relates to the technical field of biomedical engineering, in particular to a blood pressure collection method and system based on multi-modal signal fusion. On the basis of ECG (electrocardiogram) data and PPG (pulse wave data), two other physiological parameters including fingertip pressure data Pressure and posture data IMU (inertial measurement unit) are introduced, so that a multi-modal signal is formed; and processing the multi-modal signal by adopting a newly constructed blood pressure monitoring network model to obtain a real-time systolic pressure SBP and a real-time diastolic pressure DBP. Wherein the blood pressure monitoring network model adopts a hybrid model architecture, the problem of space-time fusion of multi-modal signals is solved, the fusion weight is adjusted in real time according to the signal quality, the robustness of the model is improved, and the accuracy of blood pressure collection is ensured. The problem that an effective blood pressure collection method based on multi-mode signals is lacked in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of biomedical engineering technology, and more specifically, to: 1. a blood pressure acquisition method based on multimodal signal fusion; 2. a blood pressure acquisition system based on multimodal signal fusion. Background Art

[0002] Thanks to the rapid development of electronic technology, many wearable physiological parameter collection devices have emerged. For example, patent application CN116725503A, titled "A Multi-Physiological Parameter Measurement Watch," utilizes electrocardiogram (ECG) and pulse wave (PPG) data for blood pressure monitoring. Collection electrodes are incorporated under the cover to achieve the purpose of collecting multiple physiological parameters. However, the patent does not describe how to use PPG and ECG to obtain blood pressure values.

[0003] While PPG and ECG data contain blood pressure information, deriving blood pressure values from them remains a challenging task, requiring effective feature processing of the data. Furthermore, blood pressure information is also included in other physiological parameters (such as posture data), which existing methods rarely consider and lack appropriate processing methods. Summary of the Invention

[0004] Based on this, it is necessary to provide a blood pressure collection method and collection system based on multimodal signal fusion to address the problem of the lack of effective methods for blood pressure collection based on multimodal signals.

[0005] The present invention is achieved by adopting the following technical solutions:

[0006] In a first aspect, the present invention discloses a blood pressure collection method based on multimodal signal fusion, which is used to collect the real-time systolic blood pressure (SBP) and real-time diastolic blood pressure (DBP) of a target user in real time.

[0007] Blood pressure acquisition methods based on multimodal signal fusion include:

[0008] Step 1: Obtain the target user's real-time ECG data, PPG data, fingertip pressure data, and IMU posture data;

[0009] Step 2: Input EGG, PPG, Pressure, and IMU into the trained blood pressure monitoring network model for processing to obtain SBP and DBP.

[0010] Among them, the blood pressure monitoring network model includes: preprocessing layer, feature encoding layer, and feature fusion layer.

[0011] The preprocessing layer is used to process EGG, PPG, Pressure, and IMU into two sets of multi-dimensional time series input sequences. The two sets of multi-dimensional time series input sequences include: multi-dimensional time series input sequence vital and multi-dimensional time series input sequence motion.

[0012] The feature encoding layer is used to process vital and motion based on the bidirectional LSTM network to obtain the output feature H vital , output feature H motion .

[0013] The feature fusion layer is used to: vital 、H motion Processing is performed to obtain the main modal feature representation A vital , auxiliary modal feature representation A motion Based on signal quality perception and dynamic weight fusion mechanism, vital and motion are processed to obtain the main modality fusion weight α vital , auxiliary modality fusion weight α motion ; Based on α vital , α motion To A vital 、A motion Perform weighted summation to obtain the fusion feature representation F fusion , and F fusion Superimposed with the position encoding tensor Position to obtain the time-compensated feature representation New_F fusion .

[0014] The feature aggregation layer is used to: fusion Global feature processing is performed to obtain SBP and DBP.

[0015] This blood pressure acquisition method based on multimodal signal fusion implements the method or process according to the embodiment of the present disclosure.

[0016] In a second aspect, the present invention discloses a blood pressure acquisition system based on multimodal signal fusion, including: an acquisition device and an intelligent terminal.

[0017] The acquisition device includes an acquisition module, a wireless communication module, and a power supply module. The acquisition module collects the target user's real-time ECG data, pulse wave data, fingertip pressure data, and posture data from the IMU. The wireless communication module transmits the ECG, PPG, Pressure, and IMU data to the smart terminal. The power supply module supplies power to the acquisition and wireless communication modules.

[0018] The intelligent terminal is used to store the trained blood pressure monitoring network model and adopt the blood pressure acquisition method based on multimodal signal fusion as disclosed in the first aspect.

[0019] This blood pressure acquisition system based on multimodal signal fusion implements the method or process according to the embodiment of the present disclosure.

[0020] In a third aspect, the present invention discloses a computer program product, comprising a computer program. When the computer program is executed by a processor, the computer program implements the steps of the blood pressure acquisition method based on multimodal signal fusion disclosed in the first aspect.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. This invention integrates two additional physiological parameters—fingertip pressure data and IMU posture data—into a multimodal signal based on ECG (electrocardiogram) and PPG (pulse wave) data. This multimodal signal is then processed using a newly constructed blood pressure monitoring network model to obtain real-time systolic blood pressure (SBP) and diastolic blood pressure (DBP). The blood pressure monitoring network model employs a hybrid model architecture, addressing the spatiotemporal fusion of multimodal signals. Fusion weights are adjusted in real time based on signal quality, enhancing the robustness of the model and ensuring the accuracy of blood pressure acquisition.

[0023] 2. This invention provides a blood pressure collection system based on multimodal signal fusion. This system connects the collection device to a smart terminal via wireless communication, increasing the device's flexibility. Furthermore, the collection device can be designed in a card format, offering excellent portability and a simple collection process. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 This is a flow chart of the blood pressure acquisition method based on multimodal signal fusion provided in Example 1 of the present invention;

[0026] Figure 2 for Figure 1 Structural diagram of the blood pressure monitoring network model;

[0027] Figure 3 This is a structural diagram of a blood pressure acquisition system based on multimodal signal fusion provided in Example 2 of the present invention;

[0028] Figure 4 An exploded diagram of a specific example of a collection device provided in Example 2 of the present invention;

[0029] Figure 5 for Figure 4 A top view of the acquisition device;

[0030] Figure 6 for Figure 4 Bottom view of the acquisition device. DETAILED DESCRIPTION

[0031] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0032] It should be noted that when a component is referred to as being "mounted on" another component, it may be directly on the other component or there may be a central component. When a component is considered to be "set on" another component, it may be directly set on the other component or there may be a central component. When a component is considered to be "fixed to" another component, it may be directly fixed to the other component or there may be a central component.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used herein in the specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "or / and" as used herein includes any and all combinations of one or more of the associated listed items.

[0034] Example 1

[0035] See Figure 1 , demonstrating the blood pressure acquisition method based on multimodal signal fusion provided in this embodiment 1.

[0036] First of all, it should be noted that this method is intended to collect the real-time systolic blood pressure SBP and real-time diastolic blood pressure DBP of the target user.

[0037] The special features of this method are: 1. On the basis of electrocardiogram data EGG and pulse wave data PPG, two other physiological parameters, fingertip pressure data Pressure and posture data IMU, are introduced to form a multimodal signal; 2. A newly constructed blood pressure monitoring network model is used to process the multimodal signal to obtain SBP and DBP.

[0038] like Figure 1 As shown, the method includes:

[0039] Step 1: Obtain the target user's real-time ECG data, PPG data, fingertip pressure data, and IMU posture data.

[0040] The above four types of data can all be collected from target users in real time using corresponding sensors and transmitted in the form of signals.

[0041] It should be noted that EEG includes x1 ECG signal; PPG includes x2 pulse wave signals; Pressure includes x3 fingertip pressure signals; and IMU includes x4 gesture signals. The number of EEG, PPG, Pressure, and IMU signal channels is not limited to one; they can all be multiple.

[0042] In this embodiment 1, the following settings are adopted:

[0043] X1=1; X2=4; X3=2; X4=1.

[0044] Step 2: Input EGG, PPG, Pressure, and IMU into the trained blood pressure monitoring network model for processing to obtain SBP and DBP.

[0045] A key aspect of this invention is the provision of a newly designed blood pressure monitoring network model (abbreviated as CrossGate-BiLT), which combines the advantages of bidirectional long short-term memory (Bi-LSTM) networks in temporal modeling with the global modeling capabilities of Transformers. It is important to note that this invention requires the use of a trained CrossGate-BiLT network with optimal network parameters.

[0046] See Figure 2 ,From a functional perspective, CrossGate-BiLT includes: pre-processing layer, feature encoding layer, and ,feature fusion layer.

[0047] The following are introduced one by one:

[0048] 1. The preprocessing layer is used to process EGG, PPG, Pressure, and IMU into two sets of multi-dimensional time series input sequences.

[0049] The two groups of multi-dimensional time series input sequences include: a multi-dimensional time series input sequence vital and a multi-dimensional time series input sequence motion.

[0050] Specifically, such as Figure 2 As shown, in the preprocessing layer:

[0051] First, preprocess the EGG, PPG, Pressure, and IMU (generally including but not limited to filtering, normalization, cycle segmentation, and sampling frequency alignment) to ensure that all modalities remain synchronized on the time axis; then, splice the preprocessed EGG and PPG into a multi-dimensional time series input sequence vital, and splice the preprocessed Pressure and IMU into a multi-dimensional time series input sequence motion.

[0052] This is because: ECG and PPG are both physiological signals of the cardiovascular system, so they are spliced into a set of multidimensional time series input sequences; Pressure and IMU mainly involve the external mechanics and motion state of the body, so they are spliced into another set of multidimensional time series input sequences; this merging facilitates the subsequent improvement of the feature modeling efficiency between modalities and fully utilizes the inherent correlation between the signals of each modality.

[0053] As mentioned above, each type of signal is preprocessed independently, and then multiple signals of the same type are spliced together according to the number of channels. The spliced ECG signals and pulse wave signals are then spliced together in the feature dimension to obtain vital, whose feature dimension is X1+X2. The spliced fingertip pressure signals and posture signals are spliced together in the feature dimension to obtain motion, whose feature dimension is X3+X4.

[0054] Then, in this embodiment 1, vital is 5-dimensional; motion is 7-dimensional.

[0055] 2. The feature encoding layer is used to process vital and motion based on the bidirectional LSTM network to obtain the output feature H vital , output feature H motion .

[0056] Specifically, such as Figure 2 As shown, the feature encoding layer is equipped with two bidirectional LSTM networks;

[0057] Then, in the feature encoding layer, vital is input into a bidirectional LSTM network for modeling and feature extraction to obtain H vital , input motionl into another bidirectional LSTM network for modeling and feature extraction to obtain H motion .

[0058] Among them, H vital is the main pulse-related feature; H motion Auxiliary features brought about by body motion and contact force changes.

[0059] It should be noted that the bidirectional LSTM network captures the dynamic dependency between the current moment in the sequence and its context through two sets of LSTM units, forward and backward, respectively, thereby strengthening the signal's temporal modeling capability to ensure the effectiveness of feature extraction. In this embodiment 1, the time series length of vital and motion is set to 1024; the number of hidden units in the forward and backward LSTM units is set to 64, then H vital 、H motion It is mapped to 128 dimensions.

[0060] 3. Feature fusion layer is used to: vital 、H motion Processing is performed to obtain the main modal feature representation A vital , auxiliary modal feature representation A motion Based on signal quality perception and dynamic weight fusion mechanism, vital and motion are processed to obtain the main modality fusion weight α vital , auxiliary modality fusion weight α motion ; Based on α vital , α motion To A vital 、A motion Perform weighted summation to obtain the fusion feature representation F fusion , and F fusion Superimposed with the position encoding tensor Position to obtain the time-compensated feature representation New_F fusion .

[0061] The feature fusion layer is designed to achieve deep feature interaction and adaptive fusion between multimodal signals - that is, to extract the potential correlation between different modalities and achieve adaptive weighting based on the quality of each modal signal.

[0062] Specifically, such as Figure 2 As shown:

[0063] 301. The feature fusion layer is equipped with two cross attention mechanisms.

[0064] The cross attention mechanism is designed based on the scaled dot-product attention mechanism, and its calculation formula is:

[0065]

[0066] In the formula, Attn(.) represents the attention calculation function; Q represents the query vector (Query), K represents the key vector (Key), V represents the value vector (Value), and d k is the dimension of K; softmax(.) represents the softmax activation function.

[0067] Then, in the feature fusion layer:

[0068] One of the cross attention mechanisms has Q, K, and V inputs all as H vital , the output is A vital ; That is equivalent to, A vital =Attn(H vital ,H vital ,H vital ). That is, the cross attention mechanism is for H vital The self-attention is designed to extract the correlation representation between the internal temporal sequences of the main modality and obtain the enhanced main modality feature representation A vital .

[0069] The Q input of another cross attention mechanism is H vital , K and V inputs are both H motion , the output is A motion ; That is equivalent to, A motion =Attn(H vital ,H motion ,H motion ). That is, the cross attention mechanism is for H vital 、H motion The cross-modal attention is designed to use the main modality as the query vector, guide the auxiliary modality to participate in the representation, and explore the interactive characteristics between it and the main modality, so as to generate an A that is more in line with the contextual semantics of the main modality. motion .

[0070] So, A vital 、A motion The overall representation ability is improved, and it has stronger context relevance and discriminability.

[0071] 302. In the feature fusion layer, the signal quality perception and dynamic weight fusion mechanism first calculates the quality scores of vital and motion respectively to obtain the main modality quality score Q vital , auxiliary modal quality score Q motion , then based on Q vital , Q motion Perform normalized fusion weight calculation to obtain α vital , α motion .

[0072] Among them, the quality score is calculated using the scoring function f(·), and the normalized fusion weight is calculated using the fusion function α i ;i∈{vital,motion}.

[0073] Ⅰ. For vital, it can analyze the signal-to-noise ratio (SNR) of the pulse wave signal PPG, ECG signal quality index SQI ECG ; For motion, it can analyze the fingertip pressure signal amplitude variance Var Pressure , Entropy of dynamic changes in attitude signals IMU This information serves as input for quality score calculation.

[0074] f(·) is a Sigmoid-activated linear weighting function, which is used to map multiple signal quality indicators into normalized scores in the interval [0, 1], thereby reflecting the overall reliability of the modal signal in the current acquisition window.

[0075] The specific function expression of f(·) is:

[0076] f(x1,x2)=σ(w1x1+w2x2+b);

[0077] Where x1 and x2 are input data; w1 and w2 are weight parameters; b is the bias term; and σ(·) represents the Sigmoid activation function.

[0078] So, Q vital The expression is:

[0079] Q vital =f(SNR PPG ,SQI ECG );

[0080] Q motion The expression is:

[0081] Q motion =f(Var Pressure ,Entropy IMU ).

[0082] II, α i Considering Q vital , Q motion The specific function expression is:

[0083]

[0084] Where, 10 -8 is a constant term that prevents the denominator from being zero.

[0085] 303. For F fusion 、New_F fusion For example, the calculation formula is:

[0086] F fusion =α vital Q vital +α motion Q motion;

[0087] New_F fusion =F fusion +Position;

[0088] Based on the above process, not only is context awareness and interaction enhancement between grouped modalities achieved, but it also has signal quality-driven adaptive weighting capabilities. It can automatically reduce the impact of low-quality groups in scenarios with unstable signal quality, thereby effectively improving the robustness and generalization ability of the model.

[0089] 4. Feature aggregation layer is used to: fusion Global feature processing is performed to obtain SBP and DBP.

[0090] Specifically, such as Figure 2 As shown in the figure, the feature aggregation layer includes: Transformer encoder, multi-layer perceptron, and DropOut layer.

[0091] Transformer encoder is used to: fusion Perform global modeling. The Transformer encoder is equipped with multi-head self-attention to capture long-range dependencies. In this first embodiment, the output of the Transformer encoder is set to 256 dimensions.

[0092] The multilayer perceptron is used to map and reduce the output of the Transformer encoder to obtain the feature vector h global In this embodiment 1, h global It is 64-dimensional.

[0093] The DropOut layer is used to: global Processing is performed to suppress overfitting and mapping is performed to obtain SBP and DBP.

[0094] The above process can be expressed as follows:

[0095] h global =MLP(Encoder(New_F fusion ))

[0096] |y SBP ,y DBP |=DropOut(h global )=W0h global +b0;

[0097] Where Encoder(.) represents the Transformer encoder; MLP(.) represents the multi-layer perceptron; DropOut(.) represents the DropOut layer; W0 represents the weight matrix; b0 represents the bias vector; ySBP Indicates the value of SBP; y DBP Indicates the value of DBP.

[0098] In this embodiment 1, the dimension of W0 is 2×64; the dimension of b0 is 2×1.

[0099] In summary, CrossGate-BiLT combines the advantages of Transformer in long-term time series modeling with the dimensionality reduction expression capability of the MLP structure. It can fully extract the global dependency patterns in the fused features, and cooperate with the multimodal cross-attention and dynamic gating mechanism of the previous stage to achieve unified modeling of multiple physiological signals such as ECG, PPG, pressure and posture, with excellent accuracy and robustness.

[0100] Example 2

[0101] See Figure 3 , demonstrating the blood pressure acquisition system based on multimodal signal fusion provided in this embodiment 2.

[0102] like Figure 3 As shown in the figure, from a functional perspective, the blood pressure acquisition system based on multimodal signal fusion includes: an acquisition device and an intelligent terminal.

[0103] 1. The acquisition device includes: acquisition module, wireless communication module, and power supply module.

[0104] 101. The acquisition module is used to collect the target user's real-time electrocardiogram data EGG, pulse wave data PPG, fingertip pressure data Pressure, and posture data IMU.

[0105] In this embodiment 2, the acquisition module may adopt the following design: including: 2 dual-wavelength photoelectric pulse wave sensors, 2 electrocardiogram sensors, 2 pressure sensors, 1 posture sensor, and 1 MCU microcontroller.

[0106] Two dual-wavelength photoelectric pulse wave sensors are used to collect four pulse wave signals. Because different light sources have different absorption rates for oxygenated and deoxygenated hemoglobin, dual-wavelength sensors can more accurately reflect the oxygen concentration in the finger's blood. This advantage primarily lies in the accuracy, robustness, and reliability of blood oxygen saturation (SpO2). They also offer strong anti-interference properties, preventing interference from ambient light during actual user use. Furthermore, dual-wavelength sensors adapt to measurement needs for different skin tones, body positions, and conditions. Dual-wavelength photoelectric pulse wave sensors can be used: ① The MAX30102, MAX30100, and MAX30105 from the MAX3000 series; ② The KY-039 from the KY series.

[0107] Two ECG sensors are used to collect one ECG signal. These sensors are required to amplify weak ECG signals from the human body and provide filtering for further data processing. They must also be low power and portable. Furthermore, to effectively receive low-level signals without affecting measurement accuracy and ensuring signal quality, they must have high input impedance. Finally, they must exhibit excellent stability and maintain consistent performance over a wide temperature range. Examples of ECG sensors include the AD8232, the MAX30003 from the MAX30000 series, or the ADS1298 from the ADS series.

[0108] Two pressure sensors are used to collect two fingertip pressure signals. There are many common types of pressure sensors on the market, including capacitive, piezoelectric, semiconductor, and thin-film pressure sensors. Thin-film pressure sensors are relatively thin and flexible, suitable for embedding into flexible surfaces. They can closely conform to the shape of the finger and are well-suited for touch pressure acquisition. Examples include the popular FSR series FSR402, FSR406, RP-C30-ST, and RP-C10-S. In short, when selecting a pressure sensor, consider factors such as lightness, size, and comfort with the finger.

[0109] One attitude sensor is used to collect one attitude signal. This attitude sensor uses inertial measurement unit (IMU) technology. Its integrated accelerometer, gyroscope, and magnetometer enable real-time sensing of the device's three-dimensional acceleration, angular velocity, and orientation relative to the Earth's magnetic field, enabling precise measurement of the device's tilt, rotation, and attitude changes. Possible attitude sensors include the MCU6050, MPU9250, BNO055, and LSM9DS1.

[0110] The MCU controls the dual-wavelength photoelectric pulse wave sensor, ECG sensor, pressure sensor, and posture sensor. The MCU must be capable of collecting a wide range of physiological signals and support the required peripheral functions as needed. For example, the MCU can interact with the ECG and pressure sensors via its built-in ADC, the dual-wavelength photoelectric pulse wave sensor via protocols such as I2C or SPI, and the posture sensor via a serial port. Examples of MCUs include the STMicroelectronics STM32 series, Microchip PIC series, ESP32 series, and NXP LPC series.

[0111] 102. The wireless communication module is used to wirelessly transmit EGG, PPG, Pressure, and IMU to the smart terminal.

[0112] The wireless communication module can interact with the MCU microcontroller via serial communication. The wireless transmission module must be capable of data transmission with smart terminals (mobile phones, computers, tablets, etc.). This module can utilize Wi-Fi, a high-data-rate wireless transmission method suitable for applications requiring high bandwidth, especially internet connectivity and local area network communications. For this, the ESP8266, ESP32, and other suitable devices are recommended. Alternatively, Bluetooth, a low-power, easy-to-use data transmission method suitable for short-range wireless communication, can be used. For this, the HC-05, HC-06, ESP32, and BTE-04 are suitable. Other implementation methods are also acceptable, as long as they meet the device's requirements.

[0113] 103. The power module is used to supply power to the acquisition module and the wireless communication module.

[0114] The power module may be designed with a built-in rechargeable battery or a replaceable battery.

[0115] See Figures 4 to 6 ,This embodiment 2 also provides a specific structured design for the collection device;

[0116] Structurally, the acquisition device includes: a shell, an acquisition module, a wireless communication module, and a power supply module.

[0117] The shell serves as the main structure of the device, and the entire shell is designed to be centrally symmetrical.

[0118] The top surface of the housing features two hollow sections, each housing an ECG sensor. The ECG sensor is embedded within the hollow section, leaving the contact electrodes exposed, forming two disc-shaped electrodes on the top surface. This disc-shaped design fits snugly under the fingertips, minimizing the effect of pressure.

[0119] Similar to the top surface, the bottom of the housing also features hollow sections at both ends, each housing a pressure sensor. The pressure sensors are embedded within the hollow sections, leaving the sensing surface exposed, forming two disc-shaped sensing surfaces on the bottom of the housing. This disc-like design fits snugly within the fingertips, minimizing the impact.

[0120] Of course, the above-mentioned hollowing can be designed to be circular or other shapes, but it is necessary to ensure that the sensing effect can meet the requirements.

[0121] Two dual-wavelength photoelectric pulse wave sensors are installed inside the housing, corresponding to the positions of the two disc electrodes. A through-hole is provided in the center of each disc electrode, allowing the dual-wavelength photoelectric pulse wave sensor to sense the PPG signal. Of course, the through-hole is not limited to a square hole; it can be of a reasonable size to ensure the proper function of the dual-wavelength photoelectric pulse wave sensor.

[0122] The shell also houses a posture sensor, an MCU microcontroller, a power module, and a wireless communication module. Their positions can be adjusted according to actual conditions, but should not affect the normal operation of the dual-wavelength photoelectric pulse wave sensor.

[0123] Then, the collection device designed above has good portability and simple collection process:

[0124] When in use, the left and right thumbs press on the two disc electrodes on the top surface to collect one ECG signal; the light emitted by the LED in the dual-wavelength photoelectric pulse wave sensor shines through the through hole on the left and right thumbs, and then the dual-wavelength photoelectric pulse wave sensor receives the reflected light signal, which can synchronously collect four pulse wave signals; the left and right index fingers press on the two disc-shaped sensing surfaces on the back to collect two fingertip pressure signals; the posture sensor in the shell can collect one posture signal.

[0125] 2. The intelligent terminal is used to store the trained blood pressure monitoring network model and adopt the blood pressure acquisition method based on multimodal signal fusion as disclosed in Example 1.

[0126] Smart terminals can be electronic devices such as mobile phones, computers, or tablets, which have functions such as receiving and processing physiological data and interacting with users.

[0127] Example 3

[0128] This embodiment 3 discloses a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the blood pressure acquisition method based on multimodal signal fusion disclosed in embodiment 1.

[0129] Computer devices may include: mobile terminals and fixed terminals. Examples of the former include mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), and in-vehicle terminals (such as in-vehicle navigation terminals); examples of the latter include digital TVs and desktop computers.

[0130] This embodiment 3 also discloses a readable storage medium, which stores computer program instructions. When the computer program instructions are read and executed by a processor, the steps of the blood pressure acquisition method based on multimodal signal fusion disclosed in embodiment 1 are executed.

[0131] Among them, the readable storage medium may include, but is not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0132] This embodiment 3 further discloses a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the blood pressure acquisition method based on multimodal signal fusion disclosed in embodiment 1 are implemented.

[0133] It should be noted that the computer program for executing the above-mentioned program can be written in one or more programming languages or a combination thereof. Among them, the programming language includes object-oriented programming languages such as Java, Smalltalk, C++, and also includes conventional procedural programming languages such as "C" language or similar programming languages. The above-mentioned computer program can be executed completely on the user's computer, or partially on the user's computer, or partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN).

[0134] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A blood pressure collection method based on multimodal signal fusion, which is used to collect the real-time systolic blood pressure (SBP) and real-time diastolic blood pressure (DBP) of a target user, characterized in that: It includes: Step 1: Obtain the target user's real-time ECG data, PPG data, fingertip pressure data, and IMU posture data; Step 2: Input EGG, PPG, Pressure, and IMU into the trained blood pressure monitoring network model for processing to obtain SBP and DBP; Among them, the blood pressure monitoring network model includes: The preprocessing layer is used to process EGG, PPG, Pressure, and IMU into two sets of multi-dimensional time series input sequences; wherein the two sets of multi-dimensional time series input sequences include: a multi-dimensional time series input sequence vital and a multi-dimensional time series input sequence motion; The feature encoding layer is used to process vital and motion based on a bidirectional LSTM network to obtain the output feature H vital , output feature H motion ; The feature fusion layer is used to: vital 、H motion Processing is performed to obtain the main modal feature representation A vital , auxiliary modal feature representation A motion Based on signal quality perception and dynamic weight fusion mechanism, vital and motion are processed to obtain the main modality fusion weight α vital , auxiliary modality fusion weight α motion ; Based on α vital , α motion To A vital 、A motion Perform weighted summation to obtain the fusion feature representation F fusion , and F fusion Superimposed with the position encoding tensor Position to obtain the time-compensated feature representation New_F fusion ; as well as Feature aggregation layer, which is used to: fusion Global feature processing is performed to obtain SBP and DBP.

2. The blood pressure acquisition method based on multimodal signal fusion according to claim 1, characterized in that: In the preprocessing layer, EGG, PPG, Pressure, and IMU are first preprocessed to ensure that all modalities remain synchronized on the time axis. Then, the preprocessed EGG and PPG are spliced into a multi-dimensional time series input sequence vital, and the preprocessed Pressure and IMU are spliced into a multi-dimensional time series input sequence motion.

3. The blood pressure acquisition method based on multimodal signal fusion according to claim 2, characterized in that: If EEG includes X1 channels of ECG signals and PPG includes X2 channels of pulse wave signals, then the feature dimension of vital is X1+X2; If Pressure includes X3 fingertip pressure signals and IMU includes X4 posture signals, the feature dimension of motion is X3+X4.

4. The blood pressure acquisition method based on multimodal signal fusion according to claim 1, characterized in that: The feature encoding layer is equipped with two bidirectional LSTM networks; In the feature encoding layer, vital is input into a bidirectional LSTM network for modeling and feature extraction to obtain H vital , input motionl into another bidirectional LSTM network for modeling and feature extraction to obtain H motion .

5. The blood pressure acquisition method based on multimodal signal fusion according to claim 1, characterized in that: The feature fusion layer has two cross attention mechanisms; the Q, K, and V inputs of one cross attention mechanism are all H vital , the output is A vital ; The Q input of another cross attention mechanism is H vital , K and V inputs are both H motion , the output is A motion .

6. The blood pressure acquisition method based on multimodal signal fusion according to claim 1, characterized in that: In the feature fusion layer, the signal quality perception and dynamic weight fusion mechanism first calculates the quality scores of vital and motion respectively to obtain the main modality quality score Q vital , auxiliary modal quality score Q motion , then based on Q vital , Q motion Perform normalized fusion weight calculation to obtain α vital , α motion .

7. The blood pressure acquisition method based on multimodal signal fusion according to claim 1, characterized in that: Feature aggregation layers include: Transformer encoder, multi-layer perceptron, and DropOut layer; Transformer encoder is used to: fusion Conduct global modeling; The multilayer perceptron is used to map and reduce the output of the Transformer encoder to obtain the feature vector h global ; The DropOut layer is used to: global Processing is performed to suppress overfitting and mapping is performed to obtain SBP and DBP.

8. A blood pressure acquisition system based on multimodal signal fusion, characterized in that: It includes: Collection devices and intelligent terminals; The acquisition device includes: an acquisition module, a wireless communication module, and a power supply module; wherein the acquisition module is used to collect the target user's real-time electrocardiogram (EGG) data, pulse wave (PPG) data, fingertip pressure data (Pressure), and posture data (IMU); the wireless communication module is used to wirelessly transmit the EGG, PPG, Pressure, and IMU data to the smart terminal; and the power supply module is used to power the acquisition module and the wireless communication module. The intelligent terminal is used to store the trained blood pressure monitoring network model and adopt the blood pressure acquisition method based on multimodal signal fusion as described in any one of claims 1 to 7.

9. The blood pressure acquisition system based on multimodal signal fusion according to claim 8, characterized in that: The acquisition module includes: 2 dual-wavelength photoelectric pulse wave sensors, used to collect 4-channel pulse wave signals; 2 ECG sensors, used to collect 1 channel of ECG signals; 2 pressure sensors, used to collect 2-way fingertip pressure signals; 1 attitude sensor, used to collect 1-way attitude signal; as well as 1 MCU microcontroller, which is used to control the operation of the dual-wavelength photoelectric pulse wave sensor, electrocardiogram sensor, pressure sensor, and posture sensor.

10. A computer program product, characterized in that The method comprises a computer program; when the computer program is executed by a processor, the method implements the steps of the blood pressure acquisition method based on multimodal signal fusion as described in any one of claims 1 to 7.

Images