Hybrid mode sensor intelligent system for non-invasive characterization of multiple physiological parameters

Abstract

The invention provides a mixed mode sensor intelligent system for non-invasively representing various physiological parameters. The mixed mode sensor intelligent system comprises a multi-mode sensor array, an analog-to-digital conversion module, a signal preprocessing module and a signal analysis module, wherein the multi-mode sensor array is fixed below the bed board through an iron sheet, acquires an analog signal of a multi-mode sensor, and sends the analog signal to the analog-to-digital conversion module; the analog-to-digital conversion module is used for denoising and amplifying an analog signal, converting the analog signal into a digital signal, and sending the digital signal to the signal preprocessing module; the signal preprocessing module is used for carrying out standardization and filtering processing on the digital signal and carrying out poor-quality signal screening to obtain a preprocessed signal; and the signal analysis module is used for analyzing the preprocessed signal to obtain a real-time monitoring result.

Description

Technical Field

[0001] The invention relates to a hybrid modal sensor intelligent system, in particular to a hybrid modal sensor intelligent system for non-invasively characterizing multiple physiological parameters. Background Art

[0002] This section merely provides background information related to the present disclosure and is not necessarily prior art.

[0003] Real-time monitoring of human vital signs, such as heart rate (HR), respiratory rate (RR), blood pressure (BP), and cardiac output (CO), has important application value. Among them, real-time heart rate monitoring can quickly capture abnormal conditions of heart movement, real-time monitoring of respiratory rate can make timely warnings for apnea during sleep, and cardiac output monitoring can evaluate the heart's working efficiency and blood circulation in real time, which is of great significance for the diagnosis of heart disease.

[0004] At present, real-time vital signs monitoring methods are mainly divided into three categories:

[0005] 1. Real-time monitoring of human vital signs through wearable devices. However, wearable devices are not suitable for everyone and need to come into contact with the skin, which greatly increases the user's discomfort. In addition, most wearable devices need to be charged on time, which causes great inconvenience for elderly users.

[0006] 2. Real-time monitoring of vital signs during sleep through a special mattress. This method is greatly affected by the material of the mattress and usually requires the user to place the heart directly above the sensor. At the same time, enuresis and other conditions will greatly affect the accuracy of the sensor, resulting in significant errors in the measurement results.

[0007] 3. Real-time monitoring of vital signs through bedside monitoring equipment, including radar or camera. Among them, the radar-based solution exposes the user to radar radiation for a long time, which poses potential health risks, while the camera-based system will expose the user's privacy to varying degrees, affecting the user's willingness to use it.

[0008] At the same time, the current solutions do not involve real-time monitoring of blood pressure and cardiac output, and cannot effectively replace the comprehensive monitoring functions of invasive monitors.

[0009] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0010] Purpose of the invention: The technical problem to be solved by the present invention is to provide a hybrid modality sensor intelligent system for non-invasively characterizing multiple physiological parameters in view of the deficiencies in the prior art.

[0011] In order to solve the above technical problems, the present invention discloses a hybrid modality sensor intelligent system for non-invasively characterizing multiple physiological parameters, comprising:

[0012] Multimodal sensor array, analog-to-digital conversion module, signal preprocessing module and signal analysis module; wherein,

[0013] The multimodal sensor array is fixed under the bed board through an iron sheet, collects analog signals of the multimodal sensors, and sends them to the analog-to-digital conversion module;

[0014] The analog-to-digital conversion module converts the analog signal into a digital signal after performing noise removal and amplification processing, and sends the digital signal to the signal preprocessing module;

[0015] The signal preprocessing module performs standardization and filtering on the digital signal and screens out inferior signals to obtain a preprocessed signal;

[0016] The signal analysis module analyzes the preprocessed signal to obtain real-time monitoring results.

[0017] Furthermore, the multimodal sensor array comprises:

[0018] 3 groups of multimodal sensors, each of which includes: an acceleration sensor, a displacement sensor and a temperature sensor; the acceleration sensor and the displacement sensor are used to capture vibrations caused by human heart movement; the temperature sensor is used to monitor room temperature;

[0019] The three groups of multimodal sensors in the multimodal sensor array are respectively located below the bed board positions corresponding to the carotid artery, left chest and right chest.

[0020] Furthermore, the process of converting the analog signal into a digital signal after denoising and amplifying the analog signal includes:

[0021] Use an RC bandpass filter to filter the collected analog signal to remove low-frequency noise and high-frequency noise, and keep the acceleration signal and displacement signal frequency in the analog signal within a preset range;

[0022] The gain of the acceleration signal and the displacement signal is adjusted by the amplifier to increase the amplitude of the signal;

[0023] Select the number of bits and sampling rate of the analog-to-digital converter to obtain a digital signal.

[0024] Furthermore, the standardization and filtering of the digital signal includes:

[0025] Perform Z-score transformation on the digital signal as follows:

[0026]

[0027] Where x is the input digital signal, mean(x) is the mean of the input digital signal, and std(x) is the standard deviation of the input digital signal. It is a standard signal after the input digital signal is standardized;

[0028] The standard signal is decomposed by wavelet, which is realized by using one high-pass filter and one low-pass filter, as shown below:

[0029]

[0030] Among them, y low [n] and y high [n] represents the low-frequency part and high-frequency part after filtering, l[·] and h[·] represent the low-pass filter and high-pass filter, x[·] represents the low-frequency part of the previous stage, and n and k represent the time;

[0031] The required frequency part is retained and wavelet reconstruction is performed to obtain the reconstructed signal.

[0032] Furthermore, the poor quality signal screening is to perform human body movement detection to screen out poor quality signals caused by human body movement.

[0033] Furthermore, the human body movement detection includes:

[0034] The acceleration signal and displacement signal in the reconstructed signal are segmented using a sliding window, and the kurtosis Kurt(X) of each signal segment is calculated as follows:

[0035]

[0036] Where X is the acceleration signal and displacement signal, μ is the mean of the signal, σ is the variance of the signal, and E[] is the mean;

[0037] The number of abnormal values ​​in the signal segment is obtained through the kurtosis Kurt(X). When the number exceeds the threshold, it is determined that human body movement exists in the signal segment.

[0038] Furthermore, the analysis of the preprocessed signal includes:

[0039] Heart rate calculation, respiratory rate calculation, and blood pressure and cardiac output measurement.

[0040] Furthermore, the heart rate calculation includes:

[0041] The envelope of the acceleration signal in the preprocessed signal is calculated, and a polynomial fitting is performed on the acceleration signal using a Savitzky–Golay filter;

[0042] Use a sliding window to segment the fitted acceleration signal, calculate the autocorrelation of each signal segment, obtain the autocorrelation result, and calculate the power spectrum density function of the autocorrelation result; then perform Softmax transformation on the autocorrelation result and the power spectrum respectively to obtain the posterior probability density function P(HR|ACF) and P(HR|PSD) of the heart rate for the autocorrelation result and the power spectrum; finally, multiply the two probability density functions to obtain the heart rate corresponding to the maximum probability and the heart rate calculation result.

[0043] Furthermore, the respiratory rate calculation includes:

[0044] Perform wavelet decomposition on the displacement signal and reconstruct the part that retains the target frequency range;

[0045] The reconstructed signal is smoothed using a large-scale average filter, that is, the respiratory signal is separated, and the respiratory signal is segmented using a sliding window, and the autocorrelation of the original displacement signal, the filtered signal, and the smoothed signal of each segment of the respiratory signal is calculated;

[0046] Perform Softmax transformation on the three autocorrelation results to obtain the posterior probability density functions of the respiratory rate for the three: P(RR|displacement_signal), P(RR|filter_signal), and P(RR|smoothed_signal);

[0047] The three probability density functions are multiplied to obtain the respiratory rate corresponding to the maximum probability, and the respiratory rate calculation result is obtained.

[0048] Furthermore, the blood pressure and cardiac output measurement includes:

[0049] Build pre-trained deep neural networks;

[0050] The matrix composed of the original acceleration signal and displacement signal and the temperature data measured by the temperature sensor are input into the deep neural network in real time, and the basic parameter data of the target to be measured are input as the input features of the deep neural network, and the blood pressure and cardiac output measurement results of the target to be measured are output;

[0051] The deep neural network adopts a residual-recurrent feature fusion neural network, including a residual convolution part and a recurrent neural network part; wherein the residual convolution part includes three one-dimensional convolution kernels of different scales, and the convolution kernel sizes are 3, 5 and 7 respectively. Each layer of convolution passes through an activation layer and a pooling layer, and is added to the input sequence after the last layer of pooling. At the same time, a learnable mapping matrix is ​​used to map the basic parameters of the input target to be measured into a feature map, which is stacked with the feature maps of each level in the convolutional neural network; the recurrent neural network part includes three layers of long short-term memory network layers, and a variational dropout layer is used for generalization. Finally, two linear layers are passed to obtain the systolic pressure, diastolic pressure and cardiac output of the target to be measured.

[0052] Beneficial effects:

[0053] The present invention proposes a hybrid modal sensor intelligent system for non-invasively characterizing multiple physiological parameters. Based on the original vibration signals collected by the acceleration sensor and the displacement sensor and the temperature value measured by the temperature sensor, a series of signal processing methods are used to make the multi-modal signals refer to each other and jointly calculate. At the same time, advanced artificial intelligence means are used for different vital signs to achieve a non-contact, portable and miniaturized vital sign monitoring method, which can comprehensively characterize the overall picture of cardiac hemodynamics and make up for the defects of existing vital sign monitoring methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0055] Figure 1 It is a schematic diagram of the installation of the multimodal sensor array of the present invention.

[0056] Figure 2 This is the system workflow diagram

[0057] Figure 3 It is a schematic diagram of the neural network structure in this example. DETAILED DESCRIPTION

[0058] In view of the shortcomings of existing real-time vital sign monitoring technologies, the present invention provides a hybrid modality sensor intelligent system for non-invasively characterizing multiple physiological parameters, comprising the following steps:

[0059] Step 1: fix the multimodal sensor array on the iron sheet; fix the multimodal sensor array and the iron sheet under the bed board.

[0060] Step 2: Use a multimodal sensor array to collect sensor signals, and convert the analog signals into digital signals after denoising and amplification.

[0061] Step 3: standardize and filter the original acceleration signal and displacement signal, analyze the signal quality, and filter out the inferior signal caused by human body movement;

[0062] Step 4: Apply the corresponding signal processing algorithm to analyze the obtained high-quality vibration signal, calculate the heart rate and respiratory rate through the signal processing algorithm, and apply advanced artificial intelligence algorithms to measure blood pressure, cardiac output, and other vital signs, ultimately achieving real-time monitoring of various important vital signs of the human body.

[0063] In the present invention, preferably, the multimodal sensor array described in step 1 is 3 groups of high-precision acceleration sensors, displacement sensors and temperature sensors, the acceleration sensors are used to capture tiny vibrations caused by human blood circulation; the displacement sensors are used to measure the structural displacement changes below the sensor array and capture the displacement signals caused by the weight changes of various parts of the human body; the temperature sensor is used to monitor the room temperature to guide the measurement results; the frequency response of the acceleration sensor and the displacement sensor is 0.1Hz to 15Hz.

[0064] In the present invention, preferably, the thickness of the iron sheet in step 1 is 0.5 mm. The high rigidity of the iron sheet can effectively reduce the attenuation and dispersion of vibration, ensuring that the signal collected by the multimodal sensor array is more stable and repeatable.

[0065] In the present invention, preferably, the multimodal sensor array and the iron sheet are fixed under the bed board in step 1, and the fixed position should make the three groups of sensors located at the three vertices of an equilateral triangle, and the three groups of sensors should be located under the bed board positions corresponding to the carotid artery, left chest, and right chest.

[0066] In the present invention, preferably, the denoising process of the analog signal in step 2 is to filter the collected signal through an RC bandpass filter on a hardware circuit to remove low-frequency noise and high-frequency noise to ensure that the signal is 0.01 Hz to 15 Hz.

[0067] In the present invention, preferably, the analog signal is amplified and converted into a digital signal in step 2, and the gain of the collected acceleration signal and stress signal is adjusted by the amplifier on the hardware circuit to increase the amplitude of the signal, so that it is close to the input range of the ADC and the sampling accuracy is improved. A 16-bit ADC is selected to sample the signal at 200Hz to obtain a high-precision digital signal.

[0068] In the present invention, preferably, the signal standardization and filtering processing described in step 3 is specifically implemented by first performing a Z-score transformation on the original acceleration signal and the displacement signal. The specific calculation method is as follows:

[0069]

[0070] Where x is the original acceleration signal and displacement signal, mean(x) is the mean of the signal, and std(x) is the standard deviation of the signal, thereby achieving standardized processing of the original signal. Then, the original signal is decomposed by wavelet. The decomposition process can be represented by a high-pass filter and a low-pass filter. The specific calculation method is as follows:

[0071]

[0072] where y low [n] and y high [n] represents the low-frequency part and high-frequency part after filtering, l[·] and h[·] represent the low-pass filter and high-pass filter, x[·] represents the low-frequency part of the previous stage, and n and k represent the time points. Then, the target frequency part is reconstructed to complete the standardization and filtering of the original vibration signal.

[0073] In the present invention, preferably, the human body movement detection described in step 3 is specifically implemented by segmenting the acceleration signal and the displacement signal using a sliding window, and calculating the kurtosis of each signal segment. The calculation method is as follows:

[0074]

[0075] Where X is the original acceleration signal and displacement signal, μ is the mean of the signal, σ is the variance of the signal, and E[] is the mean. By calculating the kurtosis of the signal, the number of abnormal values ​​in the signal can be obtained. Then, threshold processing can be used to determine whether there is human movement within the signal time period.

[0076] In the present invention, preferably, the specific implementation method of the heart rate calculation described in step 4 is to first perform Hilbert transform on the signal to obtain the envelope of the signal. The specific calculation method is as follows:

[0077]

[0078] where H[·] represents the Hilbert transform, x(t) and x i (t) represents the real part and imaginary part of the signal, respectively, a(t) represents the envelope of the signal, and then the Savitzky–Golay filter is used to perform polynomial fitting on the signal, setting the window size W and the polynomial degree N to minimize the difference between the signal and the filtered signal. The specific calculation process is as follows:

[0079]

[0080] Among them, ε N represents the fitted signal, M is half of the window size, a kRepresents the k-order coefficient of the polynomial, x[n] represents the original signal, n and k represent the time points. By obtaining the minimum value of the equation, the signal is fitted with a polynomial to further remove the noise in the signal. Then, the sliding window is used to segment the signal, and the autocorrelation (ACF) of each signal segment is calculated to obtain the correlation of the signal under each time delay. The specific calculation method is as follows:

[0081]

[0082] in, is the mean of the signal, and then the power spectral density function (PSD) of the autocorrelation result is calculated using the Welch method to obtain the frequency characteristics of the autocorrelation function. The Welch method involves estimating the windowed PSD from overlapping segments, and averaging these windowed PSDs to make the final PSD smoother and more robust.

[0083] Then, Softmax transformation is performed on ACF and PSD to obtain the posterior probability density function P(HR|ACF) and P(HR|PSD) of the heart rate (HR) for the autocorrelation result and the power spectrum. The calculation process is as follows:

[0084]

[0085] P(HR|ACF)=Softmax(ACF)

[0086] P(HR|PSD)=Softmax(PSD)

[0087] Where P(HR|ACF) represents the posterior probability of HR for ACF, and P(HR|PSD) represents the posterior probability of HR for PSD. Then, we only need to find the maximum frequency point in the joint probability density of HR for ACF and PSD to complete the calculation of heart rate. The specific calculation process is as follows:

[0088] P(HR|PSD,ACF)∝P(HR)·P(ACF|HR)·P(PSD|HR)

[0089]

[0090] Assuming that HR is uniformly distributed, we can further simplify by removing P(HR), and finally obtain:

[0091] P(HR|PSD,ACF)∝P(HR|ACF)·P(HR|PSD).

[0092] The heart rate measurement can be completed by taking the frequency corresponding to the maximum value in the joint probability density function.

[0093] In the present invention, preferably, the specific implementation method of the respiratory rate measurement described in step 4 is to use wavelet filtering on the original displacement signal, reconstruct the signal within the specified frequency range, and use a large-scale average filter to smooth the signal to achieve separation of the respiratory signal, and then use a sliding window to segment the respiratory signal. For each segment of the signal, the original displacement signal (displacement_signal), filtered signal (filter_signal) and smoothed signal (smoothed_signal) of the separated respiratory signal are obtained, and the autocorrelation (ACF) is calculated using the same method as the heart rate measurement. Then, the three autocorrelation functions are softmax transformed to obtain the posterior probability density function P(RR|displacement_signal), P(RR|filter_signal), and P(RR|smoothed_signal) of the respiratory rate (RR) for the three, and finally the three probability density functions are multiplied to obtain the respiratory rate corresponding to the maximum probability, thereby completing the measurement of the respiratory rate.

[0094] In the present invention, preferably, the blood pressure and cardiac output measurements described in step 4 are specifically performed by standardizing the original vibration signal and the original temperature measurement value, while inputting basic parameters such as the user's height, weight, pathological characteristics, and medication status, and then using a neural network to predict the multimodal signal.

[0095] When inputting signals, the original signals are weighted for different prediction targets. Since blood pressure is the lateral pressure per unit area of ​​blood vessels, it is related to acceleration, so the acceleration signal is used as the main input and the displacement signal as a supplement. For cardiac output, the velocity signal is the main input and the other signals are used as supplements.

[0096] The neural network part uses a learnable mapping matrix to map the basic input information into a feature map, and fuses the features with the feature maps of each level of the backbone network to make it work globally. After training with a large amount of data, it can predict the blood pressure and cardiac output of the subjects.

[0097] Example:

[0098] The following is a specific embodiment to further illustrate the hybrid modality sensor intelligent system for non-invasively characterizing multiple physiological parameters proposed by the present invention. Figure 2 As shown, the following steps are included:

[0099] Step 1, such as Figure 1 As shown, the multimodal sensor array is fixed on the iron sheet; the multimodal sensor array and the iron sheet are fixed under the bed board.

[0100] In this example, the multimodal sensor array consists of three groups of accelerometers, displacement sensors, temperature sensors, PCB circuit boards and 3D printed molds. The accelerometer is Msv6000, which is based on MEMS (micro-electromechanical system) technology. It integrates a three-axis accelerometer and a high-precision ADC, and can detect the acceleration signals of the device in the three directions of X, Y and Z. The displacement sensor is Olympus VN-8000, which is used to collect small displacement changes caused by seismic waves. The temperature sensor is STS35, with a measurement range of -40 to 125℃, which is used to monitor the room temperature and guide the measurement results. The PCB circuit board is 55mm×75mm in size, with corresponding circuits and ADCs for processing the collected signals. The 3D printed mold is 60mm×80mm×60mm, with two magnets on the mold, the iron plate is 0.5mm spring steel, with a size of 50cm×80cm, and the bed board is 120cm×200cm. The multimodal sensor array is fixed to an iron plate by a magnet, and the iron plate is fixed to the bottom of the bed. The three groups of multimodal sensor arrays are distributed in the human body's carotid artery and the area below the left and right chest cavities.

[0101] It should be understood that the size of the iron plate can be adjusted accordingly according to different bed board types.

[0102] Step 2: Use a multimodal sensor array to collect sensor signals, and convert the analog signals into digital signals after denoising and amplification.

[0103] The analog signal is de-noised by filtering the collected signal through the RC bandpass filter on the hardware circuit to remove low-frequency noise and high-frequency noise, ensuring that the signal is 0.01HZ to 10HZ. The bandpass filter realizes bandpass filtering by connecting a high-pass filter and a low-pass filter in series. The calculation method is as follows:

[0104]

[0105] Among them, f c The cut-off frequency is R, the resistor is R, and the capacitor is C. R=100kΩ, C=159μF and R=100kΩ, C=15.9μF are respectively selected to achieve filtering from 0.01Hz to 10Hz. The analog signal is amplified and converted into a digital signal, and the gain of the collected acceleration signal and stress signal is adjusted by the amplifier on the hardware circuit to increase the amplitude of the signal, so that it is close to the input range of the ADC and improve the sampling accuracy. A 16-bit ADC is selected to sample the signal at 200Hz to obtain a high-precision digital signal.

[0106] It should be understood that the RC bandpass filter and the gain circuit can adjust parameters accordingly according to different bed board types.

[0107] Step 3: standardize and filter the original vibration signal, analyze the signal quality, and filter out the inferior signal caused by human body movement;

[0108] The signal standardization and filtering in this example is specifically implemented by first performing a Z-score transformation on the original vibration signal. The specific calculation method is as follows:

[0109]

[0110] Where x is the original vibration signal, mean(x) is the mean of the signal, and std(x) is the standard deviation of the signal, thereby achieving standardization of the original signal. Then, the original signal is decomposed by wavelet. The decomposition process can be represented by a high-pass filter and a low-pass filter. The specific calculation method is as follows:

[0111]

[0112] where y low [n] and y high [n] represents the low-frequency part and high-frequency part after filtering, l[·] and h[·] represent the low-pass filter and high-pass filter, x[·] represents the low-frequency part of the previous stage, and n and k represent time points.

[0113] The wavelet basis used in this example is db12, because the wavelet basis has a high similarity with the vibration signal. The number of decomposition layers is 6, and the frequency ranges of each layer are [25,50], [12.5,25], [6.25,12.5], [3.125,6.25], [1.5625,3.125], [0.78125,1.5625] and [0,0.78125]. Then the target frequency part is reconstructed. In this example, the target frequency range is 0.8-12.5Hz, so the third to sixth detail layers are selected for reconstruction, thereby completing the standardization and filtering of the original vibration signal.

[0114] In this example, the human motion detection is implemented by segmenting the vibration signal into 10-second sliding windows and calculating the kurtosis of each signal segment. The calculation method is as follows:

[0115]

[0116] Where X is the vibration signal, μ is the mean of the signal, σ is the variance of the signal, and E[] is the mean. The number of abnormal values ​​in the signal can be obtained by calculating the kurtosis of the signal. In this example, Kurt(X)>2 is used as the threshold to filter out motion signals.

[0117] Step 4: Apply the corresponding signal processing algorithm to analyze the obtained high-quality vibration signal, calculate the heart rate and respiratory rate through the signal processing algorithm, and apply advanced artificial intelligence algorithms to measure vital signs such as blood pressure, cardiac output, dynamic body weight, etc., ultimately achieving real-time monitoring of various important vital signs of the human body.

[0118] The specific implementation method of the heart rate measurement in this example is to first perform Hilbert transform on the signal and calculate the envelope of the signal. The specific calculation method is as follows:

[0119]

[0120]

[0121] where H[·] represents the Hilbert transform, x(t) and x o (t) represents the real part and imaginary part of the signal, respectively, a(t) represents the envelope of the signal, and then the Savitzky–Golay filter is used to perform polynomial fitting on the signal, setting the window size W = 21 and the polynomial order N = 2 to minimize the difference between the signal and the filtered signal. The specific calculation process is as follows:

[0122]

[0123] Among them, ε N represents the fitted signal, M is half of the window size, a k Represents the k-order coefficient of the polynomial, x[n] represents the original signal, n and k represent the time points. By obtaining the minimum value of the equation, the signal is fitted with a polynomial to further remove the noise in the signal. The signal is divided into sliding windows of 10 seconds, and the autocorrelation (ACF) of each signal segment is calculated to obtain the correlation of the signal under each time delay. The specific calculation method is as follows:

[0124]

[0125] in is the mean value of the signal, and then the power spectral density function (PSD) of the autocorrelation result is calculated using the Welch method to obtain the frequency characteristics of the autocorrelation function. The Welch method involves estimating the windowed PSD from the overlapping segments, and averaging these windowed PSDs to make the final PSD smoother and more robust. In this example, in order to meet the accuracy requirements, the resolution of the FFT is set to 0.001.

[0126] Then, Softmax transformation is performed on ACF and PSD to obtain the posterior probability density function P(HR|ACF) and P(HR|PSD) of the heart rate (HR) for the autocorrelation result and the power spectrum. The calculation process is as follows:

[0127]

[0128] P(HR|ACF)=Softmax(ACF)

[0129] P(HR|PSD)=Softmax(PSD)

[0130] Where P(HR|ACF) represents the posterior probability of HR for ACF, and P(HR|PSD) represents the posterior probability of HR for PSD. Then, we only need to find the maximum frequency point in the joint probability density of HR for ACF and PSD to complete the calculation of heart rate. The specific calculation process is as follows:

[0131] P(HR|PSD,ACF)∝P(HR)·P(ACF|HR)·P(PSD|HR)

[0132]

[0133] Assuming that HR is uniformly distributed, we can further simplify by removing P(HR), and finally obtain:

[0134] P(HR|PSD,ACF)∝P(HR|ACF)·P(HR|PSD).

[0135] The heart rate measurement can be completed by taking the frequency corresponding to the maximum value in the joint probability density function.

[0136] The respiratory rate measurement in this example is specifically implemented as follows: first, the original displacement signal is subjected to wavelet filtering using db12 as the mother wavelet, and the signal in the range of [0-0.75] Hz is reconstructed. Then, the signal is smoothed using an average filter with a window length of 71 to achieve respiratory signal separation. The respiratory signal is divided into a sliding window of 15 seconds. For each signal segment, the original displacement signal (detrend_signal), filtered signal (filter_signal) and smoothed signal (smoothed_signal) of the separated respiratory signal are obtained. The autocorrelation (ACF) is calculated using the same method as the heart rate measurement. Then, the three autocorrelation functions are softmax transformed to obtain the posterior probability density functions P(RR|detrend_signal), P(RR|filter_signal), and P(RR|smoothed_signal) of the respiratory rate (RR) for the three. Finally, the three probability density functions are multiplied to obtain the respiratory rate corresponding to the maximum probability, and the measurement of the respiratory rate has been completed.

[0137] In this example, the blood pressure measurement method is to standardize the original multimodal vibration signal so that all samples have the same distribution, and input the user's height, weight, pathological characteristics, medication status and other basic parameters and the temperature measured by the temperature sensor, and then use Figure 3 The residual-recurrent feature fusion neural network (Res-LSTM) shown in the figure predicts multi-sequence signals, where the basic module of the residual convolution part is three one-dimensional convolution kernels of different scales, with convolution kernel sizes of 3, 5, and 7 respectively. Each layer of convolution passes through an activation layer and a pooling layer, and is added to the original sequence after the last layer of pooling. At the same time, a learnable mapping matrix is ​​used to map the basic parameters to feature maps, which are stacked with feature maps at all levels. The recurrent neural network part includes three LSTM layers, and a variational dropout layer is applied to increase the generalization of the network. Finally, two linear layers are used to obtain systolic blood pressure (SBP), diastolic blood pressure (DBP) and cardiac output (CO) to achieve the measurement of blood pressure and cardiac output.

[0138] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium can store a computer program, and when the computer program is executed by the data processing unit, the invention content of a hybrid modality sensor intelligent system for non-invasively characterizing multiple physiological parameters provided by the present invention and some or all of the steps in each embodiment can be run. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0139] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on such an understanding, the technical solutions in the embodiments of the present invention can be essentially or partly contributed to the prior art in the form of computer programs, i.e., software products, which can be stored in a storage medium and include several instructions for enabling a device including a data processing unit (which can be a personal computer, a server, a single-chip microcomputer, an MCU or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0140] The present invention provides a concept and method of a hybrid modal sensor intelligent system for non-invasively characterizing multiple physiological parameters. There are many methods and ways to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A hybrid modality sensor intelligent system for non-invasively characterizing multiple physiological parameters, characterized in that: include: Multimodal sensor array, analog-to-digital conversion module, signal preprocessing module and signal analysis module; wherein, The multimodal sensor array is fixed under the bed board through an iron sheet, collects analog signals of the multimodal sensors, and sends them to the analog-to-digital conversion module; The analog-to-digital conversion module converts the analog signal into a digital signal after performing noise removal and amplification processing, and sends the digital signal to the signal preprocessing module; The signal preprocessing module performs standardization and filtering on the digital signal and screens out inferior signals to obtain a preprocessed signal; The signal analysis module analyzes the preprocessed signal to obtain real-time monitoring results.

2. The hybrid modality sensor intelligent system for non-invasively characterizing multiple physiological parameters according to claim 1, characterized in that: The multimodal sensor array comprises: 3 groups of multimodal sensors, each of which includes: an acceleration sensor, a displacement sensor and a temperature sensor; the acceleration sensor and the displacement sensor are used to capture vibrations caused by human heart movement; the temperature sensor is used to monitor room temperature; The three groups of multimodal sensors in the multimodal sensor array are respectively located below the bed board positions corresponding to the carotid artery, left chest and right chest.

3. The hybrid modality sensor intelligent system for non-invasively characterizing multiple physiological parameters according to claim 2, characterized in that: The process of converting the analog signal into a digital signal after performing noise removal and amplification processing includes: Use an RC bandpass filter to filter the collected analog signal to remove low-frequency noise and high-frequency noise, and keep the acceleration signal and displacement signal frequency in the analog signal within a preset range; The gain of the acceleration signal and the displacement signal is adjusted by the amplifier to increase the amplitude of the signal; Select the number of bits and sampling rate of the analog-to-digital converter to obtain a digital signal.

4. The hybrid modality sensor intelligent system for non-invasively characterizing multiple physiological parameters according to claim 3, characterized in that: The standardization and filtering of the digital signal includes: Perform Z-score transformation on the digital signal as follows: Where x is the input digital signal, mean(x) is the mean of the input digital signal, and std(x) is the standard deviation of the input digital signal. It is a standard signal after the input digital signal is standardized; The standard signal is decomposed by wavelet, which is realized by using one high-pass filter and one low-pass filter, as shown below: Among them, y low [n] and y high [n] represents the low-frequency part and high-frequency part after filtering, l[·] and h[·] represent the low-pass filter and high-pass filter, x[·] represents the low-frequency part of the previous stage, and n and k represent the time; The required frequency part is retained and wavelet reconstruction is performed to obtain the reconstructed signal.

5. The hybrid modality sensor intelligent system for non-invasively characterizing multiple physiological parameters according to claim 4, characterized in that: The poor quality signal screening is to perform human body movement detection to screen out poor quality signals caused by human body movement.

6. The hybrid modality sensor intelligent system for non-invasively characterizing multiple physiological parameters according to claim 5, characterized in that: The human body movement detection comprises: The acceleration signal and displacement signal in the reconstructed signal are segmented using a sliding window, and the kurtosis Kurt(X) of each signal segment is calculated as follows: Where X is the acceleration signal and displacement signal, μ is the mean of the signal, σ is the variance of the signal, and E[] is the mean; The number of abnormal values ​​in the signal segment is obtained through the kurtosis Kurt(X). When the number exceeds the threshold, it is determined that human body movement exists in the signal segment.

7. The hybrid modality sensor intelligent system for non-invasively characterizing multiple physiological parameters according to claim 6, characterized in that: The analysis of the preprocessed signal includes: Heart rate calculation, respiratory rate calculation, and blood pressure and cardiac output measurement.

8. The hybrid modality sensor intelligent system for non-invasively characterizing multiple physiological parameters according to claim 7, characterized in that: The heart rate calculation includes: The envelope of the acceleration signal in the preprocessed signal is calculated, and a polynomial fitting is performed on the acceleration signal using a Savitzky–Golay filter; Use a sliding window to segment the fitted acceleration signal, calculate the autocorrelation of each signal segment, obtain the autocorrelation result, and calculate the power spectrum density function of the autocorrelation result; then perform Softmax transformation on the autocorrelation result and the power spectrum respectively to obtain the posterior probability density function P(HR|ACF) and P(HR|PSD) of the heart rate for the autocorrelation result and the power spectrum; finally, multiply the two probability density functions to obtain the heart rate corresponding to the maximum probability and the heart rate calculation result.

9. The hybrid modality sensor intelligent system for non-invasively characterizing multiple physiological parameters according to claim 8, characterized in that: The respiratory rate calculation includes: Perform wavelet decomposition on the displacement signal and reconstruct the part that retains the target frequency range; The reconstructed signal is smoothed using a large-scale average filter, that is, the respiratory signal is separated, and the respiratory signal is segmented using a sliding window, and the autocorrelation of the original displacement signal, the filtered signal, and the smoothed signal of each segment of the respiratory signal is calculated; Perform Softmax transformation on the three autocorrelation results to obtain the posterior probability density functions of the respiratory rate for the three: P(RR|displacement_signal), P(RR|filter_signal), and P(RR|smoothed_signal); The three probability density functions are multiplied to obtain the respiratory rate corresponding to the maximum probability, and the respiratory rate calculation result is obtained.

10. The hybrid modality sensor intelligent system for non-invasively characterizing multiple physiological parameters according to claim 9, characterized in that: The blood pressure and cardiac output measurement includes: Build pre-trained deep neural networks; The matrix composed of the original acceleration signal and displacement signal and the temperature data measured by the temperature sensor are input into the deep neural network in real time, and the basic parameter data of the target to be measured are input as the input features of the deep neural network, and the blood pressure and cardiac output measurement results of the target to be measured are output; The deep neural network adopts a residual-recurrent feature fusion neural network, including a residual convolution part and a recurrent neural network part; wherein the residual convolution part includes three one-dimensional convolution kernels of different scales, and the convolution kernel sizes are 3, 5 and 7 respectively. Each layer of convolution passes through an activation layer and a pooling layer, and is added to the input sequence after the last layer of pooling. At the same time, a learnable mapping matrix is ​​used to map the basic parameters of the input target to be measured into a feature map, which is stacked with the feature maps of each level in the convolutional neural network; the recurrent neural network part includes three layers of long short-term memory network layers, and a variational dropout layer is used for generalization. Finally, two linear layers are passed to obtain the systolic pressure, diastolic pressure and cardiac output of the target to be measured.

Images