Multi-sensor fusion physiological signal monitoring method and device

The physiological signal monitoring method using multi-sensor fusion and deep learning algorithms solves the problems of cumbersome sensor monitoring and sleep interference, and realizes non-sensory and accurate physiological signal monitoring and sleep analysis, thereby improving the level of health management.

CN119014824AInactive Publication Date: 2024-11-26NINGBO ARTIFICIAL INTELLIGENCE RES INST OF SHANGHAI JIAOTONG UNIV

Patent Information

Application Number
CN202411526336.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-30
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure CN119014824A_ABST
    Figure CN119014824A_ABST
Patent Text Reader

Abstract

The invention discloses a multi-sensor fusion physiological signal monitoring method and device, and relates to the field of artificial intelligence technology and medical health monitoring, and the method comprises the steps: 1, when a flexible sensor detects a pressure signal, an MCU is started to start working; step 2, the MCU synchronously awakens the flexible sensor and the microphone array to perform signal acquisition, the flexible sensor acquires a motion signal, and the microphone array acquires a breath sound signal; step 3, firstly enhancing and separating the breathing sound signals, and then assisting in enhancing and separating the body movement signals; 4, snore detection is conducted in the breath sound signals, and a snore result is obtained; heart rate monitoring, respiration monitoring and pulse monitoring are conducted in the body movement signals, and a heart rate result, a respiration result and a pulse result are obtained respectively; and step 5, sleep analysis is carried out by using a multi-stage segmentation depth clustering algorithm, the sleep quality of each time period is determined, and the sleep quality is divided into five states: waking, light sleep N1, light sleep N2, deep sleep and quick eye movement.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence technology and medical health monitoring, and in particular to a multi-sensor fusion physiological signal monitoring method and device. Background Art

[0002] By using artificial intelligence technology and deep learning algorithms, medical care and health testing for infants, the elderly and patients can be achieved. By monitoring heart rate, respiratory rate, snoring and other signals, potential heart problems, sleep breathing problems or other health abnormalities can be detected, and suggestions can be given on heart, sleep and physical health conditions, providing users with personalized health advice and intervention measures. For example, it can detect abnormal heart rates in time and remind users to seek medical treatment in time; it can identify users' sleep breathing problems and recommend appropriate treatment plans; it can also analyze users' sleep quality and provide suggestions for improving sleep. This health monitoring system based on artificial intelligence technology and deep learning algorithms has high practical value and can help improve the health management level of infants, the elderly and patients, prevent diseases and improve the quality of life.

[0003] At present, the main monitoring method is sensory monitoring, such as collecting data through electroencephalograms, patch-type electrocardiographs and other devices to detect physiological signals such as heart rate, breathing, and sleep. However, this collection method is cumbersome and interferes with sleep, limiting its promotion and application in daily life.

[0004] Therefore, technicians in this field are committed to developing a new physiological signal monitoring method and device to solve the above-mentioned defects in the prior art. Summary of the invention

[0005] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is how to solve the cumbersome collection method in the sensory monitoring method and the sleep interference caused to the detection object.

[0006] To achieve the above object, the present invention provides a multi-sensor fusion physiological signal monitoring method, comprising the following steps: Step 1: When the flexible sensor detects a pressure signal, the MCU is started to work; Step 2, the MCU synchronously wakes up the flexible sensor and the microphone array to collect signals, wherein the flexible sensor collects body motion signals, and the microphone array collects breathing sound signals, and the breathing sound signals include snoring signals; Step 3, first enhance and separate the breathing sound signal, and then assist in enhancing and separating the body motion signal; Step 4, performing snoring detection in the breathing sound signal to obtain a snoring result; performing heart rate monitoring, breathing monitoring and pulse monitoring in the body movement signal to obtain a heart rate result, a breathing result and a pulse result respectively; Step 5: Use a multi-level segmented deep clustering algorithm to perform sleep analysis to determine the sleep quality of each time period. The sleep quality is divided into five states: awake, light sleep N1, light sleep N2, deep sleep, and rapid eye movement.

[0007] Furthermore, the flexible sensor in step 1 adopts both a piezoelectric film sensor and a pressure sensor, wherein the pressure sensor is responsible for detecting the pressure signal, and the piezoelectric film sensor collects the body motion signal including breathing fluctuations and heart beats.

[0008] Furthermore, the flexible sensor in step 1 is laid flat on a detection plane and adapted to the detection plane, and a detection object is located directly above the detection plane, and the detection object will generate downward pressure on the flexible sensor under the action of gravity.

[0009] Furthermore, the step 3 includes the following sub-steps: Step 3.1, removing external noise in the breathing sound signal to obtain a clean breathing sound signal; Step 3.2, enhancing the body motion signal by using a Wiener filter or a low-pass filter method, filtering out high-frequency noise, retaining low-frequency signals related to body motion, and enhancing the body motion signal by using the snoring signal contained in the breathing sound signal; Step 3.3: extract the respiratory component, heart rate component and pulse component from the body motion signal.

[0010] Furthermore, the step 3.3 includes the following sub-steps: Step 3.3.1, extracting the respiratory component, first use a bandpass filter in the range of 0.1-0.4 Hz to filter out the components of the irrelevant signal, then obtain the amplitude envelope information from the signal through Hilbert transform, and then intercept a single frame of respiratory signal by finding the bipolar minimum point; Step 3.3.2, extract the heart rate component, first use a bandpass filter in the range of 0.5Hz-3Hz to preliminarily extract the cardiac impact signal, then calculate the first-order derivative or second-order derivative of the preliminarily extracted cardiac impact signal, then set a threshold, check the rising and falling parts connected above the set threshold, mark the local maximum value as the peak, set the minimum distance between peaks to avoid detecting repeated peaks, thereby removing pseudo peaks, and then take the largest point in the peak as the J point in the cardiac impact signal, the time interval between adjacent J points is called the JJ interval, and the inverse of the JJ interval is calculated to obtain the instantaneous heart rate; Step 3.3.3, extract the pulse component, use a bandpass filter in the range of 1 Hz-3 Hz to extract the pulse signal, and verify it with the heart rate component to evaluate the correctness of the extracted pulse component.

[0011] Furthermore, the step 5 includes the following sub-steps: Step 5.1, pre-processing the physiological signal obtained in step 4, dividing it into several segments according to time intervals, each segment has a time length of 30 seconds, and clustering each segment of data; Step 5.2, dividing the physiological signal obtained in step 4 into frames at different time scales, wherein the time scales include 30ms, 60ms, and 100ms, and one third of the frame time length is set as an overlapping frame, and multiplying each frame of audio by a window function so that the edge of the frame can transition smoothly, dividing the physiological signal into features of three different time scales, and using 1D-CNN to extract features of all types of the physiological signals and signals of all time scales; Step 5.3, performing K-means clustering on the features extracted in step 5.2 to generate cluster centers and cluster assignments; Step 5.4: The result of the cluster assignment is used as a pseudo label, which is the supervisory information for training the multi-level feature extraction layer + classifier, and the neural network training is performed. The weight of the network is updated by predicting the pseudo label. The loss function is the cross entropy loss. The classifier is composed of a fully connected layer, a dropout layer, and a softmax layer in sequence; Step 5.5: Alternately perform step 5.3 and step 5.4 until the training network weights and cluster assignments of the neural network converge, thereby determining the sleep quality of each time period.

[0012] Furthermore, the window function in step 5.2 is a Hamming window, a Hank window or a Blackman window.

[0013] Furthermore, the K-means clustering in step 5.3 includes the following sub-steps: Step 5.3.1, randomly select 5 initial cluster centers, namely awake, light sleep N1, light sleep N2, deep sleep and REM; Step 5.3.2: Assign each segment of data to the nearest cluster center. The distance calculation formula is: in, is a data point, It is Cluster centers, is a dimensional variable; Step 5.3.3, recalculate the center of each cluster. The new cluster center is the mean of all data points in the cluster: in, It is A set of data points that are clustered, is the number of data points in the set; Step 5.3.4: reallocate and update until the termination condition is met, where the cluster center no longer changes or the maximum number of iterations is reached.

[0014] The present invention also provides a multi-sensor fusion physiological signal monitoring device, the device comprising: A housing, and a hardware circuit board encapsulated in the housing, wherein an MCU, a microphone array, a display and a switch button are integrated on the hardware circuit board, wherein the MCU is a core component for collecting and processing physiological data; the microphone array is connected to the MCU, and includes a plurality of microphone units, a signal processor and an amplifier, and is responsible for collecting respiratory sound signals, wherein the respiratory sound signals include snoring signals; the display is connected to the MCU, and is used to display a heart rate curve, a heart rate value, a respiratory rate value, and a sleep status; A flexible sensor, which is also connected to the MCU, is laid flat on a detection plane and adapted to the detection plane. A detection object is located directly above the detection plane, and the detection object generates downward pressure on the flexible sensor under the action of gravity. The flexible sensor uses a piezoelectric film sensor and a pressure sensor at the same time. The pressure sensor is responsible for detecting the pressure signal, and the piezoelectric film sensor collects the body motion signal including breathing fluctuations and heart beats. A host computer, which is used to receive and process data from the MCU, and analyze and display the data; The cloud platform is a network server that also receives data from the MCU and stores the data in the cloud for further processing and management. The cloud platform also exchanges data with the host computer.

[0015] Furthermore, it includes the physiological signal monitoring method of multi-sensor fusion as described above.

[0016] The multi-sensor fusion physiological signal monitoring method and device provided by the present invention have at least the following technical effects: 1. The design concept of the technical solution provided by the present invention is to integrate the sensor into the mattress that is most imperceptible during sleep, and adopt non-contact heart rate, breathing, and snoring detection equipment. This method avoids the need for direct contact with the subject in traditional sleep monitoring methods, which may cause discomfort to the subject, improves the user experience, and is especially suitable for special groups such as infants, the elderly and patients; 2. The technical solution provided by the present invention improves the accuracy and efficiency of detection and analysis by utilizing artificial intelligence technology and deep learning algorithms. It also applies self-supervised algorithms in heart rate monitoring and unsupervised clustering algorithms in sleep analysis and evaluation, reducing the reliance on manually labeled data, saving labor costs and energy, and also reducing the possibility of human errors; 3. The technical solution provided by the present invention provides comprehensive monitoring, which can not only monitor heart rate and respiratory rate, but also monitor snoring and other signals, thereby more comprehensively reflecting the user's sleep state, which is conducive to early detection of heart problems, sleep breathing problems or other health abnormalities; 4. The technical solution provided by the present invention can improve the level of health management. By monitoring signals such as heart rate, respiratory rate, snoring, etc., it can detect potential health abnormalities and provide users with personalized health advice and intervention measures, thereby improving the user's health management level, preventing the occurrence of diseases, and improving the quality of life.

[0017] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flow chart of a monitoring method of a preferred embodiment of the present invention; Figure 2 It is a schematic diagram of identifying the J-J interval from the ballistocardiogram; Figure 3 This is a schematic diagram of the sleep stage criteria; Figure 4 It is a schematic diagram of the multi-level segmented deep clustering algorithm; Figure 5 is a schematic diagram of a monitoring device according to a preferred embodiment of the present invention; Figure 6 yes Figure 5 A schematic diagram of the connection relationship of the monitoring device of the illustrated embodiment. DETAILED DESCRIPTION

[0019] The following describes several preferred embodiments of the present invention with reference to the drawings in the specification, so that the technical content is clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned in the text.

[0020] Example 1 like Figure 1 As shown, the present invention provides a multi-sensor fusion physiological signal monitoring method, comprising the following steps: Step 1: When the flexible sensor detects a pressure signal, the MCU is started to work; Step 2, the MCU synchronously wakes up the flexible sensor and the microphone array to collect signals, wherein the flexible sensor collects body motion signals, and the microphone array collects breathing sound signals, and the breathing sound signals include snoring signals; Step 3: First enhance and separate the breathing sound signal, and then assist in enhancing and separating the body motion signal; Step 4: Perform snoring detection in the breathing sound signal to obtain the snoring result; perform heart rate monitoring, breathing monitoring and pulse monitoring in the body movement signal to obtain the heart rate result, breathing result and pulse result respectively; Step 5: Use a multi-level segmented deep clustering algorithm to perform sleep analysis and determine the sleep quality of each time period. The sleep quality is divided into five states: awake, light sleep N1, light sleep N2, deep sleep, and rapid eye movement.

[0021] Example 2 Based on Example 1, the flexible sensor in step 1 uses both a piezoelectric film sensor and a pressure sensor, wherein the pressure sensor is responsible for detecting pressure signals, and the piezoelectric film sensor collects body motion signals including breathing fluctuations and heart beats. The piezoelectric film sensor can sensitively respond to weak dynamic signals, such as breathing and heart rate signals.

[0022] In particular, the flexible sensor in step 1 is laid flat on the detection plane and adapted to the detection plane. The detection object is directly above the detection plane, and the detection object will exert downward pressure on the flexible sensor under the action of gravity. Specifically, the detection plane refers to the surface of the bed, and the flexible sensor is laid on the bed. When the human body is on the bed, the piezoelectric film sensor will accurately capture the signals of breathing and heartbeat, and calculate the breathing and heart rate values. However, due to the abnormal sensitivity of the piezoelectric film sensor, it sometimes leads to misjudgment of the person on the bed, because it may generate false alarms when someone passes by the bed or the ambient wind blows. In order to solve this problem, a pressure sensor is introduced as an auxiliary. The pressure sensor will only output a signal when the human body is actually lying on the bed, which effectively reduces the possibility of misjudgment and improves the anti-interference ability of the system.

[0023] Example 3 Based on Examples 1 and 2, step 3 includes the following sub-steps: Step 3.1, enhance and separate the breathing sound signal, remove external noise (such as road noise, TV noise, etc.) in the breathing sound signal, and obtain a clean breathing sound signal; Step 3.2: Enhance the body motion signal by using Wiener filtering or low-pass filtering to filter out high-frequency noise and retain low-frequency signals related to body movement (such as breathing and heartbeat signals). At the same time, use the snoring signal contained in the breathing signal to enhance the body motion signal, because when there is a snoring signal, the characteristics of the body motion signal may be more obvious and correlated with breathing; by calculating the coherence between the snoring signal and the body motion signal, it can be determined when snoring may affect the body motion signal, so that the signal can be weighted and enhanced in key events. When the snoring signal is strong, the weight of the body motion signal can be increased to enhance the overall identifiability of the signal. The body motion signal contains respiratory components, ballistocardiogram (BCG) components and noise signals.

[0024] Step 3.3, extracting the respiratory component, heart rate component and pulse component from the body motion signal, including the following sub-steps: Step 3.3.1, extract the respiratory component. This part of the signal reflects the movement of the chest and abdomen during breathing. This signal usually has an obvious periodic pattern, because human breathing is a periodic physiological process, generally composed of alternating inhalation and exhalation. The starting and ending points of the respiratory cycle can be determined according to the different stages of inhalation and exhalation. The respiratory component usually manifests as relatively low-frequency fluctuations, which are associated with the respiratory rate. According to general physiological standards, the respiratory rate of an adult is usually between 12 and 20 times per minute, which means that the normal respiratory rate is approximately between 0.2 and 0.33 Hz. First, use a bandpass filter in the range of 0.1-0.4 Hz to filter out the components of the irrelevant signal, then obtain the amplitude envelope information from the signal through the Hilbert transform, and then intercept the single-frame respiratory signal by finding the bipolar minimum point; Step 3.3.2, extract the heart rate component, which shows the blood impact of each heart beat. The BCG component usually shows a slightly higher frequency vibration than the respiratory component, with a frequency range of 0.5 to 10 Hz. In medicine, it is generally believed that the heart rate is normal between 40 and 150 beats per minute, that is, the frequency is 0.67 to 2.5 Hz. First, use a bandpass filter in the range of 0.5Hz-3Hz to preliminarily extract the heart impact signal, and then calculate the first-order derivative or second-order derivative of the preliminarily extracted heart impact signal to help identify the slope change points, which are usually candidates for peaks, and then set a threshold to identify possible peaks. Check the rising and falling parts connected above the set threshold, mark the local maximum as the peak, and compare the current sample point with the previous and next sample points to determine whether it is a peak; set the minimum distance between peaks to avoid detecting repeated peaks, thereby removing pseudo peaks, and then take the largest point in the peak as the J point in the BCG signal, and the time interval between adjacent J points is called the JJ interval.

[0025] The beating of the heart is caused by the depolarization and repolarization of myocardial cell tissue. In the normal pacemaker of the heart, the sinoatrial node, the impulse generated is transmitted to the atrium through the atrial conduction system, stimulating the atrial muscle cells, and then transmitting the impulse to the atrioventricular node. Next, the atrioventricular node stimulates the entire ventricular muscle through the left and right bundle branches to complete a heart beat. This process can be observed and recorded in electrophysiology through electrocardiographic signals. The ballistocardiogram (BCG) is a non-invasive, non-contact, and convenient means of monitoring cardiovascular function. The ballistocardiogram can be used in heart rate detection, heart rate variability monitoring, cardiac contractility, and cardiac output changes; BCG also has potential application value in heart disease, heart failure, etc. Such as Figure 2 As shown, a standard BCG heart beat mainly includes H, I, J, K, L, M, and N waves, among which the signal of the HL interval constitutes a wave group shaped like the letter "w", representing the process of heart contraction, and L, M, and N represent the process of heart relaxation.

[0026] The time interval between adjacent J peaks is called the "JJ interval". The inverse of the JJ interval is calculated to obtain the instantaneous heart rate based on the BCG signal. By locating the J peak in the BCG signal peak group, not only can the heart rate information be calculated, but also the foundation for the calculation of other sleep indicators and subsequent analysis work can be laid. Therefore, J peak positioning is the most important part of the research.

[0027] Step 3.3.3, extract the pulse component. The pulse component usually appears between 1 Hz and 3 Hz. Use a bandpass filter in the range of 1Hz-3Hz to extract the pulse signal. Since the frequency of the pulse (i.e., the pulse rate) is usually equal to or very close to the heart rate, each contraction of the heart pushes blood to the artery to form a pulse. The accuracy of the pulse component extraction can be verified with the heart rate, which can effectively evaluate the accuracy of the pulse component extraction. Ensuring measurement consistency under different conditions and situations can improve the reliability of the extraction method.

[0028] Example 4 Sleep staging is the premise for objectively evaluating sleep quality. From the sleep staging chart, we can get data such as the time and number of each sleep period, the total sleep duration, and the number of awakenings after falling asleep. Sleep is not a static process. In the AASM sleep staging criteria, for example, Figure 3 As shown in the figure, the entire sleep stage is divided into wakefulness (WAKE), non-rapid eye movement sleep (NREM), and rapid eye movement sleep (REM). The NREM stage can be divided into N1, N2, and N3 stages according to the depth of sleep. N1 and N2 can be combined into the light sleep stage, and N3 is the deep sleep stage.

[0029] Generally speaking, healthy adults will alternate between different sleep stages during a night's sleep, with about three to five sleep cycles, each lasting about ninety minutes. During the sleep of healthy adults, both NREM and REM stages may directly transition to a wakeful state, but the probability of transitioning from REM to wakefulness is much higher than the probability of transitioning from NREM to wakefulness. In the first half of sleep, the proportion of deep sleep N3 stage is higher, while in the second half, the proportion of N3 stage gradually decreases, while the proportion of REM stage gradually increases. During a night's sleep, N1 stage accounts for about 5% of sleep time, N2 accounts for about 50%, N3 accounts for about 20%, and REM accounts for about 25%. These proportions are not fixed and will vary depending on factors such as the health of the individual's autonomic nervous system and age. The transition of sleep stages can cause changes in breathing and snoring. When entering deep sleep, the degree of muscle relaxation increases, which may intensify snoring. In shallow sleep, snoring may decrease.

[0030] Sleep quality analysis is performed based on multiple indicators detected by the above process: the detection cycle is 30s, and the physiological signals used include breathing detection results, heart rate detection results, and snoring detection results within this time segment. Figure 3 As shown in the figure, sleep quality is divided into five states: awake, light sleep N1, light sleep N2, deep sleep and rapid eye movement.

[0031] Based on Example 1, 2 or 3, step 5 includes the following sub-steps: Figure 4 As shown: Step 5.1, pre-process the physiological signals obtained in step 4, divide them into several segments according to time intervals, each segment has a time length of 30 seconds, and cluster each segment of data; Step 5.2, the physiological signal obtained in step 4 is divided into frames at different time scales, including 30ms, 60ms, and 100ms. One third of the frame time length is set as an overlapping frame, and each frame of audio is multiplied by a window function so that the edge of the frame can transition smoothly. The physiological signal is divided into features of three different time scales, and 1D-CNN is used to extract features of all types of physiological signals and signals of all time scales; Step 5.3: Perform K-means clustering on the features extracted in step 5.2 to generate cluster centers and cluster assignments; Step 5.4: The result of cluster assignment is used as pseudo-label, which is the supervision information for training multi-level feature extraction layer + classifier, and neural network training is performed. The weight of the network is updated by predicting pseudo-label. The loss function is cross entropy loss. The classifier is composed of fully connected layer, dropout layer, and softmax layer. Step 5.5: Alternate steps 5.3 and 5.4 until the training network weights and cluster assignments of the neural network converge, thereby determining the sleep quality of each time period.

[0032] In particular, the window function in step 5.2 is a Hamming window, a Hank window or a Blackman window.

[0033] In particular, the K-means clustering in step 5.3 includes the following sub-steps: Step 5.3.1, randomly select 5 initial cluster centers, namely awake, light sleep N1, light sleep N2, deep sleep and REM; Step 5.3.2: Assign each segment of data to the nearest cluster center. The distance calculation formula is: in, is a data point, It is Cluster centers, is a dimensional variable; Step 5.3.3, recalculate the center of each cluster. The new cluster center is the mean of all data points in the cluster: in, It is A set of data points that are clustered, is the number of data points in the set; Step 5.3.4: Redistribute and update until the termination condition is met. The termination condition is that the cluster center no longer changes or the maximum number of iterations is reached.

[0034] Example 5 like Figure 5 As shown, a multi-sensor fusion physiological signal monitoring device is provided for an embodiment of the present invention, comprising: The housing and the hardware circuit board encapsulated in the housing have an MCU, a microphone array, a display and a switch button integrated on the hardware circuit board, wherein the MCU is a core component for collecting and processing physiological data; the microphone array is connected to the MCU and includes a plurality of microphone units, a signal processor and an amplifier, and is responsible for collecting respiratory sound signals, including snoring signals; the display is connected to the MCU and is used to display a heart rate curve, a heart rate value, a respiratory rate value and a sleep status; Flexible sensor: The flexible sensor is also connected to the MCU, laid flat on the detection plane and adapted to the detection plane. The detection object is located directly above the detection plane. The detection object will exert downward pressure on the flexible sensor under the action of gravity. The flexible sensor uses both piezoelectric film sensors and pressure sensors. The pressure sensor is responsible for detecting pressure signals, and the piezoelectric film sensor collects body motion signals including breathing and heartbeats. Host computer: The host computer is used to receive and process data from the MCU, and analyze and display the data; Cloud platform, the cloud platform is a network server that also receives data from the MCU and stores the data in the cloud for further processing and management. The cloud platform also exchanges data with the host computer.

[0035] In particular, it also includes any multi-sensor fusion physiological signal monitoring method as described above.

[0036] In particular, if Figure 6 As shown in the figure, the connection relationship between the host computer software, cloud platform, and microcontroller unit (MCU) is as follows: Host computer software: mainly for individual users, usually installed on PC, mobile device or WeChat applet, used to receive and process data from MCU, analyze and display the data. Host computer software exchanges data with MCU and cloud platform through various communication protocols. Users can view their heart rate data, generate reports, etc. through host computer software.

[0037] Cloud platform: The cloud platform is a network server, mainly for administrator users or enterprise users. It receives data from all MCUs and stores the data in the cloud for further processing and management. Its main functions are to achieve remote control, monitoring functions, and big data analysis.

[0038] MCU: MCU is the core component of the heart rate monitoring device, which is used to collect and process heart rate data, and then transmit the data to the host computer software or cloud platform through the wireless communication module. MCU uses communication technologies such as Bluetooth and Wi-Fi to connect with the host computer software and cloud platform.

[0039] The preferred specific embodiments of the present invention are described in detail above. It should be understood that ordinary technicians in the field can make many modifications and changes based on the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by technicians in the technical field based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A multi-sensor fusion physiological signal monitoring method, characterized in that: The method comprises the following steps: Step 1: When the flexible sensor detects a pressure signal, the MCU is started to work; Step 2, the MCU synchronously wakes up the flexible sensor and the microphone array to collect signals, wherein the flexible sensor collects body motion signals, and the microphone array collects breathing sound signals, and the breathing sound signals include snoring signals; Step 3, first enhance and separate the breathing sound signal, and then assist in enhancing and separating the body motion signal; Step 4, performing snoring detection in the breathing sound signal to obtain a snoring result; performing heart rate monitoring, breathing monitoring and pulse monitoring in the body movement signal to obtain a heart rate result, a breathing result and a pulse result respectively; Step 5: Use a multi-level segmented deep clustering algorithm to perform sleep analysis to determine the sleep quality of each time period. The sleep quality is divided into five states: awake, light sleep N1, light sleep N2, deep sleep, and rapid eye movement.

2. The multi-sensor fusion physiological signal monitoring method according to claim 1, characterized in that: The flexible sensor in step 1 uses both a piezoelectric film sensor and a pressure sensor, wherein the pressure sensor is responsible for detecting the pressure signal, and the piezoelectric film sensor collects the body motion signal including breathing fluctuations and heart beats.

3. The multi-sensor fusion physiological signal monitoring method according to claim 1, characterized in that: The flexible sensor in step 1 is laid flat on a detection plane and adapted to the detection plane. A detection object is located directly above the detection plane. The detection object generates downward pressure on the flexible sensor under the action of gravity.

4. The multi-sensor fusion physiological signal monitoring method according to claim 1, characterized in that: The step 3 includes the following sub-steps: Step 3.1, removing external noise in the breathing sound signal to obtain a clean breathing sound signal; Step 3.2, enhancing the body motion signal by using a Wiener filter or a low-pass filter method, filtering out high-frequency noise, retaining low-frequency signals related to body motion, and enhancing the body motion signal by using the snoring signal contained in the breathing sound signal; Step 3.3: extract the respiratory component, heart rate component and pulse component from the body motion signal.

5. The multi-sensor fusion physiological signal monitoring method according to claim 4, characterized in that: The step 3.3 includes the following sub-steps: Step 3.3.1, extracting the respiratory component, first use a bandpass filter in the range of 0.1-0.4 Hz to filter out the components of the irrelevant signal, then obtain the amplitude envelope information from the signal through Hilbert transform, and then intercept a single frame of respiratory signal by finding the bipolar minimum point; Step 3.3.2, extract the heart rate component, first use a bandpass filter in the range of 0.5Hz-3Hz to preliminarily extract the cardiac impact signal, then calculate the first-order derivative or second-order derivative of the preliminarily extracted cardiac impact signal, then set a threshold, check the rising and falling parts connected above the set threshold, mark the local maximum value as the peak, set the minimum distance between peaks to avoid detecting repeated peaks, thereby removing pseudo peaks, and then take the largest point in the peak as the J point in the cardiac impact signal, the time interval between adjacent J points is called the JJ interval, and the inverse of the JJ interval is calculated to obtain the instantaneous heart rate; Step 3.3.3, extract the pulse component, use a bandpass filter in the range of 1 Hz-3 Hz to extract the pulse signal, and verify it with the heart rate component to evaluate the correctness of the extracted pulse component.

6. The multi-sensor fusion physiological signal monitoring method according to claim 1, characterized in that: The step 5 comprises the following sub-steps: Step 5.1, pre-processing the physiological signal obtained in step 4, dividing it into several segments according to time intervals, each segment has a time length of 30 seconds, and clustering each segment of data; Step 5.2, dividing the physiological signal obtained in step 4 into frames at different time scales, wherein the time scales include 30ms, 60ms, and 100ms, and one third of the frame time length is set as an overlapping frame, and multiplying each frame of audio by a window function so that the edge of the frame can transition smoothly, dividing the physiological signal into features of three different time scales, and using 1D-CNN to extract features of all types of the physiological signals and signals of all time scales; Step 5.3, performing K-means clustering on the features extracted in step 5.2 to generate cluster centers and cluster assignments; Step 5.4, using the result of the cluster assignment as a pseudo label, which is the supervisory information for training the multi-level feature extraction layer + classifier, performing neural network training, updating the network weights by predicting the pseudo label, and the loss function is the cross entropy loss. The classifier is composed of a fully connected layer, a dropout layer, and a softmax layer in sequence; Step 5.5: Alternately perform step 5.3 and step 5.4 until the training network weights and cluster assignments of the neural network converge, thereby determining the sleep quality of each time period.

7. The multi-sensor fusion physiological signal monitoring method according to claim 6, characterized in that: The window function in step 5.2 is a Hamming window, a Hank window or a Blackman window.

8. The multi-sensor fusion physiological signal monitoring method according to claim 6, characterized in that: The K-means clustering in step 5.3 includes the following sub-steps: Step 5.3.1, randomly select 5 initial cluster centers, namely awake, light sleep N1, light sleep N2, deep sleep and REM; Step 5.3.2: Assign each segment of data to the nearest cluster center. The distance calculation formula is: in, is a data point, It is Cluster centers, is a dimensional variable; Step 5.3.3, recalculate the center of each cluster. The new cluster center is the mean of all data points in the cluster: in, It is A set of data points that are clustered, is the number of data points in the set; Step 5.3.4: reallocate and update until the termination condition is met, where the cluster center no longer changes or the maximum number of iterations is reached.

9. A multi-sensor fusion physiological signal monitoring device, characterized in that: The device comprises: A housing, and a hardware circuit board encapsulated in the housing, wherein an MCU, a microphone array, a display and a switch button are integrated on the hardware circuit board, wherein the MCU is a core component for collecting and processing physiological data; the microphone array is connected to the MCU, and includes a plurality of microphone units, a signal processor and an amplifier, and is responsible for collecting respiratory sound signals, wherein the respiratory sound signals include snoring signals; the display is connected to the MCU, and is used to display a heart rate curve, a heart rate value, a respiratory rate value, and a sleep status; A flexible sensor, which is also connected to the MCU, is laid flat on a detection plane and adapted to the detection plane. A detection object is located directly above the detection plane, and the detection object generates downward pressure on the flexible sensor under the action of gravity. The flexible sensor uses a piezoelectric film sensor and a pressure sensor at the same time. The pressure sensor is responsible for detecting the pressure signal, and the piezoelectric film sensor collects body motion signals including breathing fluctuations and heart beats. A host computer, which is used to receive and process data from the MCU, and analyze and display the data; The cloud platform is a network server that also receives data from the MCU and stores the data in the cloud for further processing and management. The cloud platform also exchanges data with the host computer.

10. The multi-sensor fusion physiological signal monitoring device according to claim 9, characterized in that: A physiological signal monitoring method using multi-sensor fusion comprising the method described in any one of claims 1 to 8.

Citation Information

Patent Citations

  • Device for monitoring sleep and screening obstructive sleep apnea syndrome

    CN104257368A

  • Method and device for processing BCG signal

    CN110742581A

  • Sleep monitoring sensing device

    CN112107299A

  • Sleep sensing belt and manufacturing method thereof

    CN116077020A

  • Information classification method, device and equipment and computer readable storage medium

    CN116910642A

Cited By

  • Respiration monitoring device and working method thereof

    CN119700074A

  • Method and system for non-contact polysomnography alternative through air pressure-based ballistocardiography and acoustic signal fusion

    KR102977162B1