Respiration monitoring analysis method and system based on multiple sensors

Abstract

The invention discloses a respiration monitoring analysis method and system based on multiple sensors, and relates to the technical field of respiration monitoring. The method comprises the following steps: collecting various signal data and collecting the signal data into a multi-dimensional signal data set; performing data preprocessing on the signal data; performing first-stage data fusion and second-stage feature fusion based on the preprocessed signal data; adopting a long-short-term memory network and a convolutional neural network to construct a respiratory anomaly detection model, and performing respiratory anomaly detection; and analyzing an anomaly detection result and carrying out visual display. Based on an advanced data fusion technology and an intelligent analysis algorithm, the defects of a traditional single-sensor monitoring method can be overcome, multi-dimensional and multi-level analysis of respiration signals is achieved, and the accuracy and real-time performance of respiration monitoring can be greatly improved; through multi-modal fusion, feature extraction and intelligent analysis of sensor data, more accurate and reliable technical support can be provided for early diagnosis and health monitoring of diseases.

Description

Technical Field

[0001] The present invention belongs to the technical field of respiratory monitoring, and in particular relates to a respiratory monitoring and analysis method and system based on multiple sensors. Background Art

[0002] Respiratory health is one of the most important indicators of human physiological status. Parameters such as respiratory rate, rhythm, and airflow directly reflect an individual's health. Respiratory system abnormalities, such as apnea and dyspnea, are often early signs of various diseases. Therefore, real-time and accurate monitoring of respiratory status has important clinical and health management value.

[0003] Traditional respiratory monitoring methods, such as chest expansion monitoring, flow sensors, or pressure sensors, can provide basic respiratory information, but they often suffer from problems such as poor signal accuracy, frequent interference, and single-dimensional analysis, making it difficult to achieve efficient and comprehensive respiratory monitoring in complex environments. With the development of sensor technology, especially the maturity of multi-sensor fusion technology, respiratory monitoring systems based on multiple sensors have gradually become a research hotspot. By combining multiple sensor technologies, respiratory signals can be collected and analyzed in multiple dimensions and channels, thereby effectively improving the accuracy and robustness of monitoring. This is especially important in application scenarios such as telemedicine, chronic disease management, and sleep monitoring, enabling real-time monitoring of patients' respiratory conditions and abnormal warnings, which has important practical significance.

[0004] However, despite the broad application prospects of multi-sensor technology in respiratory monitoring, practical applications still face challenges such as data fusion difficulties, noise interference, and significant individual variability. Furthermore, existing respiratory monitoring systems often rely on the output of a single sensor for data processing and analysis, lacking effective integration and optimization of data from multiple sensors, resulting in limited monitoring accuracy and system adaptability. Therefore, a respiratory monitoring system that integrates multi-sensor technology is urgently needed to enhance the intelligence of respiratory monitoring and enable it to meet more complex and diverse application scenarios. Summary of the Invention

[0005] The purpose of the present invention is to provide a multi-sensor based respiratory monitoring and analysis method and system, which can be achieved through the following technical solutions: In a first aspect, an embodiment of the present application provides a multi-sensor based respiratory monitoring and analysis method, comprising the following steps: Use multiple sensors to collect various signal data and aggregate them into multi-dimensional signal data sets; performing data preprocessing on the signal data in the multi-dimensional signal data set to obtain a preprocessed signal data set; Performing a first-stage data fusion based on the preprocessed signal data set to obtain an initial fused data set; then performing feature extraction based on the initial fused data set and performing a second-stage feature fusion on the extracted features to obtain a multidimensional feature vector; Based on the multidimensional feature vector, a long short-term memory network and a convolutional neural network are used to construct a breathing abnormality detection model, and breathing abnormality detection is performed according to the breathing abnormality detection model and an abnormality detection result is output; Analyze the anomaly detection results and visualize the analysis results; The use of multiple sensors to collect multiple signal data is specifically as follows: An air flow sensor is used to detect air flow conditions to obtain air flow signal data; A pressure sensor is used to monitor the pressure changes in the airway to obtain air pressure signal data; A temperature sensor is used to monitor the temperature changes of the respiratory tract gas to obtain air temperature signal data; A sound sensor is used to collect breathing sounds to obtain breathing signal data; Optical sensors are used to monitor chest and abdominal movements to obtain optical signal data; The ECG sensor is used to monitor the changes in heartbeat motion to obtain ECG signal data.

[0006] Preferably, the data preprocessing includes: Denoising: Apply filters to remove noise; Signal smoothing: using a smoothing algorithm to reduce instantaneous fluctuations in the preprocessed signal data set; Time synchronization: performing time alignment or interpolation processing on the data in the preprocessed signal data set so that all data are on the same time axis; Missing value handling: Fill missing values ​​in the data by interpolation or average filling; Spatiotemporal alignment: performing spatiotemporal alignment on the signal data collected by multiple sensors in the multi-dimensional signal data set. Preferably, the performing spatiotemporal alignment on the signal data collected by multiple sensors in the multi-dimensional signal data set includes: For signal data collected by multiple sensors, the target signal data of the same target collected by the same sensor are first associated, and then all target signal data are clustered according to the measurement characteristics. A preset number of target signal data are selected for association based on the distance to the cluster center, and other target signal data in the cluster are removed; Perform spatiotemporal alignment on target signal data within the same cluster; Perform identity determination on the target signal data after time-space alignment, and delete the target signal data that does not meet the identity from the cluster; The target signal data that have passed the identity judgment in the cluster are output and used for data fusion.

[0007] Preferably, the performing time-space alignment on target signal data within the same cluster includes time alignment and space alignment; The time alignment includes: when the number of target signal data in the cluster is less than a preset number, using a linear interpolation method to align the target signal data on each time slice; when the number of target signal data in the cluster is not less than the preset number, using a least square method to align the target signal data on each time slice; The spatial alignment uses the target signal data at a high data rate to infer the target signal data at a low data rate; The determining of the identity of the target signal data after the time-space alignment includes: Calculate the weighted average Euclidean distance of target signal data within the same cluster; The target signal data whose weighted average Euclidean distance is greater than the preset distance threshold is deleted from the cluster. Preferably, the first stage data fusion based on the preprocessed signal data set includes: Performing filtering and correction on the signal data of each sensor in the preprocessed signal data set to obtain corrected data; The weighted average method is used to calculate the inter-sensor fusion weights for data fusion between different sensors in multiple sensors, specifically: calculating a mean and a variance of the corrected data for each of the plurality of sensors; Calculate the inter-sensor fusion weights for fusion of different sensor data based on mean and variance; Performing secondary weighted fusion based on the corrected data and the inter-sensor fusion weights to obtain a final fusion result; The final fusion result is the initial fusion data set obtained after the first stage of data fusion.

[0008] Preferably, performing secondary weighted fusion based on the corrected data and the inter-sensor fusion weights to obtain a final fusion result includes: Performing a first fusion of the corrected data of the multiple sensors and the inter-sensor fusion weights to obtain a first fusion data group; Obtaining the mean and variance of the first fused data set; Calculating the secondary intra-group probability of all fused data in the first fused data group appearing in the first fused data group; The first fused data group and the secondary intra-group probability are fused for the second time to obtain a final fusion result of data fusion. Preferably, the extracting features based on the initial fused data set comprises: extracting temporal features from the time series of the initial fused dataset; Converting the time domain features into frequency domain features through Fourier transform; Combining the time domain features with the frequency domain features using a wavelet transform method to obtain time-frequency domain features; The time-frequency domain features are used as data features for the second stage feature fusion.

[0009] Preferably, the second stage feature fusion based on the extracted data features includes: Perform feature splicing based on the time-frequency domain features to form a splicing feature vector; Using correlation analysis to perform feature screening on the spliced ​​feature vector, and using principal component analysis to perform dimensionality reduction processing on the screened feature vector; The modality fusion technology is used to perform multimodal fusion on the feature vector after dimensionality reduction processing to form the multidimensional feature vector. Preferably, the breathing abnormality detection model constructed using a long short-term memory network and a convolutional neural network includes: Convolutional neural network part: Use the convolution kernel to perform convolution operation on the input multi-dimensional feature vector to extract local temporal features; Reduce the dimension of local temporal features through pooling operations; The ReLU activation function is used to increase the nonlinear representation capability of local temporal features; Long short-term memory network part: Inputting the local time series features into the long short-term memory network for time series modeling to capture the long-term dependency of the local time series features; Mapping the output of the long short-term memory network to the final prediction space through a fully connected layer; In the output layer, the activation function is selected according to the nature of the task. Specifically, if it is a classification task, the Softmax activation function is selected; if it is a regression task, the linear activation function is selected. The convolutional neural network part and the long short-term memory network part are combined to form the respiratory abnormality detection model.

[0010] In a second aspect, an embodiment of the present application provides a multi-sensor based respiratory monitoring and analysis system, which applies the respiratory monitoring and analysis method described above, including: Data acquisition module: uses multiple sensors to collect various signal data and aggregates them into multi-dimensional signal data sets; A data processing module: configured to perform data preprocessing on the signal data in the multi-dimensional signal data set to obtain a preprocessed signal data set; Data fusion module: performing a first-stage data fusion based on the preprocessed signal data set to obtain an initial fused data set; then performing feature extraction based on the initial fused data set and performing a second-stage feature fusion using the extracted features to obtain a multi-dimensional feature vector; Anomaly detection module: Based on the multidimensional feature vector, a long short-term memory network and a convolutional neural network are used to construct a respiratory anomaly detection model, and respiratory anomaly detection is performed according to the respiratory anomaly detection model and an anomaly detection result is output; Data visualization module: used to analyze the anomaly detection results and visualize the analysis results; The use of multiple sensors to collect multiple signal data is specifically as follows: An air flow sensor is used to detect air flow conditions to obtain air flow signal data; A pressure sensor is used to monitor the pressure changes in the airway to obtain air pressure signal data; A temperature sensor is used to monitor the temperature changes of the respiratory tract gas to obtain air temperature signal data; A sound sensor is used to collect breathing sounds to obtain breathing signal data; Optical sensors are used to monitor chest and abdominal movements to obtain optical signal data; The ECG sensor is used to monitor the changes in heartbeat motion to obtain ECG signal data.

[0011] The beneficial effects of the present invention are: relying on advanced data fusion technology and intelligent analysis algorithms, the present invention can make up for the shortcomings of traditional single sensor monitoring methods, not only realizing multi-dimensional and multi-level analysis of respiratory signals, but also improving adaptability to environmental changes and individual differences, greatly improving the accuracy and real-time performance of respiratory monitoring; through multimodal fusion, feature extraction and intelligent analysis of sensor data, it can provide more accurate and reliable technical support for early diagnosis of diseases and health monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] For better understanding and implementation, the technical solution of the present application is described in detail below with reference to the accompanying drawings.

[0013] Figure 1 A flowchart of a multi-sensor based respiratory monitoring and analysis method provided in an embodiment of the present application; Figure 2 A flowchart of the steps of the second stage feature fusion provided in an embodiment of the present application; Figure 3 A flowchart of the steps for constructing a respiratory abnormality detection model provided in an embodiment of the present application; Figure 4 A schematic structural diagram of a multi-sensor based respiratory monitoring and analysis system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0014] To further illustrate the technical means and effectiveness of the present invention in achieving its intended purpose, exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present application. Rather, they are merely examples of methods and systems consistent with certain aspects of the present application, as detailed in the appended claims.

[0015] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a," "an," "the," and "the" used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to any or all possible combinations of one or more of the associated listed items.

[0016] The following describes in detail the specific implementation methods, features and effects of the present invention in conjunction with the accompanying drawings and preferred embodiments.

[0017] Example 1 See also Figure 1 The present invention provides a multi-sensor based respiratory monitoring and analysis method, comprising the following steps: Use multiple sensors to collect various signal data and aggregate them into multi-dimensional signal data sets; performing data preprocessing on the signal data in the multi-dimensional signal data set to obtain a preprocessed signal data set; Performing a first-stage data fusion based on the preprocessed signal data set to obtain an initial fused data set; then performing feature extraction based on the initial fused data set and performing a second-stage feature fusion on the extracted features to obtain a multidimensional feature vector; Based on the multidimensional feature vector, a long short-term memory network and a convolutional neural network are used to construct a breathing abnormality detection model, and breathing abnormality detection is performed according to the breathing abnormality detection model and an abnormality detection result is output; Analyze the anomaly detection results and visualize the analysis results; The use of multiple sensors to collect multiple signal data is specifically as follows: An air flow sensor is used to detect air flow conditions to obtain air flow signal data; A pressure sensor is used to monitor the pressure changes in the airway to obtain air pressure signal data; A temperature sensor is used to monitor the temperature changes of the respiratory tract gas to obtain air temperature signal data; A sound sensor is used to collect breathing sounds to obtain breathing signal data; Optical sensors are used to monitor chest and abdominal movements to obtain optical signal data; The ECG sensor is used to monitor the changes in heartbeat motion to obtain ECG signal data.

[0018] Specifically, since the existing respiratory monitoring system often relies on the output results of a single sensor in data processing and analysis, it lacks effective integration and optimization of multiple sensor data, resulting in limited monitoring accuracy and system adaptability. Therefore, the present application provides a respiratory monitoring method system that integrates multi-sensor technology to improve the intelligence level of respiratory monitoring, so that it can meet more complex and diverse application scenarios, specifically including: first, using multiple sensors to collect multiple signal data and aggregate them into a multi-dimensional signal data set; then, data preprocessing is performed on the signal data in the multi-dimensional signal data set to obtain a preprocessed signal data set; then, based on the preprocessed signal data set, a first-stage data fusion is performed to obtain an initial fused data set; then, based on the initial fused data set, feature extraction is performed and the extracted data features are used to perform a second-stage feature fusion to obtain a multi-dimensional feature vector; then, based on the multi-dimensional feature vector, a respiratory anomaly detection model is constructed using a long short-term memory network and a convolutional neural network, and respiratory anomaly detection is performed according to the respiratory anomaly detection model and an anomaly detection result is output; finally, the anomaly detection result is analyzed and the analysis result is visualized. This application is based on a multi-sensor respiratory monitoring and analysis method, relying on advanced data fusion technology and intelligent analysis algorithms, which can make up for the shortcomings of traditional single-sensor monitoring methods. This technology can not only realize multi-dimensional and multi-level analysis of respiratory signals, but also improve the system's adaptability to environmental changes and individual differences, greatly improving the accuracy and real-time performance of respiratory monitoring. Through multimodal fusion, feature extraction and intelligent analysis of sensor data, it can provide more accurate and reliable technical support for early diagnosis of diseases, health monitoring and telemedicine.

[0019] It should be noted that this embodiment uses an airflow sensor to detect air flow conditions to obtain airflow signal data for analyzing respiratory frequency and tidal volume; uses a pressure sensor to monitor airway pressure changes to obtain air pressure signal data, thereby inferring respiratory patterns (such as inhalation and exhalation); uses a temperature sensor to monitor air temperature changes in the respiratory tract to obtain air temperature signal data to indirectly reflect respiratory status; uses a sound sensor to collect respiratory sounds to obtain respiratory signal data, and uses speech analysis and audio processing technology to identify respiratory sound characteristics (such as asthma sounds, snoring sounds, etc.); uses an optical sensor to monitor chest and abdominal movements to obtain optical signal data, thereby inferring respiratory rhythm; uses an electrocardiogram sensor to monitor changes in heartbeat movements to obtain electrocardiogram signal data, and combines electrocardiogram signals to evaluate the synchronization between breathing and heartbeat, especially in cases such as sleep apnea.

[0020] In one embodiment provided in this application, the data preprocessing includes: Denoising: Apply filters to remove noise; such as low-pass filtering, high-pass filtering, Kalman filtering, etc. to remove environmental noise and interference from the sensor itself to ensure the smoothness and stability of the data.

[0021] Signal smoothing: A smoothing algorithm is used to reduce instantaneous fluctuations in the preprocessed signal data set; making the signal more stable.

[0022] Time synchronization: Time alignment or interpolation processing is performed on the data in the preprocessed signal dataset so that all data are on the same time axis. Data from different sensors may have time delays or different sampling frequencies, so time synchronization is required.

[0023] Missing value handling: Fill missing values ​​in the data by interpolation or average filling; Spatiotemporal alignment: performing spatiotemporal alignment on the signal data collected by multiple sensors in the multi-dimensional signal data set.

[0024] Specifically, this embodiment ensures that the collected signal data is more stable and accurate through the above-mentioned data preprocessing, thereby providing support for subsequent data fusion.

[0025] In one embodiment provided in the present application, the performing spatiotemporal alignment on the signal data collected by multiple sensors in the multi-dimensional signal data set includes: For signal data collected by multiple sensors, the target signal data of the same target collected by the same sensor are first associated, and then all target signal data are clustered according to the measurement characteristics. A preset number of target signal data are selected for association based on the distance to the cluster center, and other target signal data in the cluster are removed; Perform spatiotemporal alignment on target signal data within the same cluster; Perform identity determination on the target signal data after time-space alignment, and delete the target signal data that does not meet the identity from the cluster; The target signal data that have passed the identity judgment in the cluster are output and used for data fusion.

[0026] In an embodiment provided by the present application, the performing of spatiotemporal alignment on target signal data within the same cluster includes time alignment and space alignment; The time alignment includes: when the number of target signal data in the cluster is less than a preset number, using a linear interpolation method to align the target signal data on each time slice; when the number of target signal data in the cluster is not less than the preset number, using a least square method to align the target signal data on each time slice; The spatial alignment uses the target signal data at a high data rate to infer the target signal data at a low data rate; The determining of the identity of the target signal data after the time-space alignment includes: Calculate the weighted average Euclidean distance of target signal data within the same cluster; The target signal data whose weighted average Euclidean distance is greater than the preset distance threshold is deleted from the cluster.

[0027] Specifically, this embodiment clusters target information through coarse clustering and fine clustering, reduces the computational complexity of spatiotemporal alignment and identity determination, effectively improves algorithm performance, and facilitates data fusion of multiple sensors for different targets.

[0028] In one embodiment provided in the present application, based on the preprocessed signal data set, a weighted average method is used to perform first-stage data fusion, including: Performing filtering and correction on the signal data of each sensor in the preprocessed signal data set to obtain corrected data; Calculate the inter-sensor fusion weights for data fusion between different sensors in multiple sensors, specifically: calculating a mean and a variance of the corrected data for each of the plurality of sensors; Calculate the inter-sensor fusion weights for fusion of different sensor data based on mean and variance; Performing secondary weighted fusion based on the corrected data and the inter-sensor fusion weights to obtain a final fusion result; The final fusion result is the initial fusion data set obtained after the first stage of data fusion.

[0029] In one embodiment provided in the present application, performing secondary weighted fusion based on the corrected data and the inter-sensor fusion weight to obtain a final fusion result includes: Performing a first fusion of the corrected data of the multiple sensors and the inter-sensor fusion weights to obtain a first fusion data group; Obtaining the mean and variance of the first fused data set; Calculating the secondary intra-group probability of all fused data in the first fused data group appearing in the first fused data group; The first fused data group and the secondary intra-group probability are fused for the second time to obtain a final fusion result of data fusion.

[0030] Specifically, this embodiment first filters and corrects the actual measurement data of each sensor to improve the credibility of the single-sensor measurement data. Secondly, the data fusion weights between sensors are obtained based on the corrected data to improve the rationality of the data fusion weight distribution. Finally, the corrected data and the data fusion weights between sensors obtained based on the corrected data are used to perform weighted fusion of multi-sensor data. The fusion result obtained by this method is closer to the actual situation and has higher credibility.

[0031] In one embodiment provided in the present application, feature extraction is performed based on the initial fused data set, including: extracting temporal features from the time series of the initial fused dataset; Converting the time domain features into frequency domain features through Fourier transform; Combining the time domain features with the frequency domain features using a wavelet transform method to obtain time-frequency domain features; The time-frequency domain features are used as data features for the second stage feature fusion.

[0032] like Figure 2 As shown, in one embodiment provided in the present application, the second stage feature fusion based on the extracted data features includes: Perform feature splicing based on the time-frequency domain features to form a splicing feature vector; Using correlation analysis to perform feature screening on the spliced ​​feature vector, and using principal component analysis to perform dimensionality reduction processing on the screened feature vector; The modality fusion technology is used to perform multimodal fusion on the feature vector after dimensionality reduction processing to form the multidimensional feature vector.

[0033] Specifically, in this application, the first stage of data fusion refers to sensor-level fusion, which directly fuses the raw data collected by the sensors, focusing on weighting or merging the raw data output by the sensors. The second stage of feature fusion is feature-level fusion, which refers to feature extraction of the fused data after sensor-level fusion. For example, statistical features (mean, standard deviation), frequency domain analysis (Fourier transform), and other means are used to further extract features meaningful to the target task. The present invention combines sensor-level fusion and feature-level fusion to optimize both the raw layer and the feature layer of the data, thereby improving the overall performance of feature fusion. The two-stage feature fusion improves the robustness, accuracy, and intelligence level of respiratory monitoring, and also provides data support for the subsequent construction of a respiratory anomaly detection model.

[0034] like Figure 3 As shown, in one embodiment provided in the present application, the breathing abnormality detection model constructed by using a long short-term memory network and a convolutional neural network includes: Convolutional neural network part: Use the convolution kernel to perform convolution operation on the input multi-dimensional feature vector to extract local temporal features; Reduce the dimension of local temporal features through pooling operations; The ReLU activation function is used to increase the nonlinear representation capability of local temporal features; Long short-term memory network part: Inputting the local time series features into the long short-term memory network for time series modeling to capture the long-term dependency of the local time series features; Mapping the output of the long short-term memory network to the final prediction space through a fully connected layer; In the output layer, the activation function is selected according to the nature of the task. Specifically, if it is a classification task, the Softmax activation function is selected; if it is a regression task, the linear activation function is selected. The convolutional neural network part and the long short-term memory network part are combined to form the respiratory abnormality detection model.

[0035] Specifically, this embodiment combines a long short-term memory (LSTM) network and a convolutional neural network (CNN) to construct a respiratory anomaly detection model. The main purpose is to capture temporal information through the LSTM network while simultaneously using the CNN to extract local signal features, fully combining the advantages of both. The CNN extracts local spatial features from the input multidimensional feature vector. For time series data, convolution operations help the model identify local patterns and regularities in the signal, such as respiratory variations across different frequency bands. Pooling operations (such as max pooling or average pooling) reduce the data dimension, reduce computational effort, and prevent overfitting. The ReLU (Rectified Linear Unit) activation function is used to enhance the model's nonlinear representation capabilities. The LSTM network captures long-term dependencies in time series data and is well suited for processing respiratory data with temporal characteristics. It can predict the current state based on past respiratory state information. The LSTM layer effectively memorizes and forgets information through gating mechanisms (forget gate, input gate, and output gate), thereby capturing long-term dependencies. Stacking multiple LSTM layers allows for deeper learning of the data's temporal characteristics.

[0036] After model training and evaluation, the respiratory anomaly detection model is put into practical application. The model is used for real-time respiratory anomaly detection. Specifically: Real-time data input: Obtain real-time signal data from respiratory monitoring equipment (multiple sensors) and pre-process these signal data into a format suitable for input model; Anomaly detection: The preprocessed data is fed into the model, local features are extracted through the convolutional neural network layer, and then into the long-short-term memory network layer to make predictions using time series information; Model output: The prediction result may be normal or abnormal. If it is abnormal, the type or degree of abnormality is output (such as mild abnormality, severe abnormality, etc.); Warning feedback: When abnormal breathing is detected, an alarm signal can be issued and medical staff can be notified.

[0037] This embodiment combines a long short-term memory network and a convolutional neural network to construct a respiratory anomaly detection model, which can effectively process the temporal characteristics and local features of respiratory data, provide high-precision anomaly detection, and help monitor and prevent respiratory anomalies in real time, thereby providing real-time and accurate health monitoring.

[0038] It should be noted that the "LSTM" in the above LSTM layer stands for "Long Short-Term Memory".

[0039] Example 2 See also Figure 4 The present invention provides a multi-sensor based respiratory monitoring and analysis system, comprising: Data acquisition module: uses multiple sensors to collect various signal data and aggregates them into multi-dimensional signal data sets; A data processing module: configured to perform data preprocessing on the signal data in the multi-dimensional signal data set to obtain a preprocessed signal data set; Data fusion module: performing a first-stage data fusion based on the preprocessed signal data set to obtain an initial fused data set; then performing feature extraction based on the initial fused data set and performing a second-stage feature fusion using the extracted features to obtain a multi-dimensional feature vector; Anomaly detection module: Based on the multidimensional feature vector, a long short-term memory network and a convolutional neural network are used to construct a respiratory anomaly detection model, and respiratory anomaly detection is performed according to the respiratory anomaly detection model and an anomaly detection result is output; Data visualization module: used to analyze the anomaly detection results and visualize the analysis results; The use of multiple sensors to collect multiple signal data is specifically as follows: An air flow sensor is used to detect air flow conditions to obtain air flow signal data; A pressure sensor is used to monitor the pressure changes in the airway to obtain air pressure signal data; A temperature sensor is used to monitor the temperature changes of the respiratory tract gas to obtain air temperature signal data; A sound sensor is used to collect breathing sounds to obtain breathing signal data; Optical sensors are used to monitor chest and abdominal movements to obtain optical signal data; The ECG sensor is used to monitor the changes in heartbeat motion to obtain ECG signal data.

[0040] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0041] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0042] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0043] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to equivalent embodiments using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A respiratory monitoring and analysis method based on multiple sensors, characterized in that: The steps include: Use multiple sensors to collect multiple signal data and aggregate them into multi-dimensional signal data sets; Performing data preprocessing on the signal data in the multi-dimensional signal data set to obtain a preprocessed signal data set; Performing a first-stage data fusion based on the preprocessed signal data set to obtain an initial fused data set; Then, feature extraction is performed based on the initial fusion data set, and second-stage feature fusion is performed using the extracted features to obtain a multi-dimensional feature vector; Based on the multidimensional feature vector, a long short-term memory network and a convolutional neural network are used to construct a breathing abnormality detection model, and breathing abnormality detection is performed according to the breathing abnormality detection model and an abnormality detection result is output; Analyze the anomaly detection results and visualize the analysis results; The use of multiple sensors to collect multiple signal data is specifically: An air flow sensor is used to detect air flow conditions to obtain air flow signal data; A pressure sensor is used to monitor the pressure change of the airway to obtain air pressure signal data; A temperature sensor is used to monitor the temperature changes of the gas in the respiratory tract to obtain air temperature signal data; A sound sensor is used to collect breathing sounds to obtain breathing signal data; Optical sensors are used to monitor chest and abdominal movements to obtain optical signal data; The ECG sensor is used to monitor the changes in heartbeat movement to obtain ECG signal data.

2. A multi-sensor based respiratory monitoring and analysis method according to claim 1, characterized in that: The data preprocessing includes: Denoising: Apply filters to remove noise; Signal smoothing: using a smoothing algorithm to reduce instantaneous fluctuations in the preprocessed signal data set; Time synchronization: performing time alignment or interpolation processing on the data in the preprocessed signal data set so that all data are on the same time axis; Missing value handling: fill missing values ​​in the data by interpolation or average filling; Time-space alignment: performing time-space alignment on the signal data collected by multiple sensors in the multi-dimensional signal data set.

3. The multi-sensor based respiratory monitoring and analysis method according to claim 1, characterized in that: The performing time-space alignment on the signal data collected by multiple sensors in the multi-dimensional signal data set includes: For signal data collected by multiple sensors, the target signal data of the same target collected by the same sensor are first associated, and then all target signal data are clustered according to the measurement characteristics, and a preset number of target signal data are selected for association according to the distance to the cluster center, and other target signal data in the cluster are removed; Performing spatial and temporal alignment on target signal data within the same cluster; Performing identity judgment on the target signal data after time-space alignment, and deleting the target signal data that does not meet the identity from the cluster; The target signal data that have passed the identity judgment in the cluster are output and used for data fusion.

4. The multi-sensor based respiratory monitoring and analysis method according to claim 3, characterized in that: The time-space alignment of the target signal data in the same cluster includes time alignment and space alignment; The time alignment includes: when the number of target signal data in the cluster is less than a preset number, using a linear interpolation method to align the target signal data on each time slice; when the number of target signal data in the cluster is not less than a preset number, using a least square method to align the target signal data on each time slice; The spatial alignment uses the target signal data with high data rate to calculate the target signal data with low data rate; The determining the identity of the target signal data after the time-space alignment includes: Calculate the weighted average Euclidean distance of target signal data within the same cluster; The target signal data whose weighted average Euclidean distance is greater than the preset distance threshold is deleted from the cluster.

5. The multi-sensor based respiratory monitoring and analysis method according to claim 1, characterized in that: The first stage data fusion based on the preprocessed signal data set includes: Performing filtering and correction on the signal data of each sensor in the preprocessed signal data set to obtain corrected data; The weighted average method is used to calculate the sensor fusion weights for data fusion between different sensors in multiple sensors, specifically: calculating a mean and a variance of the corrected data for each of the plurality of sensors; Calculate the inter-sensor fusion weights for fusion of different sensor data based on the mean and variance; Performing secondary weighted fusion based on the corrected data and the inter-sensor fusion weights to obtain a final fusion result; The final fusion result is the initial fusion data set obtained after the first stage data fusion.

6. The multi-sensor based respiratory monitoring and analysis method according to claim 5, characterized in that: The performing secondary weighted fusion based on the corrected data and the inter-sensor fusion weights to obtain a final fusion result includes: Performing a first fusion of the corrected data of the multiple sensors and the inter-sensor fusion weights to obtain a first fusion data group; Obtaining the mean and variance of the first fused data group; Calculating the secondary intra-group probability of all fused data in the first fused data group appearing in the first fused data group; The first fused data group and the secondary intra-group probability are fused for a second time to obtain a final fusion result of data fusion.

7. The multi-sensor based respiratory monitoring and analysis method according to claim 1, characterized in that: The extracting features based on the initial fusion data set comprises: Extracting time domain features from the time series of the initial fused data set; Convert the time domain features into frequency domain features through Fourier transform; Combining the time domain features with the frequency domain features using a wavelet transform method to obtain time-frequency domain features; The time-frequency domain features are used as data features for the second stage feature fusion.

8. The multi-sensor based respiratory monitoring and analysis method according to claim 7, characterized in that: The step of extracting features based on the initial fusion data set and performing second-stage feature fusion using the extracted features includes: Perform feature splicing based on the time-frequency domain features to form a spliced ​​feature vector; Using correlation analysis to perform feature screening on the concatenated feature vector, and using principal component analysis to perform dimensionality reduction processing on the screened feature vector; The modality fusion technology is used to perform multi-modal fusion on the feature vector after dimensionality reduction processing to form the multi-dimensional feature vector.

9. The multi-sensor based respiratory monitoring and analysis method according to claim 1, characterized in that: The breathing abnormality detection model is constructed by using a long short-term memory network and a convolutional neural network, including: Convolutional neural network part: Use the convolution kernel to perform convolution operation on the input multi-dimensional feature vector to extract local time series features; Reduce the dimension of local temporal features through pooling operations; The ReLU activation function is used to increase the nonlinear representation capability of local time series features; Long short-term memory network part: Inputting the local time series features into the long short-term memory network for time series modeling to capture the long-term dependency of the local time series features; Mapping the output of the long short-term memory network to the final prediction space through a fully connected layer; In the output layer, the activation function is selected according to the nature of the task. Specifically, if it is a classification task, the Softmax activation function is selected; if it is a regression task, the linear activation function is selected; The convolutional neural network part and the long short-term memory network part are combined to form the breathing abnormality detection model.

10. A multi-sensor based respiratory monitoring and analysis system, using the respiratory monitoring and analysis method according to any one of claims 1 to 9, characterized in that: include: Data acquisition module: uses multiple sensors to collect various signal data and aggregates them into multi-dimensional signal data sets; A data processing module: used for performing data preprocessing on the signal data in the multi-dimensional signal data set to obtain a preprocessed signal data set; Data fusion module: performing first-stage data fusion based on the preprocessed signal data set to obtain an initial fused data set; Then, feature extraction is performed based on the initial fusion data set, and second-stage feature fusion is performed using the extracted features to obtain a multi-dimensional feature vector; Anomaly detection module: Based on the multidimensional feature vector, a long short-term memory network and a convolutional neural network are used to construct a breathing abnormality detection model, and breathing abnormality detection is performed according to the breathing abnormality detection model and an abnormality detection result is output; Data visualization module: used to analyze the anomaly detection results and visualize the analysis results; The use of multiple sensors to collect multiple signal data is specifically: An air flow sensor is used to detect air flow conditions to obtain air flow signal data; A pressure sensor is used to monitor the pressure change of the airway to obtain air pressure signal data; A temperature sensor is used to monitor the temperature changes of the gas in the respiratory tract to obtain air temperature signal data; A sound sensor is used to collect breathing sounds to obtain breathing signal data; Optical sensors are used to monitor chest and abdominal movements to obtain optical signal data; The ECG sensor is used to monitor the changes in heartbeat movement to obtain ECG signal data.

Images