Multi-modal human respiration prediction method and system

Abstract

The invention relates to a multi-mode human body respiration prediction method and system. The method comprises the steps that a difference value time sequence T is obtained through a visible light video and a thermal infrared video, and then the respiration frequency f1 of a target human body is obtained; collecting an electrophysiological signal containing a surface diaphragm electromyogram from the thoracic skin surface by means of an electromyographic signal collection system, and processing the electrophysiological signal to obtain a separated diaphragm electromyographic signal E; the heart rate f2 of the target human body is obtained through an electrocardiosignal collecting system; acquiring an environmental parameter Y of the target human body by using an environmental parameter acquisition system; the parameters obtained in the step S1-4 are processed, and a breathing state evaluation model is built; and S4, optimizing and constructing a BP neural network, and predicting the respiratory movement state of the human body by matching the sample data obtained in the step S5 with the BP neural network.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent care, and particularly relates to a multi-modal human respiration prediction method and system thereof. Background Art

[0002] The monitoring and prediction of the human respiratory state are of great significance for the early detection, early warning, and health management of respiratory diseases. With the aggravation of air pollution, the high incidence of chronic respiratory diseases, and the continuous emergence of new respiratory diseases, how to efficiently and accurately monitor the respiratory condition of the human body has become a research hotspot in the cross-field of medicine and artificial intelligence; currently, traditional respiratory monitoring methods mainly rely on contact sensors, such as pulmonary function detectors, airflow sensors, chest and abdomen belts, etc. Although these methods can provide certain physiological parameter information, they have limitations such as discomfort in wearing, dependence on the active cooperation of patients, and inconvenience in long-term monitoring; in addition, some non-contact methods based on image analysis or radar sensing, although solving the problem of discomfort in wearing, are greatly interfered in complex environments and it is difficult to accurately identify minute respiratory changes.

[0003] In predicting the human respiratory state, existing methods often rely on single-modal data, such as single airflow changes, heart rate variability, or imaging data, resulting in limited reliability and adaptability of their prediction results; for example, the prediction method based on heart rate variability is easily affected by emotional changes and exercise states, while the imaging method that simply relies on chest movement detection may be interfered by factors such as environmental light and occlusion.

[0004] In view of the above, the present application proposes a multi-modal human respiration prediction method and system thereof; this method improves the accuracy of respiratory abnormality detection through multi-modal data information; it can not only identify the current respiratory state of the user, but also establish an individualized respiration prediction model based on long-term monitoring data to achieve early warning of abnormal trends; through intelligent algorithm analysis, it can also determine whether there is an abnormality in the respiratory state and provide suggestions for doctor intervention, so as to intervene as early as possible to prevent the occurrence of serious respiratory diseases.

[0005] In view of the above, we provide a multi-modal human respiration prediction method and system to solve the above problems. Summary of the Invention

[0006] In view of the above situation, the present invention provides a multi-modal human respiration prediction method and system, including,

[0007] S1. Obtain the respiration frequency f1: Use the visible light video and the thermal infrared video to obtain the difference time series T, and then obtain the respiration frequency f1 of the target human body;

[0008] S2. Obtain the diaphragmatic electromyogram signal E: With the help of an electromyogram signal acquisition system, collect the electrophysiological signal containing the surface diaphragmatic electromyogram from the surface of the thoracic skin, and process it to obtain the separated diaphragmatic electromyogram signal E;

[0009] S3. Obtain the heart rate f2: Use an electrocardiogram signal acquisition system to obtain the heart rate f2 of the target human body;

[0010] S4. Obtain the environmental parameter Y: Use an environmental parameter acquisition system to obtain the environmental parameter Y where the target human body is located;

[0011] S5. Build a respiratory state assessment model: Preprocess the parameters obtained in steps S1 - 4, convert the signals into time series data, extract the respiratory sound wave features and electrocardiac activity features from the time series data, capture the time-dependent changes of the respiratory sound wave according to the extracted respiratory sound wave features, detect and classify the respiratory pattern, and then evaluate the detection and classification results of the respiratory pattern and heart rate variability to obtain sample data;

[0012] S6. Predict exercise respiration: Optimize and build a BP neural network, and cooperate with the sample data obtained in step S5 to realize the prediction of the respiratory movement state of the human body.

[0013] Preferably, the steps of obtaining using the visible light video and the thermal infrared video in step S1 are as follows:

[0014] S1a: Detect key points of the visible light video image to obtain the first facial key point set of the target human body;

[0015] S1b: According to the first facial key point set and a preset registration matrix, obtain the second facial key point set of the target human body in the thermal infrared video image;

[0016] S1c: According to the thermal infrared video image and the second facial key point set, obtain the first thermal infrared data and the second thermal infrared data;

[0017] S1d: According to the difference between the first thermal infrared data and the second thermal infrared data, obtain the difference time series T;

[0018] S1e: According to the difference time series T, obtain the respiratory rate of the target human body.

[0019] Preferably, the steps of obtaining using the multivariate empirical mode decomposition in step S2 are as follows:

[0020] S2a: Collect the surface diaphragmatic electromyogram signal and use frequency domain analysis method for denoising;

[0021] S2b: Perform discrete Fourier transform on the original surface diaphragmatic electromyogram signal to obtain the surface diaphragmatic electromyogram signal after filtering out low-frequency noise;

[0022] S2c: Execute the respiratory-related signal extraction algorithm to obtain the extracted respiratory-related surface diaphragmatic electromyogram signal, and derive the diaphragmatic electromyogram signal E after filtering out interference and noise from the preprocessed original signal.

[0023] Preferably, the environmental parameter Y includes environmental temperature, air humidity, atmospheric pressure value, and environmental noise value.

[0024] Preferably, the method for preprocessing the parameters obtained in S1-4 in S5 is: perform normalization and noise reduction processing, and convert the signal after normalization and noise reduction processing into time series data T x_in 。

[0025] Preferably, the method for extracting respiratory acoustic wave features and electrocardiogram activity features from the time series data in S5 is: use a pre-trained deep convolutional neural network to extract respiratory acoustic wave features from the surface diaphragmatic electromyogram data E and the obtained respiratory frequency f1;

[0026] Use a pre-trained deep convolutional neural network to extract electrocardiogram activity features from the electrocardiogram time series data, including: P wave, QRS wave, and T wave;

[0027] Use a pre-trained deep convolutional neural network to extract environmental parameter features from the environmental parameter data Y.

[0028] Preferably, the method for detecting and classifying the respiratory pattern in S5 is: pass the deep convolutional neural network through multiple convolutional and max-pooling operations, and at the same time use activation functions and regularization techniques. Use a fully connected layer at the end of the network to integrate all features, and perform multi-class classification through the softmax layer.

[0029] Preferably, the method for realizing the detection and classification results of the respiratory pattern and heart rate variability in S5 is: analyze the respiratory and heart rate synchrony by calculating the Pearson correlation coefficient. If the Pearson correlation coefficient between the respiratory acoustic wave signal and the electrocardiogram signal exceeds 0.8, and the change trends of the respiratory frequency and heart rate are consistent, then the user's respiration and heart rate are within the normal range, that is, the respiration and heart rate are in normal synchrony; if the Pearson correlation coefficient between the respiratory acoustic wave signal and the electrocardiogram signal is lower than 0.5, and the change trends of the respiratory frequency and heart rate are inconsistent, then the respiration and heart rate are in abnormal synchrony; if the Pearson correlation coefficient between the respiratory acoustic wave signal and the electrocardiogram signal is between 0.5 and 0.8, and the change trends of the respiratory frequency and heart rate are 50% to 80% consistent, then the respiration and heart rate are in moderate asynchrony; according to the analysis results of the respiratory and heart rate synchrony and the detection and classification results of the respiratory pattern and heart rate variability by the abnormal feature capture module, perform a comprehensive health score to obtain the sample data;

[0030] Among them, the breathing patterns include: normal breathing pattern, apnea, and shallow breathing or tachypnea.

[0031] Preferably, the specific method for predicting the respiratory movement state of the human body in S6 is as follows:

[0032] S6a: Construct a BP neural network and optimize the BP neural network by integrating the simulated annealing genetic algorithm;

[0033] S6b: Optimize the weighted error of the BP neural network;

[0034] S6c: Based on the sample data obtained in the evaluation in S5, obtain respiratory movement data;

[0035] S6d: Use the processed respiratory movement data to train the BP neural network with optimized weighted error to obtain a strong regressor;

[0036] S6e: Use the strong regressor to predict the respiratory movement state of the human body.

[0037] A multi-modal human respiratory prediction system includes a data acquisition module, a preprocessing module, a feature extraction module, an abnormal feature capture module, a comprehensive health scoring module, an intervention module, and a prediction module;

[0038] The data acquisition module is used to collect the user's respiratory rate, diaphragmatic electromyogram signal, electrocardiogram signal, and environmental parameters, and wirelessly transmit the collected signals to the preprocessing module;

[0039] The preprocessing module is used to receive the monitoring signals transmitted by the wearable monitoring device, and preprocess the received respiratory sound wave signals, electrocardiogram signals, and environmental parameter signals respectively, including normalization and noise reduction processing, and convert the signals after normalization and noise reduction processing into time series data;

[0040] The feature extraction module is used to extract respiratory sound wave features and electrocardiac activity features from the time series data;

[0041] The abnormal feature capture module is used to capture the time-dependent changes of the respiratory sound wave according to the extracted respiratory sound wave features, detect and classify the breathing patterns; and at the same time, it is used to capture the time-dependent changes of the electrocardiac activity according to the extracted electrocardiac activity features, detect and classify the heart rate variability;

[0042] The comprehensive health scoring module is used to perform a comprehensive health score according to the detection and classification results of the breathing pattern and heart rate variability by the abnormal feature capture module;

[0043] The intervention module is used to generate health intervention suggestions based on the comprehensive health score obtained by the comprehensive health score module and display them to the user.

[0044] The prediction module is used to establish a neural network, and after processing, optimizing, and training the respiratory movement data obtained from the comprehensive health score module, predict the respiratory movement state.

[0045] The beneficial effects of the above technical solutions are as follows:

[0046] (1) By using multi-modal data information such as respiratory sound waves, ECG, diaphragmatic electromyogram, and environmental data, the accuracy of respiratory abnormality detection is improved.

[0047] (2) Through abnormal detection and early warning, it can help users with health management and early intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the overall structure of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0049] Regarding the foregoing and other technical contents, features, and effects of the present invention, they can be clearly presented in the following detailed description of the embodiments in conjunction with the accompanying drawings. The structural contents mentioned in the following embodiments are all with reference to the drawings of the specification. Figure 1 The present application proposes a multi-modal human respiration prediction method and system, which more accurately predicts respiratory activities based on multi-modal data fusion, improving the accuracy and reliability of prediction. Specifically, it is as follows:

[0050] First, the multi-modal human respiration prediction method includes the following steps:

[0051] S1. Obtain the respiratory frequency f1:

[0052] S1. Obtain the respiratory frequency f1:

[0053] Use the visible light video and the thermal infrared video to obtain the difference time series T and then obtain the respiratory frequency f (processed data) of the target human body. The specific implementation steps are as follows:

[0054] S1a: Perform key point detection on the visible light video image to obtain the first facial key point set of the target human body. It should be noted that the first facial key point set includes several first facial key points of the target human body in the visible light video image. The visible light video refers to the video including the target human body captured by the visible light camera, which includes image information within the spectral range visible to the human eye. The target human body refers to the person who needs to be monitored for respiration and is also the object of key point detection. The first facial key point set refers to the set of the first facial key points on the face of the target human body detected in the visible light video, usually including the coordinate information of important positions such as eyes, nose, and mouth.

[0055] S1b: Obtain a second set of facial key points of the target human body in the thermal infrared video image according to the first set of facial key points and the preset registration matrix; the specific implementation process is as follows: Obtain a second set of facial key points of the target human body in the thermal infrared video image according to the first set of facial key points and the preset registration matrix. Among them, both the visible light video image and the thermal infrared video image include images of the target human body. The preset registration matrix is used to represent the mapping relationship between pixel points in the thermal infrared video image and the visible light video image. The second set of facial key points includes a number of second facial key points of the target human body in the thermal infrared video image, and the first facial key points and the second facial key points are in one-to-one correspondence;

[0056] S1c: Obtain first thermal infrared data and second thermal infrared data according to the thermal infrared video image and the second set of facial key points; the specific implementation method is as follows: Control the thermal infrared camera and the visible light camera to take pictures of the registration object through the preset registration matrix, and obtain a thermal infrared registration image and a visible light registration image. Among them, the registration object includes a preset number of black metal heating sheets, and the black metal heating sheets have a preset temperature. Each black metal heating sheet is evenly distributed in the thermal infrared registration image and the visible light registration image; it should be noted that in this step, the thermal infrared camera and the visible light camera take pictures of the same registration object, aiming to obtain the corresponding relationship between the visible light and the thermal infrared images through the black metal heating sheets on the registration object, so as to establish the registration relationship between the thermal infrared image and the visible light image;

[0057] In view of the above, obtain first thermal infrared data and second thermal infrared data according to the thermal infrared video image and the second set of facial key points. Among them, the second set of facial key points includes a number of nose key points and a number of third facial key points other than the nose key points. The first thermal infrared data is obtained based on the nose key points, and the second thermal infrared data is obtained based on the third facial key points; specifically, use the nose key point information in the second set of facial key points to determine the nose area in the thermal infrared video. These key points may include position information such as the tip of the nose and the bridge of the nose. According to the determined nose area, extract the thermal infrared data of this area in the thermal infrared video to obtain the first thermal infrared data; use the third facial key points other than the nose key points in the second set of facial key points to obtain the thermal infrared data outside the nose area, which is recorded as the second thermal infrared data;

[0058] S1d: Obtain the difference time series T according to the difference between the first thermal infrared data and the second thermal infrared data x_in (Processed data);

[0059] In the process of value taking, the specific method includes: Sort the first thermal infrared data corresponding to each frame of the thermal infrared image, and obtain the average value of the thermal infrared data with the top 10% of the thermal infrared values to obtain the first mean sequence;

[0060] It should be noted that for each frame of thermal infrared image, the corresponding first thermal infrared data is sorted, that is, the thermal infrared data of each pixel in the nasal region is sorted from large to small, and then the top 10% with the highest values is selected. Selecting the top 10% of the thermal infrared data is to ensure that the data with higher temperatures is obtained. These data usually have higher sensitivity and accuracy for respiratory monitoring; further, during the breathing process, the temperature change in the nasal region is relatively significant. Therefore, selecting the high-temperature data in this region can better capture the characteristics of the respiratory signal. Calculate the average value of the top 10% of the thermal infrared data to obtain the first mean sequence. The first mean sequence can reflect the average temperature level of the part with higher temperature in each frame of thermal infrared image; moreover, selecting the top 10% of the thermal infrared data can filter out some low-temperature data, thereby improving the signal-to-noise ratio of the signal. By selecting the high-temperature data, the interference of other irrelevant signals such as environmental temperature changes on the respiratory signal can be reduced, thereby improving the accuracy and reliability of respiratory monitoring.

[0061] Then, the second thermal infrared data of the facial key points outside the nasal region in the second facial key point set in each frame of thermal infrared image is averaged to obtain the second mean sequence. The second mean sequence can provide a reference benchmark corresponding to the first thermal infrared data for comparative analysis of the temperature change differences; obtain the difference between the first mean sequence and the second mean sequence to obtain the initial difference sequence.

[0062] By subtracting the first mean sequence from the second mean sequence, an initial difference sequence is obtained, and the initial difference sequence is segmented according to a preset time window to obtain multiple time windows, and then the differences within each time window are combined to obtain a difference time series. By processing this difference time series, the temperature change trend within different time windows can be reflected, which helps to more accurately analyze the respiratory rate and respiratory pattern.

[0063] S1e: Obtain the respiratory rate of the target human body according to the difference time series T; its specific manifestation is: perform filtering processing on the difference time series to obtain a filtered difference time series; first, perform filtering processing on the difference time series; remove the outliers in the filtered difference time series according to a preset outlier removal method to obtain a target respiratory sequence; calculate the average value of the filtered difference sequence to obtain a standard value; replace the outliers in the filtered difference sequence according to the standard value to obtain a target respiratory sequence. According to the interval between the peaks in the target respiratory sequence, obtain the respiratory rate f1 of the target human body.

[0064] It should be noted that the purpose of the filtering process is to remove possible noise or high-frequency oscillations to obtain a smooth difference time series; common filtering methods include moving average filtering and median filtering;

[0065] S2. Obtain the diaphragmatic electromyogram signal E:

[0066] With the aid of an electromyogram signal acquisition system, collect the electrophysiological signal containing the surface diaphragmatic electromyogram from the surface of the thoracic skin, and process it to obtain the separated diaphragmatic electromyogram signal E. The specific implementation method is as follows:

[0067] S2a: Collect the surface diaphragmatic electromyogram signal and perform denoising using frequency domain analysis. The analysis example is as follows: Use an array electrode sheet of a rows × b columns to collect multi-channel surface diaphragmatic electromyogram signals C = [c1(n), c2(n), …, c i (n), …, c m (n)] T , where c i represents the surface diaphragmatic electromyogram signal containing noise and interference in the i-th channel, m represents the total number of channels, and m = a × b.

[0068] S2b: Perform discrete Fourier transform on the original surface diaphragmatic electromyogram signal C = [c1(n), c2(n), …, c i (n), …, c m (n)] T to obtain the surface diaphragmatic electromyogram signal Y = [y1, y2, …, y i , …, y m T , where y i represents the surface diaphragmatic electromyogram signal in the i-th channel after filtering out low-frequency noise, and T represents transpose;

[0069] S2c: Execute the respiration-related signal extraction algorithm to obtain the extracted respiration-related surface diaphragmatic electromyogram signal X = [x1(n), x2(n), …, x i (n), …, x m (n)] T , where x i represents the i-th respiration source signal; this signal is the diaphragmatic electromyogram signal E (the data after filtering) obtained by filtering out interference and noise from the preprocessed original signal.

[0070] S3. Obtain the heart rate f2: Use an electrocardiogram signal acquisition system to obtain the heart rate f2 (the data after filtering) of the target human body; this step is implemented by existing equipment (such as wearable devices, etc.).

[0071] S4. Obtain the environmental parameter Y: Use an environmental parameter acquisition system to obtain the environmental parameter Y of the target human body; where the environmental parameter Y includes: environmental temperature, air humidity, atmospheric pressure value, and environmental noise value.

[0072] S5. Build a respiratory state assessment model:

[0073] Establish an overall respiratory state assessment model for multi-modal data fusion. The specific operation steps are as follows:

[0074] Data collection and preprocessing:

[0075] Preprocess the received respiratory rate signal, surface diaphragmatic electromyogram signal E, electrocardiogram signal, and environmental parameter Y, including normalization and noise reduction processing, and convert the signals after normalization and noise reduction processing into time series data T x_in , to optimize the subsequent analysis process.

[0076] Feature extraction:

[0077] Use a pre-trained deep convolutional neural network to extract respiratory acoustic wave features from the surface diaphragmatic electromyogram data E and the obtained respiratory rate f1; use a pre-trained deep convolutional neural network to extract electrocardiac activity features from the electrocardiogram time series data, including: P wave, QRS wave, and T wave; use a pre-trained deep convolutional neural network to extract environmental parameter features from the environmental parameter data Y, thereby extracting respiratory acoustic wave features and electrocardiac activity features from the time series data;

[0078] The deep convolutional neural network includes multiple layers of convolution and max-pooling operations, and at the same time uses activation functions and regularization techniques. A fully connected layer is used at the end of the network to integrate all features, and multi-class classification is performed through the softmax layer.

[0079] Abnormal feature capture:

[0080] Subsequently, analyze the respiratory and heart rate synchrony by calculating the Pearson correlation coefficient. If the Pearson correlation coefficient between the respiratory acoustic wave signal and the electrocardiogram signal exceeds 0.8, and the change trends of the respiratory rate and heart rate are consistent, then the user's respiration and heart rate are within the normal range, that is, the respiration and heart rate are in normal synchrony; if the Pearson correlation coefficient between the respiratory acoustic wave signal and the electrocardiogram signal is below 0.5, and the change trends of the respiratory rate and heart rate are inconsistent, then the respiration and heart rate are in abnormal synchrony; if the Pearson correlation coefficient between the respiratory acoustic wave signal and the electrocardiogram signal is between 0.5 and 0.8, and the change trends of the respiratory rate and heart rate are 50% to 80% consistent, then the respiration and heart rate are in moderate asynchrony;

[0081] Normal breathing pattern, apnea, and shallow or rapid breathing;

[0082] Normal breathing pattern: The respiratory rate is within 12 - 20 breaths per minute;

[0083] Apnea: More than 5 apneas occur per hour, and each apnea lasts for more than 10 seconds;

[0084] Shallow breathing or tachypnea: A respiratory rate exceeding 25 breaths per minute represents tachypnea;

[0085] A respiratory rate below 10 breaths per minute represents shallow breathing;

[0086] The overall assessment model of respiratory status for multimodal data fusion is divided into three cases: normal heart rate variability, bradycardia or tachycardia, and arrhythmia;

[0087] Normal heart rate variability: The electrocardiogram signal shows that the heart rate variability index is between 50 - 100 milliseconds;

[0088] Bradycardia or tachycardia: A resting heart rate below 50 beats per minute represents bradycardia; a resting heart rate above 100 beats per minute represents tachycardia;

[0089] Arrhythmia: There are abnormal QRS waveforms or P waveforms in the electrocardiogram signal.

[0090] According to the analysis results of respiratory - heart rate synchrony and the detection and classification results of the abnormal feature capture module for respiratory patterns and heart rate variability, the comprehensive health score is as follows:

[0091] If the respiration and heart rate are in normal synchrony, and the user is in a normal respiratory pattern and has normal heart rate variability, then the score ≥ 80 points;

[0092] If the respiration and heart rate are in moderate synchrony, and the user's respiratory pattern has one abnormal phenomenon, including one of apnea, shallow breathing, and tachypnea, and at the same time the user has one heart rate abnormal phenomenon, including one of bradycardia, tachycardia, and arrhythmia, then 60 ≤ score < 80 points;

[0093] If the respiration and heart rate are in abnormal synchrony, and the user's respiratory pattern has two or more abnormal phenomena, including two or more of apnea, shallow breathing, and tachypnea, and the user has two or more heart rate abnormal phenomena, including two or more of bradycardia, tachycardia, and arrhythmia, then the score < 60 points;

[0094] After the above - mentioned step of data, through the joint analysis of respiratory sound wave signals and electrocardiogram signals combined with doctor's judgment, the health status evaluation is obtained, which is the sample data.

[0095] S6. Prediction of exercise respiration:

[0096] Optimize and construct a BP neural network. Combine the sample data obtained in step 5 with the BP neural network to achieve the prediction of the respiratory movement state of the human body. The specific implementation steps are as follows:

[0097] S6a: Construct a BP neural network and optimize the BP neural network by integrating the simulated annealing genetic algorithm; S6b: Optimize the weighted error of the BP neural network; S6c: Based on the sample data evaluated in S5, obtain the respiratory motion data, and the obtained respiratory motion data is the result of the overall evaluation of the respiratory frequency f1 of the target, the diaphragmatic electromyogram signal E, the heart rate f2, and the environmental parameter Y; S6d: Use the processed respiratory motion data to train the BP neural network with optimized weighted error to obtain a strong regressor; S6e: Use the strong regressor to predict the respiratory motion state of the human body. Thus, through the above operation method steps, the respiratory motion prediction method based on multi-modal data fusion of the neural network can improve the accuracy and reliability of respiratory health monitoring. Through precise monitoring, it is possible to better control symptoms, reduce the number of attacks, and avoid the troubles during acute symptom attacks.

[0098] A multi-modal human respiratory prediction system is as follows:

[0099] In the implementation of the multi-modal human respiratory prediction method, the present application also proposes to build a multi-modal human respiratory prediction system. It is a common and mature technology for the system to be executed by a processor through a computer program, and the present application will not elaborate again; through the system building implementation method, the system includes the following key modules:

[0100] It includes a data acquisition module, a preprocessing module, a feature extraction module, an abnormal feature capture module, a comprehensive health score module, an intervention module, and a prediction module, and is used on a computer; among them, the data acquisition module collects the respiratory frequency, diaphragmatic electromyogram signal, electrocardiogram signal, and environmental parameter data signal, and wirelessly transmits the collected signals to the preprocessing module; the operation of this data acquisition module implements the above method steps S1 to S4;

[0101] The preprocessing module, feature extraction module, abnormal feature capture module, comprehensive health scoring module, and intervention module implement the method operations of step S5; the preprocessing module receives the monitoring signals transmitted by the wearable monitoring device, and preprocesses the received respiratory sound wave signals, electrocardiogram signals, and environmental parameter signals respectively, including normalization and noise reduction processing, and converts the signals after normalization and noise reduction processing into time series data; the feature extraction module extracts respiratory sound wave features and electrocardiac activity features from the time series data; the abnormal feature capture module captures the time-dependent changes of the respiratory sound wave according to the extracted respiratory sound wave features, and detects and classifies the respiratory pattern; at the same time, it is used to capture the time-dependent changes of the electrocardiac activity according to the extracted electrocardiac activity features, and detect and classify the heart rate variability; the comprehensive health scoring module is used to perform a comprehensive health score according to the detection and classification results of the respiratory pattern and heart rate variability by the abnormal feature capture module; the intervention module is used to generate health intervention suggestions according to the comprehensive health score obtained by the comprehensive health scoring module and display them to the user;

[0102] The prediction module implements the method operations of S6; the data extracted by the modules through the above method operations of step S5 are further processed and calculated to complete the prediction of the respiratory movement state.

[0103] The specific content of the system construction is as follows:

[0104] Data acquisition module and preprocessing module: The data acquisition module is used to acquire the user's respiratory rate, diaphragmatic electromyogram signal, electrocardiogram signal, and environmental parameters, and wirelessly transmit the acquired signals to the preprocessing module. The acquired physiological signals (respiratory rate, diaphragmatic electromyogram signal, electrocardiogram signal, environmental data) may contain noise, so preprocessing is required to improve the analysis accuracy. Normalization is completed by using Min-Max normalization or Z-score normalization to make the data values on the same scale (0-1 or mean 0, standard deviation 1), and then the irrelevant frequency band signals are filtered out by a filter. For example, the electrocardiogram signal is commonly band-pass filtered at 0.5-50Hz, and then the power frequency interference (50Hz) and electromyogram noise are removed through wavelet transform to complete noise reduction. Finally, the time series data is converted into a sliding window structure, the sliding window is sliced, and the long time series data is divided into samples of a fixed length to adapt to the input of the neural network; after the operation, the data has been organized into a time series format, and structured time series data is obtained.

[0105] Feature extraction module:

[0106] It is used to extract respiratory acoustic wave features and electrocardiogram activity features from time series data. By processing the time series data with DCNN, key features in respiratory and electrocardiogram signals are extracted to improve the classification accuracy. In the processing of ECG signals, main electrocardiogram features such as P waves, QRS waves, and T waves are mainly extracted. Among them, a convolutional layer and a pooling layer are built, and its design includes a convolutional layer, a max pooling layer, and batch normalization. Local features are extracted through the convolutional layer using 3×3 and 5×5 filters. The dimension is reduced through the pooling layer to reduce the amount of calculation. Finally, the model training is stabilized through batch normalization. In the processing of respiratory acoustic wave signals, features such as respiratory rate, inspiration / expiration duration, and respiratory sound intensity are extracted, feature learning is performed through DCNN, and the dimension is reduced using the pooling layer to improve the signal resolution ability.

[0107] Through the above construction, the preliminary feature extraction of electrocardiogram and respiratory signal features is completed. It should be noted that: the DCNN structure can effectively reduce the data dimension, while retaining useful features. At the same time, different sizes of convolutional kernels can be used to extract short-term and long-term features, improving the model recognition ability.

[0108] Subsequently, further feature extraction and classification are performed through DCNN, prompting the deep learning model to extract time series features layer by layer, and finally forming high-level features for classification; the operation steps include: using activation functions (such as ReLU or tanh) and regularization techniques (such as dropout) to enhance the nonlinear processing ability and generalization ability of the model, prevent overfitting, using max pooling to reduce the dimension of the feature map while retaining the most important features, and finally designing a classifier, including a fully connected layer and a Softmax classification layer. At the end of the network, all features are integrated by using the fully connected layer, and multi-class classification is performed through the softmax layer, such as distinguishing normal breathing, shallow breathing, and coughing, etc.; after the above steps, higher-order features are extracted and classified through DCNN, which is the final feature extraction and classification; it should be noted that the features extracted through DCNN can be directly used for classification to improve the accuracy of respiratory state recognition.

[0109] It should be noted that: the convolution operation is performed within each convolutional layer, and the calculation formula is: Among them, represents the output of the kth convolutional kernel at position (i,j), are the weights of the convolutional kernel, X is the input image or the feature map of the previous layer, b k is the bias term, and σ is the activation function (such as ReLU);

[0110] The max pooling formula is as follows: P ij =max(X kl ),fork,l∈window of(i,j)Among them, P ijis the output at position (i, j) after the pooling operation, and X kl are the elements within the corresponding window on the input feature map.

[0111] Abnormal feature capture module:

[0112] Analyze the dynamic changes of time series data through a gated recurrent unit (GRU) to capture abnormal patterns for predicting future respiratory states and warning of mutations. It should be noted that the gated recurrent unit is hereinafter referred to as GRU for short. GRU model configuration: handle the dependencies of time series, effectively capture the dynamic patterns of signals changing over time; through the gating mechanism (update gate and reset gate), adjust the information flow to optimize the influence of historical data on the current state. It should be noted that the GRU structure includes an update gate, a reset gate, a candidate hidden state, and a final hidden state. The calculation formulas for the update gate and the reset gate are respectively:

[0113] Z t = σ(W z · [h t-1 , X t + b z )

[0114] r t = σ(W r · [h t-1 , X t + b r )

[0115] where Z t is the update gate, r t is the reset gate, W z and W r are weight matrices, b z and b r are bias terms, and σ is the sigmoid function.

[0116] The candidate hidden state is given by the following formula:

[0117] where W is the weight matrix, b h is the bias term, ⊙ represents element-wise multiplication, and tanh is the activation function.

[0118] The final hidden state is updated as:

[0119] This allows the model to effectively integrate current new information while retaining past information.

[0120] By optimizing the parameters of the GRU layer, adjusting the time step size, the number of hidden units, and the learning rate, the learning ability of the model for dynamic features is improved; by learning historical patterns through GRU, the future respiratory state is predicted, abnormal trends are detected in advance, mutations in respiratory patterns are identified, and early warnings (such as sleep apnea, arrhythmia, etc.) are provided. The implementation steps are as follows: Input historical respiratory signals (time series) into GRU, and through recursive calculation of GRU, learn the dynamic change patterns of the signals, and output the predicted values for future time steps: Predict the risk of apnea (detect whether there may be long periods of apnea in the future), predict the trend of arrhythmia (such as changes in HRV, which may lead to atrial fibrillation), and generate warning signals based on the prediction results to prompt potential health risks.

[0121] Flatten the extracted features, convert them into an input format acceptable to GRU, and input them into the GRU model. Calculate the change patterns in the time series through the update gate and the reset gate; GRU extracts time-dependent features and identifies abnormal points, such as: abnormal respiratory intervals (such as long resting signals during apnea), abnormal electrocardiogram signals (such as ST segment elevation, arrhythmia); Optimize the abnormal detection performance: Combine DCNN to further extract local abnormal features, improve the detection accuracy, and set abnormal thresholds (such as sudden changes in RR interval, sudden drop in heart rate, etc., to improve the detection accuracy).

[0122] GRU further processes the ECG signals, analyzes the changes in heart rhythm, detects abnormal cardiac activities, and combines heart rate variability analysis to judge the heart health status.

[0123] ECG signal processing: After feature extraction, GRU processes these time series features to capture the time-dependent changes in the electrocardiogram signals; through the update gate and reset gate mechanisms, GRU can retain long-term dependence information and filter out important instantaneous changes; thus it is particularly crucial for predicting heart rate abnormalities (such as arrhythmia) and identifying long-term trends (such as the risk of heart disease).

[0124] The heart rate variability analysis is further described as follows:

[0125] Normal heart rate variability: The electrocardiogram signals show normal heart rate variability, and the standardized heart rate variability index (SDNN) is within the normal range (50 - 100 ms), which will evaluate the user's heart function health without obvious abnormalities.

[0126] Bradycardia or tachycardia: If a significantly low or high heart rate is detected, such as a resting heart rate < 50 bpm (bradycardia) or > 100 bpm (tachycardia), it will prompt that the user may be at risk of bradycardia or tachycardia, and further medical examinations are recommended.

[0127] Arrhythmia: If abnormal QRS waveforms or P waveforms are detected in the electrocardiogram signal, such as a QRS waveform width exceeding 120 milliseconds or an abnormal P wave shape, it will prompt that the user may be at risk of arrhythmia or other heart diseases.

[0128] Respiratory acoustic signal processing: Process the respiratory acoustic features extracted by DCNN, capture the time-dependent changes in the breathing pattern. Through its gating mechanism, GRU can effectively analyze the changes in the breathing pattern, predict the future breathing state, and classify and give early warnings for abnormal breathing (such as shallow breathing, coughing).

[0129] Further description of abnormal breathing pattern detection includes the following detection modes:

[0130] Normal breathing pattern: When the breathing rate is within 12 - 20 breaths per minute, the breathing function health of the user will be evaluated and there is no obvious abnormality.

[0131] Apnea: If frequent apnea phenomena are detected, such as more than 5 apneas per hour and each apnea lasting more than 10 seconds, the system will prompt that the user may have sleep apnea syndrome and recommend further medical examinations.

[0132] Shallow breathing or tachypnea: If the breathing rate significantly increases or decreases, such as a breathing rate exceeding 25 breaths per minute (tachypnea) or less than 10 breaths per minute (shallow breathing), the system will prompt that the user may be at risk of chronic obstructive pulmonary disease (COPD) or other respiratory diseases.

[0133] The key to this step is that GRU can integrate the information in consecutive frames, thereby capturing temporal patterns in the sequence data, which is particularly important for rapidly changing dynamic lung diseases (such as lung changes caused by COVID-19).

[0134] Through the above methods, DCNN is responsible for feature extraction, extracting key features from ECG or respiratory signals. GRU processes time dependence and can learn the dynamic changes in the time series, such as sudden changes in the breathing pattern. Combining DCNN and GRU makes the detection of abnormal breathing more accurate. At the same time, traditional feature extraction requires manual design, while DCNN saves a lot of time through automated deep learning feature extraction. GRU avoids information redundancy through its gating mechanism and improves computational efficiency.

[0135] The training data of the deep learning model usually comes from the actually collected electrocardiogram and respiratory sound wave signals (refer to the above data collection process). Before using these data to train the model, strict preprocessing must be carried out, including steps such as cleaning, standardization, and feature extraction, to ensure that the data is suitable for training an efficient machine learning model; after the deep learning model is trained, the built-in integrated AI interpretation module can automatically run after each model decision, analyze and display which features have a significant impact on the decision; for example, when automatically diagnosing arrhythmia or abnormal breathing, it is clear which signal segments or feature parameters are crucial to the diagnosis result. LIME can locally linearly approximate the behavior of the model and explain a single prediction result; while SHAP evaluates the average contribution of each feature to the model prediction through game theory methods.

[0136] To further explain LIME: After each prediction, a local linear model is generated near the input data to approximate the behavior of the deep learning model. By analyzing the weights of the local model, LIME identifies the most critical features. For example: In the diagnosis of arrhythmia, LIME can show which electrocardiogram signal segments (such as QRS waveforms) have the greatest impact on the prediction result.

[0137] To further explain SHAP: Based on the Shapley value in game theory, a contribution value is assigned to each feature, and this contribution value represents the average contribution of the feature to the model prediction result; SHAP can provide global feature importance (which features are overall the most important in all predictions) and local feature importance (which features are the most important in a specific prediction). For example: In the detection of abnormal breathing, SHAP can analyze how specific changes in breathing frequency or sound wave features affect the diagnosis result.

[0138] Comprehensive health scoring module:

[0139] The comprehensive health scoring module, through the joint analysis of the respiratory sound wave signals and electrocardiogram signals obtained after the above processing, can provide the following specific health status evaluations:

[0140] Analysis of respiratory and heart rate synchronization:

[0141] The analysis of respiratory and heart rate synchronization is achieved using the Pearson correlation coefficient;

[0142] Assume the respiratory signal data sequence is: R = {r1, r2, ……, r n}, and the electrocardiogram signal data sequence is: H = {h1, h2, ……, h n};

[0143] Calculate the mean value

[0144] The mean value of the respiratory signal data:

[0145] Mean of electrocardiogram signal data:

[0146] Calculate covariance:

[0147] Calculate standard deviation

[0148] Standard deviation of respiratory signal data:

[0149] Standard deviation of electrocardiogram signal data:

[0150]

[0151] Calculate Pearson correlation coefficient:

[0152]

[0153] Normal synchronization: If the correlation coefficient between the respiratory signal and the electrocardiogram signal reaches above 0.8, and the change trends of the respiratory rate and the heart rate are consistent, and the user's respiration and heart rate are within the normal range, it indicates that the user's cardiopulmonary function is healthy.

[0154] Moderate synchronization: If the correlation coefficient between the respiratory signal and the electrocardiogram signal is between 0.5 and 0.8, and the change trends of the respiratory rate and the heart rate are partially consistent, it indicates that there are certain abnormalities in the user's cardiopulmonary function, but it has not reached a serious level. The system will make a judgment to prompt the user to conduct further observation, and conduct more detailed examinations or follow-ups in combination with the suggestions of medical staff.

[0155] Abnormal synchronization: If it is found that there is a significant asynchrony or abnormal change in time between the respiratory signal and the electrocardiogram signal, and the correlation coefficient is below 0.5, the system will make a judgment to prompt that there may be abnormalities in the cardiopulmonary function, such as respiratory failure or arrhythmia and other problems.

[0156] Comprehensive health score: Through the comprehensive analysis of the respiratory and heart rate signals, an overall health score is generated; this score is based on multiple health indicators, such as respiratory rate, heart rate variability, signal synchronization, etc., to provide an intuitive assessment of the health status. The calculation of the score is as follows:

[0157] High score: The comprehensive score is above 80 points, all key indicators are within the normal range, the respiration and heart rate are in normal synchronization (correlation coefficient ≥ 0.8), the user is in a normal breathing mode (respiratory rate is within 12 - 20 times), and the heart rate variability is normal (the electrocardiogram signal shows that the heart rate variability is between 50 - 100 milliseconds). It indicates that the user's cardiopulmonary function is healthy, the lifestyle is good, and the health risk is low.

[0158] Medium score: The comprehensive score is between 60 and 80 points. The respiration and heart rate show moderate synchronization (the correlation coefficient is between 0.5 and 0.8). And there is any abnormal phenomenon in the user's breathing pattern, including apnea (more than 5 apneas per hour, each apnea lasting more than 10 seconds), shallow breathing (respiratory rate lower than 10 breaths per minute), or tachypnea (respiratory rate higher than 25 breaths per minute). And there is any abnormal heart rate phenomenon in the user, including bradycardia (resting heart rate lower than 50 beats per minute), tachycardia (resting heart rate higher than 100 beats per minute), or arrhythmia (abnormal QRS waveform or P waveform in the electrocardiogram signal). It indicates that the user's cardiopulmonary function is basically healthy, but there are some risk factors that need attention, such as mild arrhythmia or abnormal breathing pattern.

[0159] Low score: The comprehensive score is lower than 60 points, multiple key indicators are abnormal, and there are two or more abnormal phenomena in the user's breathing pattern, including (more than 5 apneas per hour, each apnea lasting more than 10 seconds), shallow breathing (respiratory rate lower than 10 breaths per minute), or tachypnea (respiratory rate higher than 25 breaths per minute). And there are two or more abnormal heart rate phenomena in the user, including bradycardia (resting heart rate lower than 50 beats per minute), tachycardia (resting heart rate higher than 100 beats per minute), or arrhythmia (abnormal QRS waveform or P waveform in the electrocardiogram signal); it indicates that there may be potential health problems.

[0160] In view of the above evaluation calculation, in the comprehensive health module scoring module, a health score will be generated, which is the sample data obtained from the evaluation and is used in the subsequent prediction module.

[0161] Intervention module:

[0162] According to the comprehensive health score obtained in the comprehensive health scoring module, generate health intervention suggestions and display them to the user, based on determining whether there is an abnormal breathing state, and provide suggestions for doctor intervention in order to intervene as early as possible to prevent the occurrence of serious respiratory diseases.

[0163] Prediction module:

[0164] Build a neural network for predicting the respiratory movement state, including an input layer, a hidden layer, and an output layer, and fuse the simulated annealing genetic algorithm to optimize the BP neural network to improve the prediction stability and accuracy of the BP neural network;

[0165] Among them, further explanation is that the genetic algorithm improves the global search ability of the BP neural network through selection, crossover, and mutation; simulated annealing further optimizes the local search ability of the genetic algorithm to reduce the local optimum problem in BP training.

[0166] Adjust the weights of the BP neural network to minimize the error and improve the model prediction accuracy, actually achieving the optimization of the weighted error of the BP neural network;

[0167] First, build a multi-modal data fusion including input data, data preprocessing, and a fusion evaluation model; among them, the input data includes multiple feature data such as the respiratory rate, diaphragmatic electromyogram signal, heart rate, and environmental parameters obtained after the above processing (the output result of the comprehensive health score module); data preprocessing is then performed for standardization, denoising, and feature extraction to make the data suitable for BP neural network training; subsequently, these inputs are integrated through the fusion evaluation model to obtain a more stable overall respiratory state evaluation result and obtain respiratory movement data;

[0168] Train the optimized BP neural network using the above-processed respiratory movement data to improve the generalization ability of the BP network, enabling it to accurately predict the respiratory movement states of different individuals, thereby obtaining a strong regressor that can learn the respiratory movement pattern and make predictions;

[0169] Finally, use the strong regressor to predict the newly input physiological signals, thereby outputting the predicted respiratory rate, determining whether there are abnormalities (such as apnea, abnormal rhythm, etc.), and providing a further health status evaluation, accurately predicting the respiratory movement state of the human body, providing reliable support for health monitoring; thereby determining whether the respiration is normal and better promoting health management for use.

[0170] The above description is only for the purpose of illustrating the present invention. It should be understood that the present invention is not limited to the above embodiments, and various flexible forms that conform to the idea of the present invention are within the protection scope of the present invention.

Claims

1. A multi-modal human breathing prediction method, characterized in that, including S1. Obtain the respiratory rate f1: Use the visible light video and the thermal infrared video to obtain the difference time series T, and then obtain the respiratory rate f1 of the target human body; S2. Obtain the diaphragmatic electromyogram signal E: With the help of an electromyogram signal acquisition system, collect the electrophysiological signal containing the surface diaphragmatic electromyogram from the thoracic skin surface, and process it to obtain the separated diaphragmatic electromyogram signal E; S3. Obtain the heart rate f2: Use an electrocardiogram signal acquisition system to obtain the heart rate f2 of the target human body; S4. Obtain the environmental parameter Y: Use the acquisition system of environmental parameters to obtain the environmental parameter Y where the target human body is located; S5. Build a respiratory state assessment model: Preprocess the parameters obtained in steps S1 - 4, convert the signals into time series data, extract respiratory acoustic wave features and electrocardiac activity features from the time series data, capture the time-dependent changes of the respiratory acoustic wave according to the extracted respiratory acoustic wave features, detect and classify the respiratory pattern, and then evaluate the detection and classification results of the respiratory pattern and heart rate variability to obtain sample data; S6. Predict exercise respiration: Optimize and construct a BP neural network, and cooperate with the sample data obtained in step S5 through the BP neural network to realize the prediction of the respiratory movement state of the human body.

2. The multimodal human respiration prediction method according to claim 1, characterized in that The steps of using the visible light video and the thermal infrared video in step S1 are as follows: S1a: Detect key points of the visible light video image to obtain the first facial key point set of the target human body; S1b: According to the first facial key point set and the preset registration matrix, obtain the second facial key point set of the target human body in the thermal infrared video image; S1c: According to the thermal infrared video image and the second facial key point set, obtain the first thermal infrared data and the second thermal infrared data; S1d: Obtain the difference time series T according to the difference between the first thermal infrared data and the second thermal infrared data; S1e: According to the difference time series T, obtain the respiratory rate of the target human body.

3. The multimodal human respiration prediction method according to claim 1, wherein The steps of using multivariate empirical mode decomposition in step S2 are as follows: S2a: Collect the surface diaphragmatic electromyogram signal and use frequency domain analysis method for denoising; S2b: Perform discrete Fourier transform on the original surface diaphragmatic electromyogram signal to obtain the surface diaphragmatic electromyogram signal after filtering out low-frequency noise; S2c: Execute the respiratory-related signal extraction algorithm to obtain the extracted respiratory-related surface diaphragmatic electromyogram signal, and obtain the diaphragmatic electromyogram signal E after filtering out interference and noise from the preprocessed original signal.

4. The multimodal human respiration prediction method according to claim 1, wherein, The environmental parameter Y includes environmental temperature, air humidity, atmospheric pressure value, and environmental noise value.

5. The multi-modal human respiration prediction method according to claim 1, wherein The method for preprocessing the parameters obtained in S1-4 in S5 is as follows: perform normalization and noise reduction processing, and convert the signal after normalization and noise reduction processing into time series data T x_in .

6. The multi-modal human respiration prediction method according to claim 5, characterized in that The method for extracting respiratory acoustic wave features and electrocardiac activity features from the time series data in S5 is: Use a pre-trained deep convolutional neural network to extract respiratory acoustic wave features from the surface diaphragmatic electromyogram data E and the obtained respiratory rate f1; Use a pre-trained deep convolutional neural network to extract electrocardiac activity features from the electrocardiogram time series data, including: P wave, QRS wave, and T wave; Use a pre-trained deep convolutional neural network to extract environmental parameter features from the environmental parameter data Y.

7. The multimodal human respiration prediction method according to claim 6, wherein The method for detecting and classifying the breathing pattern in S5 is as follows: A deep convolutional neural network is used through multiple convolutional and max-pooling operations, while using activation functions and regularization techniques. A fully connected layer is used at the end of the network to integrate all features, and multi-class classification is performed through a softmax layer.

8. The multimodal human respiration prediction method according to claim 7, wherein, The method for detecting and classifying the breathing pattern and heart rate variability in S5 is as follows: The synchrony between breathing and heart rate is analyzed by calculating the Pearson correlation coefficient. If the Pearson correlation coefficient between the breathing sound wave signal and the electrocardiogram signal exceeds 0.8, and the changing trends of the breathing frequency and heart rate are consistent, then the user's breathing and heart rate are within the normal range, that is, the breathing and heart rate are in normal synchrony. If the Pearson correlation coefficient between the breathing sound wave signal and the electrocardiogram signal is below 0.5, and the changing trends of the breathing frequency and heart rate are inconsistent, then the breathing and heart rate are in abnormal synchrony. If the Pearson correlation coefficient between the breathing sound wave signal and the electrocardiogram signal is between 0.5 and 0.8, and the changing trends of the breathing frequency and heart rate are 50% to 80% consistent, then the breathing and heart rate are in moderate asynchrony. According to the analysis result of the breathing and heart rate synchrony and the detection and classification results of the breathing pattern and heart rate variability by the abnormal feature capture module, a comprehensive health score is performed to obtain sample data. Among them, the breathing pattern includes: normal breathing pattern, apnea, and shallow breathing or tachypnea.

9. The multimodal human respiration prediction method according to claim 1, wherein The specific method for predicting the breathing motion state of the human body in S6 is as follows: S6a: Construct a BP neural network and optimize the BP neural network by integrating the simulated annealing genetic algorithm. S6b: Optimize the weighted error of the BP neural network. S6c: Based on the sample data evaluated in S5, obtain breathing motion data. S6d: Use the processed breathing motion data to train the BP neural network with optimized weighted error to obtain a strong regressor. S6e: Use the strong regressor to predict the breathing motion state of the human body.

10. A multimodal human breathing prediction system, comprising the multimodal human breathing prediction method according to any one of claims 1-9, characterized in that, It includes a data acquisition module, a preprocessing module, a feature extraction module, an abnormal feature capture module, a comprehensive health score module, an intervention module, and a prediction module. The data acquisition module is used to collect the user's breathing frequency, diaphragmatic electromyogram signal, electrocardiogram signal, and environmental parameters, and wirelessly transmit the collected signals to the preprocessing module. The preprocessing module is used to receive the monitoring signals transmitted by the wearable monitoring device, and preprocess the received breathing sound wave signal, electrocardiogram signal, and environmental parameter signal respectively, including normalization and noise reduction processing, and convert the signals after normalization and noise reduction processing into time series data. The feature extraction module is used to extract breathing sound wave features and electrocardiac activity features from the time series data. The abnormal feature capture module is used to capture the time-dependent changes of the breathing sound wave according to the extracted breathing sound wave features, detect and classify the breathing pattern; at the same time, it is used to capture the time-dependent changes of the electrocardiac activity according to the extracted electrocardiac activity features, detect and classify the heart rate variability. The comprehensive health scoring module is used to perform a comprehensive health score based on the detection and classification results of the abnormal feature capture module on the breathing pattern and heart rate variability; The intervention module is used to generate health intervention suggestions and display them to the user according to the comprehensive health score obtained by the comprehensive health scoring module. The prediction module is used to establish a neural network, and process, optimize, and train the breathing motion data obtained from the comprehensive health scoring module to predict the breathing motion state.

Images