CNN-LSTM neural network-based breathing pattern classification method and system

Abstract

The invention provides a breathing mode classification method and system based on a CNN-LSTM neural network, and the method comprises the steps: collecting body surface point cloud data in a non-shielding state and an arm shielding state in a human body breathing process, and extracting a breathing motion feature on the basis of extracting the breathing motion feature; the invention provides an optimization method for abnormal respiratory movement characteristics of a significant respiratory movement area, solves the interference of environmental noise and newborn clinical characteristics on respiratory signals, and comprises the following steps of: firstly, partitioning a thoracic and abdominal voxel model, and extracting the respiratory movement characteristics of the significant area based on KPCA (Kernel Principal Component Analysis); motion artifacts are removed through a Savitzky-Golay filter, body motion interference is inhibited by using a method of fusing a peak threshold method and a local variance threshold, and baseline drift is removed by using a grey wolf optimization algorithm. According to the method, a CNN-LSTM neural network is constructed to classify four breathing modes of normal, rapid, slow and pause, model parameters and evaluation indexes are determined, effective classification and recognition of the breathing modes are achieved, and the monitoring precision and clinical application value are improved.

Description

Technical Field

[0001] This application relates to the field of medical data processing technology, and in particular to a respiratory pattern classification method and system based on CNN-LSTM neural network. Background Technology

[0002] In recent years, the health of newborns and infants has received widespread attention, and neonatal respiratory monitoring is a crucial aspect of ensuring their health. In both clinical and home care settings, respiratory rate and pattern are important indicators for assessing vital signs. Traditional contact-based monitoring methods rely heavily on wearable sensors, such as electrode patches, which are not only susceptible to physical movement but may also cause skin allergies, detachment, and decreased comfort in newborns. Therefore, research into non-contact respiratory monitoring technology is particularly important, as its advantages include improved patient comfort, reduced infection risk, and enhanced monitoring convenience, continuity, and accuracy.

[0003] Among numerous non-contact technologies, depth cameras are widely used for continuous monitoring of respiratory rate due to their cost-effectiveness and safety. However, existing methods still face the following shortcomings: First, the surface point cloud data acquired by depth cameras is easily affected by factors such as changes in ambient lighting, noise, and arm occlusion. Second, for respiratory movements in the chest and abdomen, the degree of movement varies in different local areas; a model of the entire chest and abdomen respiratory movement without filtering for the most salient areas makes it difficult for algorithms to accurately extract the obvious features of the salient respiratory regions. Third, in actual monitoring scenarios, newborns may experience random hand and foot tremors, body rotation, or other body movement interference, or body artifacts caused by lighting. These noisy signals can lead to errors in respiratory feature extraction and confusion in respiratory pattern classification. Fourth, respiratory curves suffer from baseline drift and amplitude instability; if not effectively corrected, this will affect the accuracy of subsequent respiratory pattern recognition. In addition, existing respiratory classification algorithms mostly rely on traditional machine learning or single neural network structures, making it difficult to simultaneously consider local deformation features and time-series dependencies, resulting in limited ability to identify and classify different respiratory patterns.

[0004] Therefore, there is an urgent need for a method that can accurately extract real and pure respiratory motion features from the body surface and achieve high-precision respiratory pattern recognition and classification, so as to improve the reliability and clinical applicability of non-contact respiratory monitoring. Summary of the Invention

[0005] In view of this, the purpose of this application is to propose a breathing pattern classification method that can specifically solve existing problems. A breathing pattern classification network combining convolutional neural networks (CNNs) and long short-term memory networks (LSTMs) is proposed. Pure breathing features are obtained through steps such as constructing a voxel model from body surface point clouds, salient region screening, motion artifact removal, body motion interference suppression, and baseline drift correction. By combining the advantages of convolutional networks and long short-term memory networks, effective classification of four modes—normal breathing, rapid breathing, slow breathing, and sleep apnea—is achieved.

[0006] To achieve the above objectives, this application proposes a breathing pattern classification method based on a CNN-LSTM neural network, including: Step S1: Collect surface point cloud data of the human body in an unobstructed state during breathing and surface point cloud data of the human body in an arm-obstructed state, and establish a surface voxel model based on the surface point cloud data; Step S2: Preprocess the voxel model, including dividing the body surface region into blocks and extracting salient regions; Step S3: Extract breathing features from the above salient regions, and remove motion artifacts, body motion interference, and baseline drift to obtain clean features, and construct training and testing sets; Step S4: Construct a CNN-LSTM neural network to classify four breathing patterns: normal, rapid, slow, and paused. Determine the model parameters and evaluation metrics, and output the effective classification of the breathing patterns.

[0007] To achieve the above objectives, this application also proposes a breathing pattern classification system based on a CNN-LSTM neural network, comprising: The body surface breathing model establishment module is used to collect body surface point cloud data in an unobstructed state and body surface point cloud data in an arm-obstructed state during human breathing, and to establish a voxel model based on the body surface point cloud data. The preprocessing module is used to preprocess the voxel model, including segmenting the body surface region and extracting salient regions; The feature extraction module is used to construct training and testing sets, and remove motion artifacts, body motion interference and baseline drift from the above-mentioned significant breathing regions to obtain clean features; The classification module is used to construct a CNN-LSTM neural network to classify four breathing patterns: normal, rapid, slow, and paused. It also determines the model parameters and evaluation metrics, and outputs the effective classification results of the breathing patterns.

[0008] In summary, the advantages of this application and the user experience it brings are as follows: It proposes a breathing pattern classification method and system based on CNN-LSTM neural network, constructs a CNN-LSTM classification model that integrates the local feature extraction capability of CNN and the temporal modeling capability of LSTM, and achieves high-precision classification of four types of breathing patterns: normal, rapid, slow and paused. It solves the problems of inaccurate breathing feature extraction, susceptibility to motion artifacts and body activity interference, significant baseline drift, and difficulty in accurately distinguishing multiple breathing patterns in non-contact breathing monitoring. Attached Figure Description

[0009] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0010] Figure 1 This is a flowchart of the respiratory pattern classification method implemented according to this application.

[0011] Figure 2 A flowchart for collecting point cloud data of the chest and abdomen and establishing a voxel model.

[0012] Figure 3 This is a block diagram of the surface voxel model region.

[0013] Figure 4 PCA dimensionality reduction diagrams for different regions of the body surface voxel model.

[0014] Figure 5 Evaluation function graph extracted from regions of significant respiratory movement on the body surface.

[0015] Figure 6 This is a set of data showing significant motion analysis of the body surface. Figure (a) is a significant analysis of the respiratory motion region, and Figure (b) is a visualization of the significant respiratory motion region.

[0016] Figure 7 The diagram illustrates how dimensionality reduction of the salient regions in this dataset yields respiratory features, followed by filtering and denoising to obtain clean respiratory features, and peak detection is used to calculate respiratory frequency. Figure (a) shows the original signal and peak detection map of the salient regions, Figure (b) shows the smoothed signal and peak detection map after removing body artifacts, Figure (c) shows the respiratory feature map after removing body movement, and Figure (d) shows the peak detection map after removing baseline drift.

[0017] Figure 8 This is a schematic diagram of the convolution calculation process.

[0018] Figure 9 This is a diagram of the CNN-LSTM model structure.

[0019] Figure 10 A diagram showing the parameter settings for the CNN-LSTM model.

[0020] Figure 11 This shows the accuracy trend of the CNN-LSTM model on the training and validation sets.

[0021] Figure 12 The confusion matrix diagram for the validation set.

[0022] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] This application proposes a CNN-LSTM classification model that integrates convolutional neural networks (CNNs) and long short-term memory networks (LSTMs) to classify four breathing modes: normal breathing, rapid breathing, slow breathing, and apnea. CNN-LSTM not only inherits the local feature extraction capabilities of CNNs but also utilizes the temporal modeling capabilities of LSTMs to achieve efficient and accurate classification of different breathing modes. Both theoretical and experimental results demonstrate that CNN-LSTM significantly improves the classification accuracy for the four breathing modes (normal, rapid, slow, and apnea), addressing the problems of inaccurate respiratory feature extraction, susceptibility to motion artifacts and body activity interference, significant baseline drift, and difficulty in accurately distinguishing multiple breathing modes in non-contact respiratory monitoring.

[0026] In order to achieve the purpose of this application, such as Figure 1 As shown, in one embodiment of this application, a breathing pattern classification method based on a CNN-LSTM neural network is provided, including the following steps: Step S1: Collect surface point cloud data of the human body in an unobstructed state during breathing and surface point cloud data of the human body in an arm-obstructed state, and establish a surface voxel model based on the surface point cloud data; Step S2: Preprocess the voxel model, including dividing the body surface region into blocks and extracting salient regions; Step S3: Extract breathing features from the above salient regions, and remove motion artifacts, body motion interference, and baseline drift to obtain clean features, and construct training and testing sets; Step S4: Construct a CNN-LSTM neural network to classify four breathing patterns: normal, rapid, slow, and paused. Determine the model parameters and evaluation metrics, and output the effective classification of the breathing patterns.

[0027] In the specific method for respiratory pattern classification based on CNN-LSTM neural network described in this application, in step S1, a calibration plate is used to unify the coordinates of two RGB-D depth cameras fixed on both sides of the incubator (for newborns) to the same coordinate system. The two cameras are placed on the left and right sides of the subject's body to collect point cloud information of the chest and abdomen. Noise points are removed using a statistical filtering algorithm. Subsequently, coarse registration of the point cloud is performed. The FPFH feature is used to calculate the matching corresponding points in two adjacent frames of the point cloud. The FPFH feature is obtained by calculating the normal vector of each point and the normal vector of the points in the neighborhood. Point pairs are obtained by matching based on the FPFH feature, and RANSAC is used to remove incorrect matching points. Rigid body transformation is estimated from the matching point pairs, and the source point cloud is registered to the target point cloud. The source and target point cloud information are precisely registered using the ICP algorithm; then, RGB and boundary threshold segmentation are used to remove background areas and redundant information; the registered point cloud data is smoothed using the moving least squares algorithm; the processed registered point cloud information is inserted into the Octomap octree map, and a voxel map is created by defining the occupied and idle states of the point cloud in space. The process of step S1 is as follows: Figure 2 As shown.

[0028] In step S2, to achieve effective tracking and analysis of body surface movement, such as Figure 3 As shown, the human body surface is first spatially divided into several equal-area sub-regions, and voxel data for each sub-region is extracted to analyze their ability to reflect respiratory movements on the body surface. When processing continuous frame point cloud data, the three-dimensional array formed by the coordinates of the voxel block center points has high data dimensionality and redundant information. To effectively extract respiratory movement features, such as... Figure 4 As shown, the PCA algorithm is used to reduce the dimensionality of the respiratory motion region, extracting one-dimensional respiratory motion features from each region. Figure 5 As shown, further analysis of these surface respiratory motion characteristics reveals three key indicators (data periodicity, stability, and motion amplitude) to characterize the significance of respiratory motion in each surface region. To reduce the dimensionality of the region, For the evaluation function, output This serves as a regional significance indicator. After obtaining three key indicators for each region, KPCA is used to reduce the dimensionality of the attribute values ​​to identify significant respiratory motion regions, thus obtaining the significance indicator. KPCA maps the data to a high-dimensional feature space through a kernel function, making nonlinear data linearly separable. After completing this mapping, PCA dimensionality reduction is performed in the high-dimensional feature space to extract the main components that best represent the data variability.

[0029] In step S3, after extracting the respiratory motion features of the significant respiratory motion region, a series of signal processing algorithms are needed to address the impact of environmental noise and neonatal clinical features on the respiratory motion features, such as baseline shift, abnormal data points, and motion interference.

[0030] In the part that removes motion artifacts, the Savitzky-Golay filter is used to remove abnormal breathing motion features. Unlike the traditional moving average filter, the Savitzky-Golay filter can reduce high-frequency noise interference while preserving signal trends and peak information. Its filtering process can be represented as a weighted summation, where the weights are calculated by the least squares method. After filtering and denoising, the clean breathing features with motion artifacts removed are obtained.

[0031] In the part that removes the influence of significant body movement, a body movement detection algorithm that integrates peak thresholding and local variance thresholding is used. By introducing local variance thresholding and dynamic threshold adjustment, a dual-condition decision mechanism is formed. During body movement, muscle activity causes drastic signal fluctuations, with local variance significantly higher than that of calm breathing, and the amplitude of movement increases significantly. Only when both amplitude mutation and variance exceeding the limit are met simultaneously is it determined to be a body movement event. After identifying significant body movement, this part of the feature is removed, resulting in pure features that have removed the influence of significant body movement.

[0032] In the part that removes the influence of baseline drift, the gray wolf optimization algorithm is used to fit the parameters in the quasi-periodic respiratory motion model. The optimal parameters are obtained by simulating the gray wolf swarm search mechanism to minimize the error between the fitted model and the real respiratory curve, thereby realizing the estimation and subtraction of the baseline trend term and obtaining the pure respiratory characteristics after baseline drift correction.

[0033] After processing abnormal respiratory motion features, pure respiratory motion features are extracted. The peak position is detected and the respiratory rate is calculated using the AMPD algorithm based on automatic multi-scale peak detection. A set of data is taken for display. First, significant respiratory regions are extracted. Then, artifacts, body motion interference and baseline drift are removed to obtain pure respiratory motion features.

[0034] like Figure 6As shown, after dividing one set of data into 16 regions, kernel principal component analysis was used to select the fifth region as a significant respiratory motion region. Dimensionality reduction of this significant region yielded a one-dimensional respiratory motion feature. Figure 6 (a) is a saliency analysis diagram of respiratory movement regions. Figure 6 Image (b) is a visualization of the salient respiratory movement regions. The respiratory features obtained from dimensionality reduction were observed... Figure 7 (a) The original respiratory signal feature diagram shows a slow breathing rhythm and a generally stable respiratory signal, indicating a normal respiratory waveform. Peak detection was performed on the original respiratory motion features, and the respiratory rate was calculated to be 52.43 BPM. Figure 7 In (b), after removing motion artifacts using the Savitzky-Golay filter, the calculated respiratory rate is 51.31 BPM. Figure 7 (c) and Figure 7 After removing body motion and baseline drift, the calculated respiratory rate in (d) is 49.37 BPM, which is the actual respiratory rate of the infant monitor. The comparison shows that the respiratory rate calculated from the clean respiratory features after noise reduction filtering is more accurate, which also helps the subsequent classification network to make accurate identifications.

[0035] After processing abnormal respiratory motion features, this application extracted clean respiratory motion features. Based on the features of different breathing modes, dataset samples of four breathing modes were constructed: normal breathing, rapid breathing, slow breathing, and apnea.

[0036] In step S4, a CNN-LSTM classification model is constructed that integrates the local feature extraction capability of convolutional networks (CNN) with the temporal modeling capability of long short-term memory networks (LSTM) to achieve high-precision classification of four breathing patterns: normal, rapid, slow, and pause.

[0037] Convolutional Neural Networks (CNNs) demonstrate significant advantages in processing multidimensional data. For input signals of different dimensions, a CNN structure consists of an input layer, convolutional layers, pooling layers, fully connected layers, and an output layer. Through the sliding calculation mechanism of the convolutional kernels, CNNs can effectively process high-dimensional data and extract local feature information. Figure 8This paper demonstrates the process of convolutional computation of a two-dimensional matrix using a convolution kernel. When processing time-series data, such as in respiratory pattern classification tasks, LSTM can effectively capture long-term dependencies and global features. However, relying solely on LSTM may be insufficient to extract local detail features from the input sequence, which are crucial for accurately identifying different respiratory patterns. Therefore, this application proposes a hybrid model combining a one-dimensional convolutional neural network (CNN) and LSTM to leverage the advantages of both. The CNN focuses on extracting more meaningful local information from these features, while the LSTM is used to capture the long-term dependencies in the time-series data. Figure 9 The CNN-LSTM model structure of this application is shown, including one input layer, two convolutional layers, two pooling layers, one dropout layer, one LSTM layer, and one output layer. The input layer has dimensions of (None, 900, 1), indicating that each respiratory sequence is 900 bytes long and has 1 channel. It is used to feed the clean respiratory feature sequence into the network as the raw data for subsequent feature extraction. In convolutional layer 1, there are 32 convolutional kernels with a kernel size of 2×1. The ReLU activation function is used to extract local change patterns from the respiratory signal. Pooling layer 1 performs max pooling on the convolution result, reducing the length by half. This reduces the feature dimensionality, preserves more important local changes, and reduces noise and overfitting. In convolutional layer 2, there are 64 convolutional kernels with a kernel size of 2×1. The ReLU activation function is used to capture a wider range of respiratory local features. Pooling layer 2 further compresses the feature length, allowing the network to focus on key features. The LSTM layer has 128 memory units for time series modeling. The output layer outputs four types of breathing modes: normal breathing, rapid breathing, slow breathing, and sleep apnea.

[0038] To optimize the CNN-LSTM model built in this application and determine the optimal parameter values, experiments were conducted using the controlled variable method to find the optimal parameter settings. First, chest and abdominal respiratory motion signals from 15 subjects were collected using a depth camera respiratory monitoring system. A four-category respiratory sample library was selected from the dataset, covering four modes: normal breathing, rapid breathing, slow breathing, and apnea. This library was then divided into training (n=100), validation (n=200), and test (n=100) sets in a 1:2:1 ratio. Second, abnormal respiratory motion feature processing algorithms were applied to the training and validation sets for preprocessing to ensure data reliability. Next, model validation and hyperparameter tuning were performed using the validation set: unlabeled validation data was input into the current model, and the classification accuracy was calculated by comparing the prediction results with the true labels. An early stopping strategy (patience=10) was used to dynamically optimize the network weights, and the model with the highest validation set accuracy was selected as the best classifier. Finally, an independent test set was used to evaluate the model's generalization performance. Figure 10As shown, in the CNN architecture adopted in this application, the two convolutional layers are configured with 32 and 64 convolutional kernels respectively to balance feature richness and model complexity. This application uses a single-layer LSTM network with 64 hidden layer units to ensure that the model has sufficient memory capacity without becoming overly complex.

[0039] This application evaluates classification performance using four key metrics: accuracy, precision, recall, and F1 score. Figure 11 This shows the accuracy trends of the model on the training and validation sets. From Figure 11 As can be seen, the model's accuracy stabilizes after approximately 300 training iterations. Notably, the accuracy on the training and validation sets is consistent, indicating that the model does not exhibit significant overfitting. Figure 12 The confusion matrix of the validation set is shown, revealing that the CNN-LSTM model performs well overall in classifying the four breathing patterns. Although there are a few misclassified samples, the overall classification accuracy is high, indicating that the model has good robustness and reliability in distinguishing different breathing patterns. In the test results of the CNN-LSTM breathing pattern classification model constructed in this application, both normal breathing and rapid breathing achieved 100% accuracy, precision, recall, and F1-score, demonstrating extremely high recognition reliability. For the slow breathing and apnea patterns, the model's classification accuracy reached 98.5% in both cases, with slow breathing achieving 100% precision, 94% recall, and an F1-score of 96.91%, and apnea achieving 94.34% precision, 100% recall, and an F1-score of 97.09%. Overall, the model achieved a classification accuracy of 98.50%, a precision of 98.59%, a recall of 98.50%, and an F1-score of 98.50%, demonstrating that this application can maintain stable and high-level classification performance in multi-class breathing pattern recognition tasks.

[0040] This application also provides an electronic device corresponding to the CNN-LSTM neural network-based breathing pattern classification method described in the foregoing embodiments, for executing the breathing pattern classification method described above. This application does not limit the scope of the application.

[0041] Please refer to Figure 13 The electronic device may include: a processor, a memory, a communication interface, and a bus, wherein the processor, the communication interface, and the memory are connected via the bus; the memory stores a computer program that can run on the processor, and when the processor executes the program, it implements the breathing pattern classification method described in any of the foregoing embodiments.

[0042] The memory may include high-speed random access memory (RAM) or non-volatile memory (such as flash memory, disk storage, etc.) for storing program instructions and intermediate data. The processor may be a general-purpose central processing unit (CPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device.

[0043] During the operation of the method in this application, the processor loads and executes the program instructions stored in the memory, and performs functions such as establishing a body surface voxel model, extracting salient regions, purifying respiratory features, and classifying and reasoning the CNN-LSTM classification model by executing steps S1 to S4.

[0044] The electronic device provided in this application embodiment is based on the same inventive concept as the above-described method and can achieve the same technical effect as the method.

[0045] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, can implement all the steps of the breathing pattern classification method based on the CNN-LSTM neural network described in the foregoing embodiments.

[0046] The computer-readable storage medium may include, but is not limited to: phase-change memory (PRAM), flash memory, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), hard disk, optical disk, etc.

[0047] It should be noted that: The algorithms and displays provided herein are not inherently related to any pre-existing computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this application is not directed to any pre-existing programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of pre-existing languages ​​is for the purpose of disclosing the best mode of implementation of this application.

[0048] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0049] Similarly, it should be understood that, in order to simplify this application and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0050] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0051] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0052] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors can be used in practice to implement some or all of the functions of some or all of the components in the virtual machine creation system according to the embodiments of this application. This application can also be implemented as a device or system program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0053] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several systems, several of these systems may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0054] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method and system for classifying breathing patterns based on CNN-LSTM neural networks, characterized in that, include: Step S1: Collect surface point cloud data of the human body in an unobstructed state during breathing and surface point cloud data of the human body in an arm-obstructed state, and establish a surface voxel model based on the surface point cloud data; Step S2: Preprocess the voxel model, including dividing the body surface region into blocks and extracting salient regions; Step S3: Extract breathing features from the above salient regions, and remove motion artifacts, body motion interference, and baseline drift to obtain clean features, and construct training and testing sets; Step S4: Construct a CNN-LSTM neural network to classify four breathing patterns: normal, rapid, slow, and paused. Determine the model parameters and evaluation metrics, and output the effective classification of the breathing patterns.

2. The method according to claim 1, characterized in that, Step S1 includes: The coordinates of the two RGB-D cameras were unified to the same coordinate system using a calibration board. The two cameras were placed on the left and right sides of the subject's body to collect point cloud information of the chest and abdomen. Noise points in the point cloud information are removed using a statistical filtering algorithm. By coarsely registering the point cloud, the matching corresponding points in the point cloud of two adjacent frames are calculated using FPFH features. The FPFH features are obtained by calculating the normal vector of each matching corresponding point and the normal vector of the matching corresponding points in the neighborhood. Point pairs are obtained by matching based on the FPFH features, and RANSAC is used to remove erroneous matching points. The rigid body transformation is estimated by matching point pairs, the source point cloud is registered to the target point cloud, and the ICP algorithm is used to perform fine registration of the information of the source point cloud and the target point cloud to obtain the registered point cloud data. RGB and boundary threshold segmentation are used to remove background areas and redundant information from the registration point cloud data; The registration point cloud data was smoothed using the moving least squares algorithm. The processed and registered point cloud data is inserted into the Octomap octree map, and a voxel map is created by defining the occupied and idle states of the point cloud in space.

3. The method according to claim 1, characterized in that, Step S2 includes: The body surface voxel model was divided into 16 regions. Respiratory motion features of 16 regions were extracted using PCA dimensionality reduction. Using nuclear principal component analysis, representative and significant respiratory motion regions were further screened based on the periodicity, stability, and amplitude of respiratory motion characteristics.

4. The method according to claim 1, characterized in that, Step S3 includes: The Savitzky–Golay filter was used to perform local polynomial fitting on the original respiratory motion signal. By solving the filter weights by least squares within the sliding window, the signal was able to suppress random noise while maintaining the peak, trough and transient change characteristics, thus obtaining a smoothed respiratory feature signal. A body motion detection algorithm that integrates peak threshold method and local variance threshold analysis is used to remove the interference of significant body motion on respiratory features. Pure respiratory characteristic signals were obtained after baseline drift correction; After abnormal respiratory motion feature processing, pure respiratory motion features are extracted, and the peak position is detected and the respiratory rate is calculated using the AMPD algorithm based on automatic multi-scale peak detection. A dataset with multiple breathing modes was constructed based on the characteristics of different breathing modes.

5. The method according to claim 4, characterized in that, The body motion detection algorithm includes: First, the AMPD algorithm is used to extract the peak and trough points of respiration; Then, the local variance between adjacent peaks and valleys is calculated and compared with the global average variance threshold. When the peak amplitude changes abruptly and the local variance exceeds the threshold at the same time, it is determined to be a body motion event, and the start and end positions of the body motion event are marked, and the data segment at the start and end positions is removed.

6. The method according to claim 4, characterized in that, The pure respiratory feature signal obtained after baseline drift correction includes: A baseline drift estimation function based on a quasi-periodic model is constructed, and the gray wolf optimization algorithm is used to fit the model parameters. The optimal parameters are obtained by simulating the gray wolf swarm search mechanism to minimize the error between the fitted model and the real respiratory curve, thereby realizing the estimation and subtraction of the baseline trend term and obtaining the pure respiratory feature signal after baseline drift correction.

7. The method according to claim 1, characterized in that, Step S4 includes: Construct a convolutional neural network (CNN) module to extract local features from the input respiratory time series signal; A Long Short-Term Memory (LSTM) neural network module was constructed to extract the temporal dependence features of respiratory time series signals; Construct a CNN-LSTM hybrid neural network for classifying four breathing patterns; Perform model training and dataset construction; Determine the model parameters and establish evaluation metrics to verify the classification accuracy of the model.

8. A respiratory pattern classification system based on CNN-LSTM neural network, characterized in that, include: The body surface breathing model establishment module is used to collect body surface point cloud data in an unobstructed state and body surface point cloud data in an arm-obstructed state during human breathing, and to establish a voxel model based on the body surface point cloud data. The preprocessing module is used to preprocess the voxel model, including segmenting the body surface region and extracting salient regions. The feature extraction module is used to construct training and testing sets, and remove motion artifacts, body motion interference and baseline drift from the above-mentioned significant breathing regions to obtain clean features; The classification module is used to construct a CNN-LSTM neural network to classify four breathing patterns: normal, rapid, slow, and paused. It also determines the model parameters and evaluation metrics, and outputs the effective classification results of the breathing patterns.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 8.

Images