A respiratory pattern classification method based on direction time sequence and dynamics characteristic fusion

Abstract

The application discloses a respiratory mode classification method based on direction time sequence and dynamic feature fusion. The method comprises the following steps: collecting three-dimensional coordinate time sequence of multiple motion marker points of human chest and abdomen, and pre-processing the collected data; three-dimensional motion coordinate matrices of the chest and the abdomen are constructed respectively, the main motion direction of the corresponding region is determined through principal component analysis, and the three-dimensional coordinate time sequence is projected to the main motion direction to form double-channel respiratory waveforms of the chest and the abdomen; respiratory dynamic features of amplitude, frequency, phase, time lag and correlation are calculated according to the double-channel respiratory waveforms; the double-channel respiratory waveforms are input into a one-dimensional convolutional neural network to extract deep time sequence features, the respiratory dynamic features are input into a multilayer perceptron to extract interpretable features, and the two types of features are fused to complete chest breathing, abdominal breathing and chest-abdominal combined breathing classification. The application can fuse local waveform features and chest-abdominal coordination features, and effectively classify and identify the respiratory mode.

Description

Technical Field

[0001] This application relates to the field of medical data processing technology, and in particular to a method and system for classifying thoracic breathing, abdominal breathing and combined thoracic and abdominal breathing by extracting the main directions of chest and abdominal movement based on principal component analysis and by fusing a one-dimensional convolutional neural network and a multilayer perceptron in a dual-branch manner. Specifically, it relates to a breathing pattern classification method based on the fusion of directional temporal and dynamic features. Background Technology

[0002] During human respiration, the thoracic cavity and abdominal wall undergo periodic spatial displacement. Differences in the degree of involvement of the chest and abdomen can reflect different respiratory exertion methods. Therefore, continuous acquisition and classification of chest and abdominal movements can be used for respiratory training assessment, assisted ventilation effect analysis, and respiratory function status monitoring.

[0003] Existing methods for recognizing chest and abdominal breathing patterns typically employ breathing belts, pressure sensors, flow meters, or vision devices to acquire single-channel or multi-channel breathing signals. Contact sensors are susceptible to variations in wearing tightness, installation position, and local skin deformation. While visual and 3D motion capture devices can obtain richer spatial motion information, their output usually includes the coordinate time sequence of multiple motion markers along multiple coordinate axes, resulting in high data dimensionality. Furthermore, different subjects, installation angles, and acquisition postures can cause the actual breathing motion to be dispersed across different coordinate axes.

[0004] If a fixed coordinate axis is directly selected as the respiratory waveform, problems may arise such as inconsistent amplitude and direction, dispersed motion energy, or phase reversal for the same breathing pattern among different subjects. If all three-dimensional coordinates are directly input into the classification network, the network also needs to learn the differences in coordinate direction and breathing pattern simultaneously, which can easily reduce the model's generalization ability when the sample size is limited.

[0005] On the other hand, while convolutional neural networks alone can learn deep temporal features such as peaks, troughs, inspiratory-expiratory transitions, and local morphology, they struggle to explicitly represent physiologically significant synergistic relationships such as the ratio of chest-abdominal amplitude, chest-abdominal phase difference, chest-abdominal time delay, and correlation. Although artificially constructed dynamic features alone are interpretable, they cannot fully express the subtle morphological and local changes across multiple respiratory cycles.

[0006] Therefore, a respiratory pattern classification method is needed that can adaptively extract the main respiratory motion direction from three-dimensional chest and abdominal motion data and jointly model the depth temporal features with chest and abdominal respiratory dynamics features to reduce the impact of differences in coordinate orientation and marker installation, while improving the stability and interpretability of the classification results. Summary of the Invention

[0007] In view of this, the purpose of this application is to provide a respiratory pattern classification method and system based on the fusion of the main motion direction temporal sequence and the thoracic and abdominal respiratory dynamics features, so as to solve the problems of coordinate direction dependence in three-dimensional motion data, lack of physiological interpretation of single depth features, and difficulty in fully representing respiratory waveforms by single artificial features.

[0008] Specifically, this application proposes a respiratory pattern classification method based on the fusion of directional temporal and dynamic characteristics, including: Step S1: Collect three-dimensional coordinate time-series data of chest marker set and abdominal marker set during human respiration. Organize the three-dimensional coordinate time-series data into a multi-channel respiratory motion sequence according to the chest region or abdominal region where each motion marker is located. Step S2, preprocessing the multi-channel respiratory motion sequence, the preprocessing includes missing data imputation, trend term removal, filtering, respiratory cycle segmentation, fixed-length resampling and standardization; Step S3: Construct chest coordinate matrix and abdominal coordinate matrix based on the three-dimensional coordinate time series data of the chest marker set and abdominal marker set, respectively. Perform principal component analysis on the chest coordinate matrix and abdominal coordinate matrix to determine the main motion direction of the chest and the main motion direction of the abdomen. Project the corresponding three-dimensional coordinate time series data onto the determined main motion direction to obtain the main motion waveform of the chest and the main motion waveform of the abdomen. Step S4: Based on the main chest motion waveform and the main abdominal motion waveform, calculate the thoracic and abdominal respiratory dynamics characteristics used to characterize the amplitude relationship, frequency relationship, phase relationship, time delay relationship and correlation relationship between the chest and abdomen; Step S5: The main chest motion waveform and the main abdominal motion waveform are combined into a dual-channel respiratory time sequence and input into a one-dimensional convolutional neural network to obtain deep temporal features; the chest and abdominal respiratory dynamic features are input into a multilayer perceptron to obtain a respiratory dynamic feature representation. Step S6: The deep temporal features and the respiratory dynamics features are fused together, and the fused features are input into the classifier to output one of the following breathing modes: thoracic breathing, abdominal breathing, and thoracoabdominal breathing.

[0009] To achieve the above objectives, this application also provides a respiratory pattern classification system, including a data acquisition module, a preprocessing module, a main motion waveform extraction module, a respiratory dynamics feature calculation module, a dual-branch feature extraction module, and a fusion classification module, each module being used to execute the corresponding processing steps in the above method.

[0010] This application also provides an electronic device and a computer-readable storage medium, wherein when a program stored in the electronic device or computer-readable storage medium is executed, the above-described breathing pattern classification method is implemented.

[0011] Compared with the prior art, this application has at least the following beneficial effects: First, by performing principal component analysis on the three-dimensional motions of the chest and abdomen respectively, we can adaptively find the spatial direction of the energy concentration of the actual respiratory motion, thereby reducing the impact of the fixed coordinate axis selection and the change of the marker installation angle on the respiratory waveform.

[0012] Second, by correcting the direction of the principal motion for consistency, the chest and abdomen phase reversal caused by the uncertainty of the principal component direction sign is avoided, making the chest and abdomen phase relationship between different subjects and different samples comparable.

[0013] Third, the one-dimensional convolutional neural network branch can extract peaks, troughs, inhalation-exhalation transitions, local jitters, and morphological changes between multiple respiratory cycles, while the multilayer perceptron branch can explicitly encode chest and abdominal amplitude, frequency, phase, time delay, and correlation. The fusion of the two takes into account both waveform features and physiologically interpretable features. Attached Figure Description

[0014] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0015] Figure 1 This is a flowchart illustrating the overall process of the respiratory pattern classification method in this application. Figure 2 This is a schematic diagram of the acquisition of three-dimensional motion data of the chest and abdomen and the arrangement of marker points in this application; Figure 3 This is a schematic diagram of the preprocessing flow of the multi-channel respiratory motion sequence in this application; wherein, (a) is the waveform of the main motion direction before preprocessing; (b) is the preprocessing flow; and (c) is the waveform of the main motion direction after preprocessing. Figure 4 This is a schematic diagram illustrating the extraction of the main motion direction and waveform of the chest and abdomen using principal component analysis in this application. Figure 5 This is a schematic diagram of the dual-branch fusion structure of a one-dimensional convolutional neural network and a multilayer perceptron in this application. Figure 6 This is a schematic diagram illustrating the extraction of thoracic and abdominal respiratory dynamics features in this application; Figure 7 This is a schematic diagram showing the changes in the macro-average F1 score and balanced accuracy of the first fold validation set in the four-fold independent cross-validation of this application as a function of training iterations; Figure 8 This is a schematic diagram of the confusion matrix formed by summing the out-of-fold prediction results of each fold test set in the four-fold independent cross-validation of the subjects in this application; Figure 9 This application presents a schematic diagram of the module structure of the respiratory pattern classification system. Detailed Implementation

[0016] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the application. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present application are shown in the accompanying drawings.

[0017] It should be noted that, unless otherwise specified, the embodiments and technical features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] This application proposes a respiratory pattern classification method based on the fusion of temporal sequence of the main motion direction and thoracic and abdominal respiratory dynamics features. This method utilizes principal component analysis to extract one-dimensional waveforms reflecting the main respiratory movements from the three-dimensional motions of the chest and abdomen. It then uses a one-dimensional convolutional neural network to learn the local morphology and cross-cycle variation features of the chest and abdominal waveforms. A multilayer perceptron is used to encode the amplitude, frequency, phase, time delay, and correlation of the chest and abdomen. Finally, a two-branch feature fusion method is used to classify thoracic breathing, abdominal breathing, and combined chest and abdominal breathing.

[0019] In order to achieve the purpose of this application, such as Figure 1 As shown, in one embodiment of this application, a breathing pattern classification method based on the fusion of the main motion direction temporal sequence and the thoracic and abdominal respiratory dynamics features is provided, including the following steps: Step S1: Collect three-dimensional coordinate time-series data of chest marker set and abdominal marker set during human respiration. Based on the chest region or abdominal region where each motion marker is located, organize the three-dimensional coordinate time-series data into a multi-channel respiratory motion sequence. Step S2: Preprocess the multi-channel respiratory motion sequence, including missing data imputation, trend term removal, filtering, respiratory cycle segmentation, fixed-length resampling, and standardization. Step S3: Construct chest coordinate matrices and abdominal coordinate matrices based on the three-dimensional coordinate time series data of the chest marker set and abdominal marker set, respectively. Perform principal component analysis on the chest coordinate matrix and abdominal coordinate matrix to determine the main motion direction of the chest and the main motion direction of the abdomen. Project the corresponding three-dimensional coordinate time series data onto the main motion direction to obtain the main motion waveform of the chest and the main motion waveform of the abdomen. Step S4: Calculate the thoracic and abdominal respiratory dynamics characteristics based on the main chest motion waveform and the main abdominal motion waveform to characterize the amplitude relationship, frequency relationship, phase relationship, time delay relationship and correlation relationship between the chest and abdomen; Step S5: Combine the main chest motion waveform and the main abdominal motion waveform into a dual-channel respiratory time sequence, and input it into a one-dimensional convolutional neural network to obtain deep temporal features; input the chest and abdominal respiratory dynamics features into a multilayer perceptron to obtain a respiratory dynamics feature representation; Step S6: Fuse the deep temporal features and the respiratory dynamics features, input the fused features into the classifier, and output one of the following breathing modes: thoracic breathing, abdominal breathing, and thoracoabdominal breathing.

[0020] In the breathing pattern classification method described in this application, step S1 involves using a three-dimensional motion capture device capable of continuously outputting spatial coordinates to collect the respiratory movements of the subject's chest and abdomen. The three-dimensional motion capture device can be an optical motion capture device, a depth vision device, or other device capable of acquiring three-dimensional motion information of the human body surface. In one specific embodiment, a general-purpose optical motion capture system is used, and the relative positions of the positioning device and the experimental area remain unchanged within the same acquisition task.

[0021] like Figure 2 As shown, a first chest marker is placed on the left side of the subject's thorax, a second chest marker is placed on the right side of the thorax, and an abdominal marker is placed near the upper abdomen or navel. The first and second chest markers together reflect the expansion and contraction of the left and right thoracic regions, while the abdominal markers reflect the rise and fall of the abdominal wall during respiration. Each marker synchronously outputs coordinate values ​​along three spatial axes, forming a raw respiratory motion sequence composed of multiple three-dimensional coordinate channels. The number and location of the markers can be adjusted according to the acquisition device and application scenario, as long as the markers can be divided into a chest marker set and an abdominal marker set.

[0022] Before data collection began, subjects were positioned in a preset sitting, supine, or semi-reclining posture. Initial data was recorded, and the stability of coordinate output at each marker point was checked. During the formal data collection, the three-dimensional coordinates and timestamps of each marker point were recorded using a unified sampling clock. The start and end times of the breathing task, as well as the category label, were also recorded. Subjects completed thoracic breathing, abdominal breathing, and thoracoabdominal breathing tasks as instructed. Thoracic breathing primarily involved chest wall movement, abdominal breathing primarily involved abdominal wall movement, and thoracoabdominal breathing involved both the chest wall and abdominal wall.

[0023] In one specific implementation, 23 subjects were actually included, each of whom completed three breathing tasks: thoracic breathing, abdominal breathing, and combined thoracic and abdominal breathing. The continuously collected respiratory motion sequences were segmented into respiratory cycles, and a preset number of consecutive complete respiratory cycles were used to form a respiratory sample. Samples were extracted from the continuous respiratory cycle sequences using an overlapping sliding window with a preset step size. After data quality screening, a total of 6273 valid respiratory samples were obtained. The collected data were stored according to subject number, respiratory task category, collection batch, and timestamp. Each subject's data was treated as an independent data set, used for subsequent division into training, validation, and test sets based on subject identity. After collection, the missing marker rate, timestamp continuity, effective coordinate range, and coordinate abrupt changes between adjacent sampling times were checked. Data segments with long-term missing points, continuous occlusion, or significant tracking jumps were excluded from further processing.

[0024] In step S2, as Figure 3 As shown, the missing coordinates in the original multi-channel respiratory motion sequence are first processed. For missing data with short durations, linear interpolation, cubic interpolation, or interpolation based on adjacent valid points is used for completion; data segments with consecutive missing durations exceeding a preset threshold are marked as invalid data segments. Subsequently, trend term removal is performed on each coordinate channel to reduce low-frequency changes caused by the initial position of the marker points, slow overall movement of the subject, and posture drift. Trend term removal can be achieved by subtracting the average position during the initialization phase, linear detrending, polynomial detrending, or sliding baseline estimation.

[0025] The time-series data after removing the trend term is filtered. The filter can be a low-pass filter, a band-pass filter, a Savitzky-Golay smoother, or a combination of the above filtering methods. The filtering parameters are set according to the sampling frequency and the target respiratory frequency range to suppress positioning noise and high-frequency jitter, while preserving the peaks, troughs, local slopes, and inspiratory-expiratory transition characteristics of the respiratory waveform.

[0026] After filtering, respiratory peaks and troughs are detected based on a reference time series representing the chest, abdomen, or a combination of chest and abdomen time series. The boundaries of complete respiratory cycles are determined based on adjacent feature points of the same type. A preset number of consecutive complete respiratory cycles are combined into a respiratory sample. In one specific implementation, each sample contains three complete respiratory cycles. Due to the different cycle lengths of different subjects and different respiratory tasks, each sample is resampled along the time axis to a uniform length. In one specific implementation, 512 time points are resampled, and the same resampling position is used for each channel to maintain the original time and phase relationship between the chest and abdomen.

[0027] The resampled data is standardized. Standardization parameters are calculated only based on the training data and used to process the training, validation, and test sets, thus preventing statistical information from the validation or test data from entering the model training process. The preprocessed fixed-length multi-channel samples are used for subsequent extraction of the main chest and abdominal movement directions and calculation of respiratory dynamics features.

[0028] In step S3, the actual respiratory movement direction of the chest and abdomen is affected by the marker installation angle, the subject's position, and the coordinate system orientation of the acquisition device. If a fixed coordinate axis is directly selected as the respiratory waveform, inconsistencies in direction between different subjects or different acquisition batches can easily occur. Therefore, this application performs principal component analysis on the three-dimensional movements of the chest and abdomen respectively. Figure 4 This is a simplified diagram illustrating the extraction of the main motion direction, which extracts the main motion direction from a respiratory signal with a 2.5-second cycle.

[0029] For the set of chest markers, the coordinates of the left and right chest markers at the same time can be averaged to obtain the three-dimensional trajectory of the chest center. Alternatively, the coordinates of multiple chest markers can be combined to form a chest coordinate matrix. For the set of abdominal markers, the three-dimensional trajectory of the abdominal center can be used, or the coordinates of multiple abdominal markers can be combined to form an abdominal coordinate matrix. The chest and abdominal coordinate matrices are centered respectively, and the motion changes in different spatial directions are analyzed. The component that can explain the maximum motion change is determined as the main motion direction of the corresponding region.

[0030] Because of the uncertainty in the sign of the principal component direction, waveforms of the same type of respiratory sample may exhibit opposite directions. Therefore, based on the pre-defined outward direction of the body surface, the direction of change of the initial respiratory cycle, the correspondence between the reference sensor direction and the chest and abdominal waveforms, sign consistency correction is performed on the principal motion direction to ensure that the inhalation and exhalation in different samples maintain a consistent direction of change. After direction correction, the three-dimensional motion of the chest is projected onto the principal motion direction of the chest to obtain the principal motion waveform of the chest; the three-dimensional motion of the abdomen is projected onto the principal motion direction of the abdomen to obtain the principal motion waveform of the abdomen. The two waveforms are arranged at the same time position to form a dual-channel respiratory timing sequence.

[0031] In step S4, to explicitly describe the degree of involvement and coordination between the chest and abdomen in the respiratory process, the dynamic features of chest and abdominal respiration are extracted based on the main motion waveforms of the chest and abdomen. Figure 5 The diagram illustrates the root mean square amplitude ratio of the chest and abdomen, the peak-to-peak ratio of the chest and abdomen, the spectral entropy, and the chest-abdomen phase difference. The respiratory dynamics characteristics may also include the dominant frequency difference, the maximum cross-correlation degree, the time delay corresponding to the maximum cross-correlation, the phase difference fluctuation degree, and the correlation coefficient, used to describe the chest-abdomen coordination state from multiple aspects such as amplitude, frequency, phase, time delay, and correlation. Figure 5The formulas for calculating various respiratory dynamic characteristics are as follows: Root mean square of energy in the chest and abdomen: (1) (2) (3) in, and represents the amplitude of the main motion waveform of the abdomen and chest at the t-th sampling time, respectively; N is the total number of samples in a breathing segment, and the root mean square of the chest and abdomen energy represents the average motion amplitude of the chest or abdomen during these breathing cycles. and These are the root mean square amplitudes of the main motion waveforms of the abdomen and chest, respectively, reflecting the average motion intensity of the corresponding regions within that segment. ε is the root mean square amplitude ratio of the abdomen and chest at the t-th sampling time, reflecting the degree of their relative motion participation; ε is a very small positive number set to prevent the denominator from being zero.

[0032] Thoracic and abdominal spectral entropy: (4) (5) in, Let n be the amplitude of the main motion waveform of the chest or abdomen to be analyzed at the nth sampling point, where n = 0, 1, ..., N-1; for The kth frequency component obtained after FFT, where k is the frequency index and N is the total number of sampling points; Let be the normalized spectral distribution probability of the k-th frequency component. is the spectral amplitude of the j-th frequency component used for normalization, K is the total number of frequency components participating in the statistics, and FFT represents the Fourier transform.

[0033] (6) Among them, spectral entropy It is used to describe the distribution of frequency components in a breathing waveform, reflecting the spectral complexity and regularity of the signal. A smaller frequency indicates that the spectral energy is concentrated in a few key frequencies. A larger value indicates a more dispersed spectral energy distribution and a relatively complex frequency composition of the breathing waveform.

[0034] Peak-to-peak ratio: (7) Where T is the time range of the entire respiratory segment. Xa(t) and Xc(t) are the main motion waveforms of the abdomen and chest, respectively; max and min represent the maximum and minimum values ​​within the time range T, respectively, and the difference between the maximum and minimum values ​​is the peak-to-peak value of the corresponding waveform; PTP X(t) The ratio of the peak-to-peak values ​​of the main motion waveforms of the lower abdomen and chest at time t is used to characterize the relative amplitude of the chest and abdomen motion.

[0035] Thoracic-abdominal phase difference: (8) Phase difference indicates the degree of synchronization and sequence of respiratory movements of the chest and abdomen in time. Indicates chest phase, This indicates the abdominal phase. The closer |Δφ| is to 0, the more synchronized the chest and abdominal movements are. The sign of Δφ indicates the relative leading or lagging direction of the two movements.

[0036] Amplitude relationship features include the root mean square amplitude ratio of the chest and abdomen and the peak-to-peak amplitude ratio of the chest and abdomen, used to reflect the relative motion amplitude of the chest and abdomen in a respiratory sample. Frequency relationship features include the dominant frequency of the chest, the dominant frequency of the abdomen, the difference between their dominant frequencies, and the spectral entropy, used to reflect the periodic rhythm and frequency domain stability of chest and abdominal respiratory movements. Phase and time delay relationship features include the maximum cross-correlation degree, the time delay corresponding to the maximum cross-correlation, the average phase difference, and the degree of phase difference fluctuation, used to reflect the synchronicity, sequence, and stability of the phase relationship of chest and abdominal movements. Correlation relationship features are used to reflect the degree to which the overall chest and abdominal waveforms change in the same or opposite directions.

[0037] In one specific implementation, the thoracic and abdominal respiratory dynamics features include nine features: the root mean square amplitude ratio of the thoracic and abdominal regions, the peak-to-peak ratio of the thoracic and abdominal regions, the difference in dominant frequencies of the thoracic and abdominal regions, the spectral entropy of the chest or abdomen, the maximum cross-correlation degree, the time delay corresponding to the maximum cross-correlation, the average phase difference, the standard deviation of the phase difference, and the correlation coefficient. Figure 5 The root mean square amplitude ratio, peak-to-peak ratio, spectral entropy, and phase difference shown are representative examples. These nine features are standardized according to the mean and standard deviation of the training data to form the inputs to the multilayer perceptron branches. All of these features can be used, or multiple features can be selected and combined based on data quality and specific application scenarios.

[0038] In step S5, a dual-branch feature extraction model is constructed, consisting of a one-dimensional convolutional neural network (CNN) branch and a multilayer perceptron (MLP) branch. The model structure is as follows: Figure 6 As shown. Figure 6The chest and abdominal optical motion capture signals shown are used to simplify the representation of the main chest and abdominal motion waveforms obtained after missing value processing, detrending, filtering, standardization, and projection onto the main motion direction. A one-dimensional convolutional neural network branch takes the aforementioned dual-channel fixed-length time sequence as input to learn peaks, troughs, local slopes, inspiratory-expiratory transitions, waveform irregularities, and local morphology between multiple respiratory cycles.

[0039] The one-dimensional convolutional neural network branch includes a first one-dimensional convolutional block, a second one-dimensional convolutional block, and a global pooling layer connected sequentially. Each convolutional block includes one-dimensional convolution, batch normalization, and ReLU nonlinear activation. A max-pooling layer is applied after the second one-dimensional convolutional block to compress the temporal dimension and enhance the translation invariance of local features. In one specific implementation, the first one-dimensional convolutional block outputs 64 feature channels, and the second one-dimensional convolutional block outputs 128 feature channels. After adaptive global average pooling, 128-dimensional deep temporal features are obtained. The kernel size, number of channels, and pooling parameters can be adjusted according to the sampling frequency and the length of the input sequence.

[0040] The multilayer perceptron branch uses standardized thoracic and abdominal respiratory dynamics as input to encode the thoracic-abdominal amplitude ratio, energy relationship, frequency characteristics, phase difference, time delay, and correlation. For example... Figure 6 As shown, in one specific embodiment, the multilayer perceptron branch sequentially includes a fully connected layer, a ReLU nonlinear activation layer, a random deactivation layer, and an MLP feature encoding layer, which encodes the input respiratory dynamics features into a 64-dimensional respiratory dynamics feature representation.

[0041] In step S6, the deep temporal features output by the one-dimensional convolutional neural network are fused with the respiratory dynamics feature representation output by the multilayer perceptron. In one specific embodiment, the 128-dimensional deep temporal features and the 64-dimensional respiratory dynamics feature representation are concatenated according to feature dimensions to obtain a 192-dimensional fused feature. After the fused feature is randomly deactivated and passed through a fully connected classification layer, it outputs classification scores for three categories: thoracic breathing, abdominal breathing, and thoracoabdominal breathing, and the breathing pattern is determined based on the classification scores.

[0042] To train and validate the dual-branch classification model, the data are grouped according to the subject's identity, and a hierarchical grouping method is used to construct training, validation, and test sets. This ensures that all samples from the same subject appear in only one subset of the data, preventing waveform features from the same subject from simultaneously entering both training and test data. In one specific implementation, a four-fold independent cross-validation is used. In each fold, test subjects are first identified, and then the training and validation sets are divided from the remaining subjects, ensuring that each subset contains samples of thoracic breathing, abdominal breathing, and combined thoracic and abdominal breathing.

[0043] During the training process of each fold, the model parameters are updated using the training set of that fold, and the model performance under different training iterations is evaluated using a validation set that is independent of both the training and test sets. The optimal model parameters for that fold are selected based on the macro-average F1 score of the validation set. The validation set is only used for model selection and early stopping and does not participate in the calculation of the final test metric.

[0044] After model training is complete, the network parameters, standardized parameters of the dual-channel respiratory timing, standardized parameters of the thoracic and abdominal respiratory dynamics features, feature names, and category names are saved. During model inference, the same missing data processing, detrending, filtering, respiratory cycle segmentation, resampling, standardization, main motion direction extraction, direction consistency correction, and respiratory dynamics feature calculation are performed on new 3D motion data as in the training phase. Then, the dual-channel main motion waveforms and thoracic and abdominal respiratory dynamics features are input into the saved bi-branch classification model to obtain the corresponding breathing patterns.

[0045] Figure 7 The figure shows a representative set of training curves for the first fold of the four-fold cross-validation. The first fold validation set contains 1461 breathing samples. The figure shows the changes in the validation set macro-average F1 and balance accuracy with the number of training iterations. Notably, the validation set macro-average F1 reaches 0.951 in the fifth training iteration, therefore the model parameters corresponding to the fifth training iteration are determined as the optimal model parameters for the first fold.

[0046] The first test set contains 624 breath samples, and the resulting macro-mean F1 score is 0.842. The validation set is used to determine the optimal number of training rounds, while the test set is used to evaluate the model's generalization ability to unseen subjects; these two sets correspond to different datasets and evaluation phases.

[0047] The remaining three folds followed the same model training, validation, and testing procedures as the first fold. The four test subject groups did not overlap, and each of the 23 subjects was tested only in one fold; each overlapping window of respiratory samples generated only one out-of-fold test prediction that was not involved in the corresponding model training and model selection.

[0048] The out-of-fold prediction results of the four folds corresponding to the test set were summarized, and the final prediction results for a total of 6273 respiratory samples were obtained, forming... Figure 8 The confusion matrix is ​​shown. The sample numbers for abdominal breathing, thoracic breathing, and combined thoracic and abdominal breathing are 3799, 661, and 1813, respectively, with correctly identified samples of 3128, 572, and 1425, corresponding to recall rates of approximately 82.3%, 86.5%, and 78.6%. Based on the aggregated results, the overall accuracy is approximately 81.7%, the balanced accuracy is approximately 82.5%, and the F1 score is approximately 79.8%.

[0049] This application also provides a respiratory pattern classification system corresponding to the aforementioned respiratory pattern classification method. For example... Figure 9 As shown, the respiratory pattern classification system includes a data acquisition module, a preprocessing module, a main motion waveform extraction module, a respiratory dynamics feature calculation module, a dual-branch feature extraction module, and a fusion classification module.

[0050] The data acquisition module is used to collect the three-dimensional coordinate time-series data, timestamps, and corresponding breathing task tags of the chest and abdominal marker sets during human breathing. Based on the chest or abdominal region where each motion marker is located, the three-dimensional coordinate time-series data is organized into a multi-channel breathing motion sequence.

[0051] The preprocessing module is connected to the data acquisition module and is used to perform missing data imputation, trend term removal, filtering, respiratory cycle segmentation, fixed-length resampling and standardization on the multi-channel respiratory motion sequence to obtain a preprocessed fixed-length multi-channel respiratory motion sequence.

[0052] The main motion waveform extraction module is connected to the preprocessing module. It is used to construct chest coordinate matrices and abdominal coordinate matrices based on the three-dimensional coordinate time series of chest marker set and abdominal marker set, respectively. Principal component analysis is performed on the chest coordinate matrix and abdominal coordinate matrix to determine the main motion direction of chest and abdomen, and the direction consistency correction is performed on the main motion direction of chest and abdomen. The corresponding three-dimensional coordinate time series is projected onto the corrected main motion direction to obtain the main motion waveform of chest and abdomen.

[0053] The respiratory dynamics feature calculation module is connected to the main motion waveform extraction module and is used to calculate the thoracic and abdominal respiratory dynamics features that characterize the amplitude relationship, frequency relationship, phase relationship, time delay relationship and correlation relationship of the chest and abdomen based on the main motion waveform of the chest and abdomen.

[0054] The dual-branch feature extraction module includes a one-dimensional convolutional neural network feature extraction unit and a multilayer perceptron feature encoding unit. The one-dimensional convolutional neural network feature extraction unit is connected to the main motion waveform extraction module and is used to receive a dual-channel respiratory time series composed of the main chest motion waveform and the main abdominal motion waveform, and extract deep temporal features to characterize the local morphology of the respiratory waveform, the inhalation-exhalation transition process, and the changing patterns of multiple respiratory cycles. The multilayer perceptron feature encoding unit is connected to the respiratory dynamics feature calculation module and is used to receive the chest and abdominal respiratory dynamics features and obtain respiratory dynamics feature representations to characterize the amplitude, frequency, phase, time delay, and correlation of the chest and abdomen.

[0055] The fusion classification module is connected to the one-dimensional convolutional neural network feature extraction unit and the multilayer perceptron feature encoding unit, respectively. It is used to concatenate or weightedly fuse the deep temporal features and the respiratory dynamics feature representations, and input the fused features into the classifier to output one of the following breathing modes: thoracic breathing, abdominal breathing, and thoracoabdominal breathing.

[0056] In one implementation, the fusion classification module is further configured to output a classification score or classification probability corresponding to each candidate breathing pattern, and determine the subject's current breathing pattern based on the classification score or classification probability. The classification results can be displayed on a display device or transmitted to a breathing training device, assisted breathing device, or other breathing monitoring terminal via a communication interface.

[0057] This application also provides an electronic device corresponding to the aforementioned respiratory pattern classification method, for executing the aforementioned respiratory pattern classification method. The electronic device may include a processor, a memory, a communication interface, an output interface, and a bus, wherein the processor, memory, communication interface, and output interface are connected via the bus. The memory stores a computer program, model parameters, and standardized parameters that can run on the processor. When the processor executes the computer program, it implements the respiratory pattern classification method described in any of the foregoing embodiments. The communication interface is used to receive three-dimensional motion capture data, and the output interface is used to display or transmit the respiratory pattern classification results.

[0058] The memory may include high-speed random access memory or non-volatile memory for storing program instructions and intermediate data; the processor may be a general-purpose central processing unit, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic devices. During the operation of the method in this application, the processor loads and executes the program instructions stored in the memory, and through executing steps S1 to S6, achieves the acquisition of three-dimensional chest and abdominal motion data, preprocessing of respiratory motion sequences, extraction of main chest and abdominal motion waveforms, calculation of respiratory dynamic features, extraction of dual-branch features, and fusion classification.

[0059] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, can implement all the steps of the breathing pattern classification method described in the foregoing embodiments. The computer-readable storage medium may include, but is not limited to, phase-change memory, flash memory, read-only memory, electrically erasable programmable read-only memory, magnetic disk, optical disk, or other media capable of storing computer programs.

[0060] It should be noted that the algorithms and displays provided herein are not inherently related to any pre-existing computer, virtual system, or other device. Various general-purpose systems can also be used with the technical solutions based on this application. Based on the above description, the structure required to construct such a system is obvious. Furthermore, this application is not targeted at any pre-existing programming language, and the technical content described herein can be implemented using various programming languages.

[0061] Several specific details are set forth in the specification provided herein. However, it should be understood that embodiments of this application may be implemented without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail to avoid obscuring the understanding of the technical solutions of this application.

[0062] Similarly, it should be understood that, in order to simplify this application and aid in understanding one or more aspects of the invention, various technical features of this application are sometimes combined into a single embodiment, drawing, or description thereof in the above description of exemplary embodiments. However, this disclosure should not be construed as requiring the claimed application to have more technical features than expressly recited in each claim. Each claim itself constitutes an independent embodiment of this application.

[0063] Those skilled in the art will understand that the modules in the embodiments can be adapted and placed in one or more devices different from the embodiments described above; multiple modules can also be combined into one module, or one module can be divided into multiple sub-modules. Except for mutually exclusive technical features, the technical features and method steps disclosed in this specification, claims, and drawings can be combined in any suitable manner.

[0064] The various modules of this application can be implemented in hardware, in software modules running on one or more processors, or in a combination of hardware and software. This application can also be implemented as a device program, system program, or computer program product for performing some or all of the steps of the method.

[0065] It should be noted that the above embodiments are illustrative of this application and not limiting of it. Those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, the term "comprising" does not exclude the presence of elements or steps not listed, and the term "a" preceding an element does not exclude the presence of a plurality of said elements. The expressions "first," "second," etc., are only used to distinguish corresponding technical features and do not indicate their order or importance.

[0066] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations, substitutions, or equivalent solutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A respiratory pattern classification method based on the fusion of directional temporal and dynamic characteristics, characterized in that, include: Step S1: Collect three-dimensional coordinate time-series data of chest marker set and abdominal marker set during human respiration. Based on the chest region or abdominal region where each motion marker is located, organize the three-dimensional coordinate time-series data into a multi-channel respiratory motion sequence. Step S2: Preprocess the multi-channel respiratory motion sequence, including missing data imputation, trend term removal, filtering, respiratory cycle segmentation, fixed-length resampling, and standardization. Step S3: Construct chest coordinate matrices and abdominal coordinate matrices based on the three-dimensional coordinate time series data of the chest marker set and abdominal marker set, respectively. Perform principal component analysis on the chest coordinate matrix and abdominal coordinate matrix to determine the main motion direction of the chest and the main motion direction of the abdomen. Project the corresponding three-dimensional coordinate time series data onto the determined main motion direction to obtain the main motion waveform of the chest and the main motion waveform of the abdomen. Step S4: Based on the main chest motion waveform and the main abdominal motion waveform, calculate the thoracic and abdominal respiratory dynamics characteristics used to characterize the amplitude relationship, frequency relationship, phase relationship, time delay relationship and correlation relationship between the chest and abdomen; Step S5: Combine the main chest motion waveform and the main abdominal motion waveform into a dual-channel respiratory time sequence, and input it into a one-dimensional convolutional neural network to obtain deep temporal features; The chest and abdominal respiratory dynamics features are input into a multilayer sensor to obtain a respiratory dynamics feature representation. Step S6: Fuse the deep temporal features and the respiratory dynamics features, input the fused features into the classifier, and output one of the following breathing modes: thoracic breathing, abdominal breathing, and thoracoabdominal breathing.

2. The method according to claim 1, characterized in that, Step S1 includes: At least one chest movement marker is set in the subject's thoracic region, and at least one abdominal movement marker is set in the subject's abdominal wall region; Using a three-dimensional motion capture device, the coordinate values ​​of each motion marker point along the three spatial coordinate axes are synchronously collected at a preset sampling frequency; The collected three-dimensional coordinate time series data are grouped according to the chest region or abdominal region to which each motion marker belongs, forming a chest three-dimensional motion sequence and an abdominal three-dimensional motion sequence. Simultaneously record the breathing tasks performed by the subjects and the corresponding breathing pattern labels.

3. The method according to claim 1, characterized in that, Step S2 includes: Missing coordinate values ​​in the multi-channel respiratory motion sequence are detected, and interpolation methods are used to complete the missing coordinate values; Remove trend terms caused by the initial position of motion markers, overall human displacement, or slow posture changes from the time series of each channel coordinate; Savitzky–Golay filters are used to suppress measurement noise; The boundaries of a complete respiratory cycle are determined based on the peak and trough values ​​in the respiratory motion sequence; A preset number of consecutive complete respiratory cycles are divided into a respiratory sample, and respiratory samples of different time lengths are resampled to a uniform length. The statistical information from the training data is used to standardize the data from each channel.

4. The method according to claim 1, characterized in that, Step S3 includes: The three-dimensional coordinate temporal sequence of the chest marker set and the abdominal marker set are centered respectively; Based on the time series of the three-dimensional coordinates of the chest and abdomen after centralization, the motion changes in each spatial direction are analyzed, and the direction with the greatest degree of motion change is determined as the main motion direction of the corresponding region. The three-dimensional motion of the chest marker set is projected onto the main motion direction of the chest to obtain the main motion waveform of the chest; The three-dimensional motion of the set of abdominal markers is projected onto the main motion direction of the abdomen to obtain the waveform of the main motion of the abdomen.

5. The method according to claim 1, characterized in that, The thoracic and abdominal respiratory dynamics characteristics in step S4 include one or more of the following: The root mean square amplitude ratio of the main motion waveform of the chest to the main motion waveform of the abdomen; Peak-to-peak ratio of the main chest movement waveform to the main abdominal movement waveform; The difference in dominant frequency between the main motion waveforms of the chest and abdomen; The spectral entropy of the main motion waveforms of the chest and abdomen; The maximum cross-correlation between the main chest motion waveform and the main abdominal motion waveform; The phase difference between the main motion waveform of the chest and the main motion waveform of the abdomen; The time delay corresponding to the maximum cross-correlation between the main motion waveforms of the chest and abdomen; Standard deviation of the phase difference between the main chest motion waveform and the main abdominal motion waveform; The correlation coefficient between the main motion waveform of the chest and the main motion waveform of the abdomen.

6. The method according to claim 1, characterized in that, Step S5 includes: The main chest movement waveform and the main abdominal movement waveform are each used as an input channel to form a dual-channel breathing sequence; Using a one-dimensional convolutional neural network including multiple one-dimensional convolutional blocks and global pooling layers, feature extraction is performed on the dual-channel respiratory time series to obtain deep time series features that characterize the local morphology of the respiratory waveform, the inhalation-exhalation transition process, and the variation patterns of multiple respiratory cycles. The thoracic and abdominal respiratory dynamics features are encoded using a multilayer perceptron including at least one fully connected layer and a nonlinear activation layer to obtain a respiratory dynamics feature representation that characterizes the amplitude, frequency, phase, time delay, and correlation of the thoracic and abdominal regions.

7. The method according to claim 1, characterized in that, Step S6 includes: The deep temporal features output by the one-dimensional convolutional neural network are concatenated or weighted and fused with the respiratory dynamics features output by the multilayer perceptron. The fused features are input into a fully connected classification layer to obtain the classification results corresponding to each candidate breathing pattern. Based on the classification results of each candidate breathing pattern, the subject's current breathing pattern was determined to be thoracic breathing, abdominal breathing, or combined thoracic and abdominal breathing.

8. A respiratory pattern classification system based on the fusion of directional temporal and dynamic characteristics, characterized in that, include: The data acquisition module is used to acquire three-dimensional coordinate time-series data of chest and abdominal marker sets and form a multi-channel respiratory motion sequence. The preprocessing module is used to perform missing data imputation, trend term removal, filtering, respiratory cycle segmentation, fixed-length resampling, and standardization on the multi-channel respiratory motion sequence. The main motion waveform extraction module is used to construct chest coordinate matrices and abdominal coordinate matrices based on the three-dimensional coordinate time series data of chest marker set and abdominal marker set, respectively. Principal component analysis is performed on the chest coordinate matrix and abdominal coordinate matrix to determine the main motion direction of chest and abdomen. The main motion direction of chest and abdomen is corrected for direction consistency, and the corresponding three-dimensional coordinate time series data is projected onto the corrected main motion direction to obtain the main motion waveform of chest and abdomen. The respiratory dynamics feature calculation module is used to calculate the thoracic and abdominal respiratory dynamics features that characterize the amplitude relationship, frequency relationship, phase relationship, time delay relationship and correlation relationship between the chest and abdomen, based on the main chest motion waveform and the main abdominal motion waveform. The dual-branch feature extraction module includes a one-dimensional convolutional neural network feature extraction unit and a multilayer perceptron feature encoding unit. The one-dimensional convolutional neural network feature extraction unit is used to extract deep temporal features from the dual-channel respiratory time series composed of the main chest motion waveform and the main abdominal motion waveform. The multilayer perceptron feature encoding unit is used to extract respiratory dynamic feature representations from the chest and abdominal respiratory dynamic features. The fusion classification module is used to fuse the deep temporal features and the respiratory dynamics features, and output one of the following breathing modes: thoracic breathing, abdominal breathing, and thoracoabdominal breathing.

9. An electronic device comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor executes the computer program, it implements the respiratory pattern classification method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the respiratory pattern classification method according to any one of claims 1 to 7.

Images