Human-machine asynchronization recognition method based on multi-task learning and class activation map feedback

Abstract

The application provides a man-machine asynchronization recognition method based on multi-task learning and class activation map feedback, a deep learning model is trained in a parameter hard sharing manner by using multi-task learning, and a visualized explanation of an output result of the trained deep learning model is obtained in a class activation mapping manner; meanwhile, a feature region is set according to a class activation map activation domain of each man-machine asynchronization type. Then, an original breathing signal actually collected is input into the trained deep learning model to obtain a recognition result of the current breathing signal; finally, the recognition result is corrected according to the feature region set for the man-machine asynchronization type. The application trains only one network model in a multi-task combination parameter hard sharing manner, so that a plurality of man-machine asynchronization types can be output simultaneously by one forward calculation, and the recognition efficiency of the existing method is improved. The self-corrected classification result based on the class activation map feedback makes the method have high accuracy and interpretability.

Description

TECHNICAL FIELD

[0001] The application relates to a multi-human-asynchronous-type recognition method in mechanical ventilation based on multi-task learning and class activation map feedback. BACKGROUND

[0002] With the development and perfection of deep learning theory, neural networks are widely used in the medical field. In the field of automatic recognition of human-machine asynchrony in mechanical ventilation, some scholars have proposed human-machine asynchrony recognition methods based on recurrent neural networks, one-dimensional convolutional neural networks and two-dimensional convolutional neural networks. However, in this field, most of the current neural network-based detection methods are single-task learning binary classification networks, which can only recognize a certain type of human-machine asynchrony. If the method module is embedded into the analysis platform to recognize multiple types of human-machine asynchrony, the algorithm can only be integrated in the later stage, which undoubtedly increases the time cost of the method in the forward calculation stage. Although certain achievements have been made in the research of binary classification networks, the existing methods still have a lot of room for improvement in performance when they are truly applied to clinical practice. At the same time, class activation maps or gradient class activation maps are introduced into neural networks to visualize the classification decision process and enhance the explainability of the method in clinical practice. However, the existing methods only use visualized class activation maps as the basis for network output classification results, ignoring the possibility of further improving the network recognition accuracy by using class activation maps as the positioning function of the feature region of human-machine asynchrony waveforms. SUMMARY

[0003] The purpose of the application is to integrate multiple binary classification networks end-to-end to shorten the time consumed by the network in the forward calculation stage and to reduce the false positive rate of classification by correcting the network classification results through class activation maps, and to provide a multi-task learning and class activation map feedback method for recognizing multiple types of human-machine asynchrony in mechanical ventilation. The multi-task learning refers to a one-dimensional convolutional neural network structure main body learning multiple binary classification modes in a parameter hard sharing manner.

[0004] The purpose of the application is achieved as follows:

[0005] A human-machine asynchrony recognition method based on multi-task learning and class activation map feedback, the method comprising:

[0006] A deep learning model training for human-machine asynchrony type recognition and a multiple human-machine asynchrony type class activation map feature region limitation rule acquisition step:

[0007] Collect labeled respiratory signals containing multiple human-machine asynchrony types and other types, the respiratory signals being composed of one flow rate and one pressure data. Standardize each respiratory signal, and then process all samples in the data set into equal-length sequences through resampling;

[0008] According to the category to which the collected data belongs, a plurality of sets of training labels are constructed, taking the respiratory signal as input and the category as output, and a deep learning model for identifying the human-machine asynchronization type is trained in a multi-task learning manner;

[0009] Through class activation mapping, the visualized explanation of the output result of each sample by the trained deep learning model is obtained; and the feature region is limited according to the class activation domain and the standard feature region of all samples. The feature region limiting rule is the threshold of the class activation domain.

[0010] According to the trained deep learning model and the feature region limiting rule, the human-machine asynchronization type identification step is performed.

[0011] After the original respiratory signal is preprocessed and input into the trained deep learning model, the human-machine asynchronization identification result of the current respiratory signal can be obtained; at the same time, the class activation map of the identification type in the identification result is obtained through class activation mapping; if the activation domain of the class activation map of the identification type meets the feature region limiting rule of the corresponding human-machine asynchronization type, it is judged as the corresponding human-machine asynchronization type, otherwise, it is judged as other types.

[0012] Further, in an undersampling manner, the data set is balanced by randomly extracting a number of non-human-machine asynchronization samples equivalent to the number of human-machine asynchronization samples; all samples are first subjected to z-score standardization:

[0013]

[0014] wherein x i is the signal amplitude of the i-th sampling point in each independent sample, L is the signal length of each independent sample, μ and σ are the mean and standard deviation of the signal amplitude of each independent sample.

[0015] Further, the data of each sample is resampled to 300 lengths, so that all sample sequences are equal in length.

[0016] Further, the multi-task learning fixes the feature extraction capabilities of different tasks in one deep learning model structure through parameter hard sharing, and the deep learning model structure includes a plurality of one-dimensional convolutional neural networks, a feature fusion layer, a global average pooling layer and a plurality of classification heads; the one-dimensional convolutional neural network is a double-channel parallel structure, which is respectively used to extract pressure and flow rate waveform features, and then the two feature information is added through the feature fusion layer Add, and then classified through the plurality of classification heads after the global average pooling layer.

[0017] Further, the one-dimensional convolutional neural network includes a one-dimensional convolutional layer, a batch normalization layer and a one-dimensional maximum pooling layer.

[0018] Further, the end condition of the deep learning model training is that the loss function of the deep learning model converges or the number of iterations of the deep learning model reaches a preset number of iterations.

[0019] Further, the training label is constructed by using one-hot encoding.

[0020] Further, the trained deep learning model is obtained by class activation mapping, and the visualized explanation of the output result of each sample of the trained deep learning model is obtained, specifically as follows:

[0021] a. After the feature map output by the sample in the last convolutional layer of the deep learning model is subjected to a feature fusion layer Add, the two extracted feature maps after convolution are combined in an additive manner: wherein represents a feature map of pressure data at time t activated by a filter unit k of the last convolutional layer, represents a feature map of flow rate data at time t activated by a filter unit k;

[0022] b. After a global average pooling operation, the feature dimension output by the feature fusion layer Add is reduced to 1:

[0023] F k =∑ t f k (t)

[0024] wherein, F k represents the output of the global average pooling layer, t represents the time dimension, and k represents the filter unit.

[0025] c. For a certain class c, the classification head h first calculates the input S c of the class before the classification result is output.

[0026]

[0027] wherein, represents the weight of the feature map corresponding to each filter unit k in the last one-dimensional convolutional neural network, i.e., the output of the feature map corresponding to each filter unit k after global average pooling, which represents the degree of contribution of the classification head h to the classification of the input respiratory signal as class c;

[0028] wherein, represents the class activation map corresponding to the output class c of the classification head h, which directly represents the importance of the classification of the input respiratory signal as class c by the classification head h after the convolutional features are activated by the filter unit k.

[0029] d. The class activation map is up-sampled to the dimension of the original input respiratory signal, and is mapped to the original respiratory data in the form of a heat map to obtain a visualized explanation of the deep learning model in the classification decision process.

[0030] Further, the feature region is set according to the activation domain of the class activation map of all samples and a standard feature region, and the setting rule is specifically:

[0031] The activation domain of the class activation map of each sample is normalized by maximum and minimum, that is,

[0032]

[0033] wherein X is the activation domain of the class activation map, X min is the minimum value in the activation domain, X max is the maximum value in the activation domain, and X norm is the normalized activation domain.

[0034] An adjustment threshold N is set until the connected domain greater than N in the activation domain of the class activation map of all samples is closest to the standard feature region of the corresponding sample class activation map. The standard feature region is obtained by professional labeling.

[0035] Compared with the prior art, the present application has the beneficial effects that:

[0036] The present application is a multi-person-machine asynchronization type recognition method in mechanical ventilation based on multi-task learning and class activation map feedback. Through the multi-task learning combined with the model parameter hard sharing mode, one model body has the ability to recognize multiple person-machine asynchronization types. Moreover, the end-to-end integrated multiple binary classification networks greatly shorten the time consumed by the network in the forward calculation stage. At the same time, the multi-task learning greatly reduces the risk of training overfitting. Because the more recognition tasks learned by one network body at the same time, the more difficult it is to find a representation containing all tasks, so the possibility of overfitting the original task is smaller. In addition, the important feature region in the classification decision process is mapped by visualizing the class activation map, which enhances the explainability of the method. The class activation map is combined with the limitation rule of the corresponding person-machine asynchronization region feature to reduce the false positive rate of the model and further improve the accuracy of the model.

[0037] The application is a multi-human-machine asynchronization type recognition method based on multi-task learning and class activation map feedback in mechanical ventilation, which is not limited to the detection of a certain type of human-machine asynchronization, and has universal applicability for the recognition and analysis of other types of human-machine asynchronization. Compared with the prior art, the human-machine asynchronization recognition method proposed in the application can output multiple human-machine asynchronization types recognized at the same time under the condition of training only one network model, which reduces the time complexity of the method in the training and testing stages. And the self-correcting classification result technology based on class activation map further improves the accuracy of the existing method. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 The application is a multi-human-machine asynchronization type recognition method based on multi-task learning and class activation map feedback in mechanical ventilation, which is not limited to the detection of a certain type of human-machine asynchronization, and has universal applicability for the recognition and analysis of other types of human-machine asynchronization. Compared with the prior art, the human-machine asynchronization recognition method proposed in the application can output multiple human-machine asynchronization types recognized at the same time under the condition of training only one network model, which reduces the time complexity of the method in the training and testing stages. And the self-correcting classification result technology based on class activation map further improves the accuracy of the existing method.

[0039] Figure 2 The application is a multi-human-machine asynchronization type recognition method based on multi-task learning and class activation map feedback in mechanical ventilation, which is not limited to the detection of a certain type of human-machine asynchronization, and has universal applicability for the recognition and analysis of other types of human-machine asynchronization. Compared with the prior art, the human-machine asynchronization recognition method proposed in the application can output multiple human-machine asynchronization types recognized at the same time under the condition of training only one network model, which reduces the time complexity of the method in the training and testing stages. And the self-correcting classification result technology based on class activation map further improves the accuracy of the existing method.

[0040] Figure 3 The application is a multi-human-machine asynchronization type recognition method based on multi-task learning and class activation map feedback in mechanical ventilation, which is not limited to the detection of a certain type of human-machine asynchronization, and has universal applicability for the recognition and analysis of other types of human-machine asynchronization. Compared with the prior art, the human-machine asynchronization recognition method proposed in the application can output multiple human-machine asynchronization types recognized at the same time under the condition of training only one network model, which reduces the time complexity of the method in the training and testing stages. And the self-correcting classification result technology based on class activation map further improves the accuracy of the existing method.

[0041] Figure 4 The application is a multi-human-machine asynchronization type recognition method based on multi-task learning and class activation map feedback in mechanical ventilation, which is not limited to the detection of a certain type of human-machine asynchronization, and has universal applicability for the recognition and analysis of other types of human-machine asynchronization. Compared with the prior art, the human-machine asynchronization recognition method proposed in the application can output multiple human-machine asynchronization types recognized at the same time under the condition of training only one network model, which reduces the time complexity of the method in the training and testing stages. And the self-correcting classification result technology based on class activation map further improves the accuracy of the existing method.

[0042] Figure 5 The application is a multi-human-machine asynchronization type recognition method based on multi-task learning and class activation map feedback in mechanical ventilation, which is not limited to the detection of a certain type of human-machine asynchronization, and has universal applicability for the recognition and analysis of other types of human-machine asynchronization. Compared with the prior art, the human-machine asynchronization recognition method proposed in the application can output multiple human-machine asynchronization types recognized at the same time under the condition of training only one network model, which reduces the time complexity of the method in the training and testing stages. And the self-correcting classification result technology based on class activation map further improves the accuracy of the existing method.

[0043] Figure 6 The application is a multi-human-machine asynchronization type recognition method based on multi-task learning and class activation map feedback in mechanical ventilation, which is not limited to the detection of a certain type of human-machine asynchronization, and has universal applicability for the recognition and analysis of other types of human-machine asynchronization. Compared with the prior art, the human-machine asynchronization recognition method proposed in the application can output multiple human-machine asynchronization types recognized at the same time under the condition of training only one network model, which reduces the time complexity of the method in the training and testing stages. And the self-correcting classification result technology based on class activation map further improves the accuracy of the existing method. DETAILED DESCRIPTION

[0044] The application is a multi-human-machine asynchronization type recognition method based on multi-task learning and class activation map feedback in mechanical ventilation, which is not limited to the detection of a certain type of human-machine asynchronization, and has universal applicability for the recognition and analysis of other types of human-machine asynchronization. Compared with the prior art, the human-machine asynchronization recognition method proposed in the application can output multiple human-machine asynchronization types recognized at the same time under the condition of training only one network model, which reduces the time complexity of the method in the training and testing stages. And the self-correcting classification result technology based on class activation map further improves the accuracy of the existing method.

[0045] The application is a multi-human-machine asynchronization type recognition method based on multi-task learning and class activation map feedback in mechanical ventilation, which is not limited to the detection of a certain type of human-machine asynchronization, and has universal applicability for the recognition and analysis of other types of human-machine asynchronization. Compared with the prior art, the human-machine asynchronization recognition method proposed in the application can output multiple human-machine asynchronization types recognized at the same time under the condition of training only one network model, which reduces the time complexity of the method in the training and testing stages. And the self-correcting classification result technology based on class activation map further improves the accuracy of the existing method.

[0046] Reference Figure 1The application discloses a multi-person-machine-asynchronous-type identification method based on multi-task learning and class activation map feedback in mechanical ventilation.

[0047] S1, pretreatment;

[0048] The labeled respiratory signals containing multiple person-machine-asynchronous types and non-person-machine-asynchronous types are collected, each respiratory signal is subjected to data standardization, and then all samples in the data set are processed into equal-length sequences through resampling;

[0049] Specifically, the respiratory waveform data is labeled by professional doctors. Since the number of non-person-machine-asynchronous samples in the labeled data set is much larger than that of person-machine-asynchronous samples, the data set is balanced in sample quantity in an undersampling manner by randomly extracting non-person-machine-asynchronous samples approximately equal in quantity to person-machine-asynchronous samples; all samples are subjected to z-score standardization processing.

[0050]

[0051] Wherein, x i is the signal amplitude of the i-th sampling point in each independent sample, L is the signal length of each independent sample, and mu and sigma are the mean and standard deviation of the signal amplitude of each independent sample.

[0052] Since the one-dimensional convolutional neural network can only take a fixed tensor as input, the data of each sample is resampled to 300 lengths, so that all sample sequences are equal in length.

[0053] S2, a multi-layer deep learning model with a one-dimensional convolutional neural network as the core is constructed, and different tasks are fixed in one model structure through parameter hard sharing.

[0054] Preferably, a specific embodiment of this step is that the one breath signal is composed of one flow rate and one pressure data, a five-layer one-dimensional convolutional neural network is constructed in a double-channel parallel manner, and the flow rate and pressure dimension feature information extracted respectively is added through a feature fusion layer Add after feature extraction. Unlike common convolutional neural networks, a global average pooling layer is set after the convolution module composed of multiple one-dimensional convolutional neural networks instead of a traditional fully connected layer, with the purpose of reducing the parameter amount while extracting class activation maps for visual analysis of the output results. The last layer adopts multiple classification head fully connected layers with softmax as the activation function for binary classification. Table 1 lists the definitions of the detailed network parameters used in the embodiments.

[0055] Table 1 Detailed network parameter definitions used in the embodiments

[0056]

[0057]

[0058] S3, the preprocessed breath waveform data samples are divided into a training set, a validation set and a test set;

[0059] In this embodiment, samples of five types of labels, i.e., too long cycle, too short cycle, double triggering, ineffective inspiratory effort and other types, are obtained. According to the other type label samples composed of a single breath cycle, other type samples composed of two breath cycles are additionally constructed in a quantity corresponding to the number of human-machine asynchronization types, with the purpose of training the network to distinguish the waveform features formed by double triggering and two breath cycles. In this embodiment, a total of 7521 samples of too long cycle, 4407 samples of too short cycle, 4639 samples of double triggering, 7696 samples of ineffective inspiratory effort, 5000 samples of other type composed of a single breath cycle, and 4693 samples of other type composed of two breath cycles are obtained. Then, each type of sample is randomly divided into a training part and a test part according to a ratio of 9:1, and the training part is randomly divided into a training set and a validation set according to a ratio of 7:3. The training set is input into the deep learning model for training, and the validation set and the test set are used to optimize the model parameters and evaluate the classification performance of the model, respectively.

[0060] S4, according to the category to which the data belongs and the requirements of the identification task, a plurality of binary training labels are constructed and converted into one-hot encoding, and then the deep learning model is trained in a multi-task learning manner;

[0061] In specific embodiments, four binary classification heads are set. For a training sample, according to the plurality of training labels, the plurality of binary classification heads are trained in parallel, and the total loss function of the training is the sum of the loss functions of the plurality of binary classification heads, and the loss function of each classification head is a cross-entropy loss function:

[0062] L = ∑i ω i ×L i

[0063] wherein, L is the total loss function, i represents the corresponding task, i.e., the classification head, and ω represents the weight assigned to the corresponding task. L i is the cross-entropy loss function of the corresponding classification head. In this embodiment, the weight of all classification tasks is 1.

[0064] Table 2 shows the classification performance of the network model trained in the multi-task learning manner in the embodiment.

[0065] Table 2 Classification results of the network trained in the multi-task learning manner in the embodiment

[0066]

[0067]

[0068] In this embodiment, the parameter derivation is also based on the feature map output by the deep learning model, and the visualized explanation of the model in the classification decision process of each sample is obtained through the class activation mapping, as follows:

[0069] S5-1, for a respiratory signal, and are the outputs of the last convolution layer after parallel feature extraction of the sequence data; represents that the pressure data at time t is activated by filter unit k, represents that the flow rate data at time t is activated by filter unit k;

[0070] S5-2, after the feature fusion layer Add, the two features extracted after convolution are combined in the form of addition, i.e.,

[0071] S5-3, after the global average pooling operation, the feature dimension of the fusion layer output is reduced to 1;

[0072] F k =∑ t f k (t)

[0073] wherein, F k represents the output of the global average pooling layer, t represents the time dimension, and k represents the filter unit;

[0074] S5-4, for a certain class c, the classification head h first calculates the input S c of the class before the softmax classification result output.

[0075]

[0076] wherein, represents the weight of the feature map corresponding to each filter unit k in the last layer of the one-dimensional convolutional neural network, i.e., the contribution degree of the output of the feature map corresponding to each filter unit k after global average pooling to the classification head h for judging the input respiratory signal as the category c;

[0077] S5-5, F k =∑ t f k (t) is substituted into the classification score formula i.e.

[0078]

[0079] wherein, represents the class activation map corresponding to the output category c of the classification head h, which directly represents the importance of the activated feature after convolution by the filter unit k to the classification head h for judging the input respiratory signal as the category c.

[0080] S5-6, the class activation map is up-sampled to the dimension of the original input respiratory signal, and then is mapped to the original respiratory data in the form of a heat map, so that the visualized explanation of the classification decision process of the model can be obtained. Figure 3 、 4 , 5 and 6 respectively show the visualized effect diagram of the classification results of the output of the multiple binary classification heads of the model trained under the multi-task learning. In the diagram, the highlighted area is the feature area, which is the main basis for the model judgment. Through the visualization of the local feature, it can be explained that the model training is effective, and the learned feature is not false, which is convincing.

[0081] S6, further, the activation domain of the class activation map of all samples and the standard feature area set the limitation rule of the feature area, which is as follows:

[0082] S6-1, after obtaining the class response map, i.e., the class activation map, on the feature map of the last layer, the activation domain, i.e., the output value, of the class activation map is normalized by maximum and minimum, i.e.,

[0083]

[0084] wherein, X is the output value of the class activation map, X min is the minimum value in the output value, X max is the maximum value in the output value, and X norm is the normalized output value.

[0085] S6-2, set and adjust threshold N until all sample class activation map activation domains greater than N are closest to the standard feature area of the corresponding sample class activation map. The standard feature area is obtained by professional annotation. In this embodiment, the doctor specifies that the feature area of the long cycle type occurs from the beginning to the end of the inspiration phase, the feature area of the short cycle type occurs between the beginning of the expiration phase and 400 ms, the feature area of the double trigger type occurs between the expiration phase of the first breathing cycle and the end of the inspiration phase of the second breathing cycle, and the feature area of the ineffective inspiration effort type occurs between 400 ms after the beginning of the expiration phase and the end of the expiration phase.

[0086] N is usually greater than 0.5, and in this embodiment, is finally adjusted to 0.9, but in application, it can be appropriately adjusted according to actual situation analysis. The real-time acquired breathing signal is input into the trained deep learning model after being preprocessed by the method of step S1, the recognition result is obtained, and according to the class activation map output by the classification head, if the activation domain greater than 0.9 is in the corresponding feature area of the human-machine asynchronization type, it is determined that the human-machine asynchronization type is determined, otherwise it is determined as other types.

[0087] The present application is a kind of based on multi-task learning and class activation map feedback in mechanical ventilation in multiple human-machine asynchronization type identification method. In the embodiment, four kinds of human-machine asynchronization types of long cycle, short cycle, double trigger and ineffective inspiration effort are detected, recognized and visualized, but the present application is not limited to the detection of the above-mentioned types of human-machine asynchronization, and has universal applicability to the recognition and analysis of other types of human-machine asynchronization.

[0088] The human-machine asynchronization recognition method proposed in the present application can train a model with high detection precision based on multi-task learning under the condition of training only one network model. Once the forward calculation is completed, the multiple human-machine asynchronization types can be output simultaneously, avoiding the need to integrate multiple binary classification networks for multi-task recognition in the later stage, greatly shortening the time consumed by the network in the detection stage. At the same time, the global average pooling is introduced into the network structure, which not only reduces the parameter amount of the model, but also enhances the explainability of the method by visualizing the classification result and analyzing the classification result. When multiple classification heads obtain the recognition types at the same time, the class activation map is combined with the limitation rules of the corresponding human-machine asynchronization area characteristics to correct the recognition result, which can reduce the false positive rate of the model and further improve the recognition accuracy of the model.

[0089] In the above embodiments, the present application is described only as an example, but those skilled in the art can make various modifications to the present application without departing from the spirit and scope of the present application after reading this patent application.

Claims

1. A method for recognizing human-computer asynchrony based on multi-task learning and class activation graph feedback, characterized in that, The method includes: Steps for training deep learning models for human-machine asynchrony type recognition and obtaining class activation map feature region constraint rules for various human-machine asynchrony types: The collected respiratory signals include various types of human-machine asynchrony and non-human-machine asynchrony. The respiratory signals consist of one flow rate and one pressure data. Each respiratory signal is first standardized, and then resampled to process all samples in the dataset into a sequence of equal length. Based on the category of the collected data, multiple sets of training labels are constructed. With respiratory signals as input and categories as output, a deep learning model for human-machine asynchronous type recognition is trained using a multi-task learning approach. By using class activation mapping, a visual interpretation of the output results of the trained deep learning model for each sample is obtained; at the same time, the limitation rules of the feature regions are set according to the activation domains of the class activation maps and the standard feature regions of all samples; the limitation rules of the feature regions are the thresholds of the activation domains of the class activation maps; Steps for identifying human-machine asynchrony types based on a trained deep learning model and the constraints of feature regions: After preprocessing the raw respiratory signal, it is input into a trained deep learning model to obtain the human-machine asynchrony recognition result for the current respiratory signal. Simultaneously, a class activation map of the recognition type is obtained through class activation mapping. If the activation domain of the class activation map of the recognition type satisfies the feature region constraint rules, it is judged as the corresponding human-machine asynchrony type; otherwise, it is judged as another type. The specific rules for setting the feature region constraint based on the activation domain of the class activation map and the standard feature region of all samples are as follows: Perform min-max normalization on the activation region of the class activation map for each sample, i.e. Where X is the activation domain of the class activation graph, X min It is the minimum value in the activation domain, X max It is the maximum value in the activation domain, X norm It is the normalized activation domain; A threshold N is set until the connected regions with a continuous value greater than N in the activation domain of the class activation map of all samples are closest to the standard feature region of the corresponding sample's class activation map. The obtained threshold N is the limitation rule for the feature region. The standard feature region is obtained by professional personnel. The standard feature region for the excessively long cycle type occurs from the beginning to the end of the inspiratory phase, the standard feature region for the excessively short cycle type occurs from the beginning to 400ms of the expiratory phase, the standard feature region for the double-trigger type occurs from the beginning of the expiratory phase of the first respiratory cycle to the end of the inspiratory phase of the second respiratory cycle, and the standard feature region for the ineffective inspiratory effort type occurs from 400ms after the beginning of the expiratory phase to the end of the expiratory phase.

2. The human-computer asynchronous recognition method based on multi-task learning and class activation graph feedback according to claim 1, characterized in that: By undersampling, the dataset is balanced by randomly selecting a number of non-human-machine asynchronous samples that are roughly equivalent in number to the number of human-machine asynchronous samples; all samples are then z-score standardized. Where, x i is the signal amplitude at the i-th sampling point in each independent sample, L is the signal length of each independent sample, and μ and σ are the mean and standard deviation of the signal amplitude of each independent sample, respectively.

3. The human-computer asynchronous recognition method based on multi-task learning and class activation graph feedback according to claim 2, characterized in that: The data for each sample was resampled to a length of 300, so that all sample sequences were of equal length.

4. The human-computer asynchronous recognition method based on multi-task learning and class activation graph feedback according to claim 1, characterized in that: The multi-task learning method uses parameter hard sharing to solidify the feature extraction capabilities of different tasks into a single deep learning model structure. This deep learning model structure includes a multi-layer one-dimensional convolutional neural network, a feature fusion layer, a global average pooling layer, and multiple classification heads. The one-dimensional convolutional neural network is a dual-channel parallel structure, used to extract pressure and flow velocity waveform features respectively. Finally, the feature fusion layer Add adds the two feature information, which is then passed through a global average pooling layer and classified using multiple classification heads.

5. The human-computer asynchronous recognition method based on multi-task learning and class activation graph feedback according to claim 1, characterized in that: The one-dimensional convolutional neural network includes a one-dimensional convolutional layer, a batch normalization layer, and a one-dimensional max pooling layer.

6. The human-computer asynchronous recognition method based on multi-task learning and class activation graph feedback according to claim 1, characterized in that: The training of the deep learning model ends when the loss function of the deep learning model converges or the number of iterations of the deep learning model reaches a preset number of iterations.

7. The human-computer asynchronous recognition method based on multi-task learning and class activation graph feedback according to claim 1, characterized in that: The training labels are constructed using one-hot encoding.

8. The human-computer asynchronous recognition method based on multi-task learning and class activation graph feedback according to claim 1, characterized in that: By using class activation mapping, a visual interpretation of the output results of the trained deep learning model for each sample is obtained, specifically as follows: a. After passing the feature map output from the last convolutional layer of the deep learning model through the feature fusion layer Add, the feature maps extracted from the two convolutions are merged by addition: in The feature map representing the pressure data activated by the filter unit k of the last convolutional layer at time t. The feature map representing the flow velocity data activated by filter unit k at time t; b. After global average pooling, the feature dimension of the feature fusion layer Add will be reduced to 1: F k =∑ t f k (t) Among them, F k This represents the output of the global average pooling layer, where t represents the time dimension and k represents the filtering unit. c. For a specific category c, the classification head h first calculates the input S of that category before the classification result is output. c ; in, This represents the weight of the feature map corresponding to each filter unit k in the last layer of the one-dimensional convolutional neural network. That is, the output of the feature map corresponding to each filter unit k after global average pooling, and the degree of contribution made by the classification head h to classify it as class c. in, This represents the class activation map corresponding to category c output by the classification head h, directly indicating the importance of the convolutional features being activated by the filter unit k, causing the classification head h to classify the input respiratory signal as category c. d. Upsample the class activation map to the dimension of the original input respiratory signal, and then map it onto the original respiratory data in the form of a heatmap to obtain a visual explanation of the deep learning model in the classification decision process.

Images