Method and system for constructing human-computer asynchrony recognition model based on active contrastive learning

Through active comparative learning and fine-tuning strategies, a deep neural network model in the field of mechanically ventilated human-computer asynchronous recognition is constructed, which solves the dependence of deep learning models on large-scale annotation data, and achieves high generalization performance under a small number of samples and Robustness reduces labeling costs.

CN117828317BActive Publication Date: 2024-11-12ZHEJIANG UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311699133.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-12
Publication Date
2024-11-12
Estimated Expiration
2043-12-12

AI Technical Summary

Technical Problem

Deep learning models rely on large-scale labeled data in the field of mechanically ventilated human-machine async recognition, which leads to high training difficulty and weak generalization performance, especially in the medical field, and the model's performance on real-world data is degraded.

Method used

Using an active contrast learning method, unsupervised contrast learning pre-trained feature extractor, combined with fine-tuning and active learning strategies, select information-rich data from a small number of samples for annotation, build a multi-head classification model, and evaluate the diversity of the training set. and test set coverage, iterative optimization model.

Benefits of technology

In small sample scenarios, the requirement for labeled data volume is significantly reduced, the robustness and generalization performance of the model are improved, the deviation of random data collection is avoided, the anti-noise effect is improved, and time and labor costs are saved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117828317B_ABST
    Figure CN117828317B_ABST
Patent Text Reader

Abstract

The present invention provides a method and system for constructing a human-machine asynchronous recognition model based on active contrastive learning. The present invention first pre-trains a deep neural network based on contrastive learning using massive unlabeled data to obtain a robust feature extractor. Secondly, based on active learning hybrid strategy sampling, samples with information content are selected from the unlabeled data pool for labeling. Finally, the selected samples are used to fine-tune the multi-task classification layer of the human-machine asynchronous recognition model. The present invention does not rely on large-scale, high-quality labeled data. By contrastive learning, the same characteristics of similar data can be autonomously learned from a large amount of data, which can not only save the time and labor costs required for labeling, but also make the feature extractor robust. In addition, combined with the active learning strategy, the data points with information content in the original data set can be quickly determined based on the test set, and the benchmark model can also be quickly adjustable.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of automated identification of human-machine asynchrony during mechanical ventilation, and in particular to a method and system for constructing a human-machine asynchrony identification model based on active contrast learning in a small sample scenario, as well as a method and system for identifying human-machine asynchrony. Background Art

[0002] Due to its excellent feature extraction and learning capabilities, deep learning models have also been widely used in the field of mechanical ventilation human-machine asynchrony recognition. However, deep learning relies on a large amount of labeled data, and large-scale high-quality annotation requires a lot of time and manpower costs. Especially in the medical field, annotators need to have certain professional knowledge and skills, so it is more difficult to obtain large, well-annotated datasets for deep neural network training and learning. Secondly, deep neural networks may perform well on some benchmark datasets, but due to domain differences between datasets, such as real-world data outside the training dataset, the performance of the model may be severely degraded. In general, given a test dataset for a related task, how to determine the diversity of the constructed training dataset and whether the deep neural network can fully fit and cover different related objects is a key issue. Therefore, it is of great significance to design a method that can enable deep neural networks to achieve excellent generalization performance in specific tasks through learning with a small number of samples. Summary of the invention

[0003] The purpose of the present invention is to train a deep neural network with strong generalization performance through a small number of samples and to determine through the model whether the constructed training set fully covers different related objects in the test set. A method and system for constructing a human-computer asynchronous recognition model based on active contrastive learning are provided.

[0004] The object of the present invention is achieved in that:

[0005] A method for constructing a human-machine asynchronous recognition model based on active contrastive learning, comprising:

[0006] S1. Acquire unlabeled mechanical ventilation data and construct an unlabeled data pool; the unlabeled data pool includes original samples and positive sample pairs constructed based on the original samples;

[0007] S2. Construct a feature extractor and a nonlinear projection head and train them based on an unlabeled data pool using unsupervised contrastive learning to obtain a trained feature extractor; wherein the feature extractor is used to extract feature representations of input samples, and the nonlinear projection head is used to map the feature representations to features in a space for calculating contrast loss;

[0008] S3. Building a human-machine asynchronous recognition model based on the trained feature extractor, wherein the human-machine asynchronous recognition model is composed of replacing the last global average pooling layer in the feature extractor with a flattening layer and then splicing it with a gradient embedding layer and a multi-head classification layer;

[0009] S4. Fine-tune the human-machine asynchronous recognition model based on the training set X; the training set X includes M original samples randomly selected from the unlabeled data pool U without duplication and the corresponding labels annotated by professionals;

[0010] S5. Fine-tune the trained multi-task classification model to predict the test dataset Y and evaluate the sample coverage of each category of the training set X on the test dataset Y; the sample coverage of each category of the training set X on the test dataset Y is CVG(X,Y n ) means that for each test sample in the test dataset Y, the most similar sample of the corresponding category in the training set X is found, and then the shortest distance of all samples in the test dataset Y is averaged;

[0011] S6. Set the sample ratio according to the sample coverage of each category (the higher the sample coverage of each category, the smaller the sample ratio of the corresponding category), and then randomly extract B original samples from the unlabeled data pool U in proportion and without duplication based on the pseudo labels obtained by the model's prediction of the samples; and predict the B original samples based on the human-computer asynchronous recognition model to construct a gradient embedding sequence;

[0012] S7, clustering the B gradient embedding sequences and selecting the original sample corresponding to the first gradient embedding sequence of the cluster center, after being labeled by professionals, into the training set X and repeatedly executing steps S4 to S7 until the iteration round is satisfied.

[0013] Furthermore, in step S1, the original sample includes a pressure-time and flow rate-time signal corresponding to a breath, and the positive sample pair is two related samples obtained by applying two independent random data enhancement operators based on an original sample.

[0014] Furthermore, in step S2, when unsupervised contrastive learning is used for training, a normalized temperature scale cross entropy loss function is used to calculate the contrast loss.

[0015] Furthermore, the feature extractor is composed of several one-dimensional convolutional layers, batch normalization layers, one-dimensional maximum pooling layers, feature fusion layers and global average pooling layers.

[0016] Furthermore, the sample coverage of each category of the training set X to the test data set Y is expressed as first finding the most similar sample of the corresponding category in the training set X for each test sample in the test data set Y, and then averaging the shortest distances of all samples in the test data set Y, as follows:

[0017]

[0018] Where a∈X n , d∈Y n , n is one of the categories c, Y n represents the sample set of category n in the test set Y, X n represents the set of samples whose categories are all n in the training set X, |Y n | indicates Y n The number of samples in the set, D represents the distance between the breathing sequences of two samples in the model space feature.

[0019] Furthermore, the distance between the breathing sequences of two samples in the model space feature is represented by the gradient weighted class activation signal of the samples:

[0020]

[0021] Where, L Grad-CAM Represents the gradient weighted class activation signal of the sample, and the superscript represents the corresponding sample; the gradient weighted class activation signal is calculated by first calculating the probability y of the output category of the model classification layer to the feature map f output by the last convolutional layer of the feature extractor on each channel k k The gradient of (t) is then calculated, and the global average of the gradient of each channel is used as the channel weight. Finally, the channel weight is weighted and summed with the feature map and obtained through the ReLU activation function, as follows:

[0022] L Grad-CAM =ReLU(∑ k a k f k (t));

[0023] Channel Weight in Represents the maximum probability y of the possible categories output by the sample model for the feature map f output by the last convolutional layer of the feature extractor on channel k k The gradient of (t); where the constant Z = ∑ t 1, t is the feature map f k (t) dimension.

[0024] Furthermore, the sample ratio is set according to the sample coverage of each category, as follows:

[0025] In the formula, CVG(X,Y n ) represents the sample coverage of category n of the training set X to the test data set Y; max(CVG(X,Y n )) represents the sample coverage of the category with the largest coverage in the training set X to the test data set Y; W nRepresents the proportion of samples in the nth category.

[0026] Further, the gradient embedding sequence is expressed as follows:

[0027] S=[s1,s2,…,s h ]

[0028] Among them, h is the number of classification heads in the multi-head classification layer, and the gradient embedding sequence constructed by each classification head is represented as s:

[0029]

[0030] p and l are the predicted probability vector output by the model and the output vector of the model gradient embedding layer, respectively. argmax(p) means finding the index of the position with the maximum probability in the predicted probability vector p.

[0031] A system for constructing a human-machine asynchronous recognition model based on active contrastive learning, used for executing a method for constructing a human-machine asynchronous recognition model based on active contrastive learning, comprising:

[0032] A data acquisition module, used to acquire unlabeled mechanical ventilation data and construct an unlabeled data pool; the unlabeled data pool includes original samples and positive sample pairs constructed based on the original samples;

[0033] The feature extractor construction and training module is used to construct a feature extractor and a nonlinear projection head and train them based on an unlabeled data pool using unsupervised contrastive learning to obtain a trained feature extractor;

[0034] A human-machine asynchronous recognition model construction module is used to construct a human-machine asynchronous recognition model based on a trained feature extractor. The human-machine asynchronous recognition model is composed of replacing the last global average pooling layer in the feature extractor with a flattening layer and then splicing it with a gradient embedding layer and a multi-head classification layer;

[0035] The human-machine asynchronous recognition model training module is used to fine-tune the human-machine asynchronous recognition model based on the training set X; the training set X includes M original samples randomly selected from the unlabeled data pool U without duplication and the corresponding labels annotated by professionals;

[0036] Predict the test dataset Y based on the fine-tuned multi-task classification model and evaluate the sample coverage of each category of the training set X on the test dataset Y;

[0037] The sample ratio is set according to the sample coverage of each category, and then B original samples are randomly selected from the unlabeled data pool U in proportion and without duplication based on the pseudo labels obtained by the model's prediction of the samples; and the B original samples are predicted based on the human-computer asynchronous recognition model to construct a gradient embedding sequence;

[0038] The B gradient embedding sequences are clustered and the original sample corresponding to the first gradient embedding sequence of the cluster center is selected and labeled by professionals, and then included in the training set X and steps S4 to S7 are repeated until the iteration round is met.

[0039] A method for identifying human-machine asynchrony based on active contrastive learning, specifically:

[0040] A breathing sequence to be identified is obtained, and the breathing sequence to be identified is input into the human-computer asynchrony recognition model constructed by the human-computer asynchrony recognition model construction method based on active contrast learning to obtain a human-computer asynchrony recognition result.

[0041] A human-machine asynchronous recognition system based on active contrast learning, comprising:

[0042] The recognition module is used to input the breathing sequence to be recognized into the human-computer asynchrony recognition model constructed by the human-computer asynchrony recognition model construction method based on active contrast learning to obtain the human-computer asynchrony recognition result.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The present invention is a method for constructing a human-machine asynchronous recognition model based on active contrastive learning under small samples. Through massive unlabeled data in a contrastive learning pre-training manner, the same characteristics of similar data are autonomously learned from a large amount of data, and encoded into high-level representations, and then fine-tuned according to different tasks, which can greatly reduce the difficulty of training. At the same time, self-supervised contrastive learning is unrelated to downstream tasks, so the representation can be well optimized separately, and the potential information of the data can be retained to the greatest extent. Moreover, through data enhancement, the model sees more samples and remembers more things, which improves the anti-noise effect. Active learning is involved in the process of fine-tuning downstream tasks. By using a small amount of existing labeled data, the model and the labeling personnel can interact efficiently, and then the most valuable and informative samples are selected for labeling. Under the condition of meeting the preset standards, the amount of labeled data required for model learning can be effectively reduced. By actively screening sample strategies, the deviation of random data collection in classical machine learning is also avoided.

[0045] The present invention is not limited to the detection of a certain type of human-computer asynchrony, but has general applicability to the identification and analysis of other types of human-computer asynchrony. Compared with the prior art, the human-computer asynchrony identification method based on deep learning proposed in the present invention does not rely on large-scale, high-quality labeled data. By comparing and learning the pre-trained feature extractor, it can not only save the time and manpower costs required for labeling, but also make the model as a whole robust and generalizable. In addition, combined with the active learning strategy, the data points with information content in the original data set can be quickly determined based on the test data set, which also enables the benchmark model to have the ability to be quickly adjusted. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The present invention is a flowchart of a method for constructing a human-machine asynchronous recognition model based on active contrastive learning.

[0047] Figure 2 This is a schematic diagram of the distance calculation of the breathing sequences of two samples in terms of spatial features proposed by the present invention.

[0048] Figure 3 This is a diagram showing the effect of several rounds of active learning iterations on the performance of the classification model in an embodiment of the present invention.

[0049] Figure 4 This is a system structure diagram for building a human-machine asynchronous recognition model based on active contrast learning in the present invention. DETAILED DESCRIPTION

[0050] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and examples. The following examples are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the invention.

[0051] In order to solve the problems of weak generalization performance of deep learning models in human-computer asynchrony detection and time-consuming and labor-intensive methods that rely on large-scale labeled data, the present invention provides a method for constructing a human-computer asynchrony recognition model based on active contrastive learning. The method of the present invention comprises: first pre-training a feature extractor based on a deep neural network using a massive unlabeled data pool based on contrastive learning to obtain a robust feature extractor, then selecting informative samples from the unlabeled data pool for labeling based on active learning hybrid strategy sampling, and finally fine-tuning the multi-head classification layer in the constructed human-computer asynchrony recognition model using the selected samples. The present invention provides an evaluation strategy for constructing the diversity of the training set and the coverage of the test set based on the similarity of the model feature space, and sampling is performed based on the evaluation results, so as to maintain the diversity of sample categories under small samples, and a human-computer asynchrony recognition model with excellent generalization performance can be trained by using only a small amount of existing labeled data. See. Figure 1 The method of the present invention specifically comprises the following steps:

[0052] S1. Obtain unlabeled mechanical ventilation data and construct an unlabeled data pool, which specifically includes the following sub-steps:

[0053] S1-1. First, a large amount of unlabeled mechanical ventilation data is collected and divided into individual respiratory waveform samples x as original samples according to the respiratory cycle. Each original sample x contains a pressure-time and flow-rate-time signal corresponding to a breath.

[0054] In a specific embodiment, a total of 70 million respiratory waveform samples are obtained as original samples.

[0055] S1-2, apply two independent random data enhancement operators to each original sample x, and obtain two related samples and

[0056] In a specific embodiment, the data enhancement operator includes no transformation, one-dimensional data jitter, scale scaling, signal amplitude distortion, time scale distortion, random shuffling, random resampling, X-axis flipping, Y-axis flipping, random position signal masking, random channel shielding, cropping and scaling, random smoothing, etc. Each data enhancement operator is set with a probability of 0.5 and operators can be superimposed to form an independent random data enhancement operator. Two independent random data enhancement operators are applied to each original sample x, and two related samples of the same sample are generated. and In one training batch, they are considered as positive sample pairs, and the rest are considered as negative sample pairs. Then resampling is performed, and all sample pairs after random data augmentation are and The pressure-time and flow-time signals are processed into time series data of equal length with a length of L=300, and then the z-score normalization is performed on each channel data:

[0057]

[0058] Among them, x m is the signal amplitude of the mth sampling point in each sample, L is the signal length of each independent sample, μ and σ are the mean and standard deviation of the signal amplitude of each sample, respectively, x' m It is the signal amplitude after normalization at the mth sampling point in each sample.

[0059] S1-3. The aforementioned original samples and positive sample pairs are combined to construct an unlabeled data pool U.

[0060] S2, construct a feature extractor f(·) and a nonlinear projection head g(·) and train them using unsupervised contrastive learning based on an unlabeled data pool to maximize the consistency between two related samples of the same sample after different data augmentation, which specifically includes the following sub-steps:

[0061] S2-1. Construct a feature extractor f(·) with a multi-layer one-dimensional convolutional neural network as the core and a nonlinear projection head g(·) with a multi-layer perceptron as the core. Table 1 and Table 2 respectively list the network structure composition and parameter definitions of the feature extractor f(·) and the nonlinear projection head g(·) used in the embodiment.

[0062] Table 1 Network structure composition and parameter definitions of the feature extractor f(·) used in the embodiment

[0063]

[0064] Table 2 Network structure composition and parameter definitions of the nonlinear projection head g(·) used in the embodiment

[0065]

[0066] S2-2. In each training batch, N original samples are randomly selected from the unlabeled data pool U without repetition, and 2N positive sample pairs generated by independent random data enhancement operators are formed into a batch training data set; among them, given a positive sample pair, the remaining 2 (N-1) data-enhanced samples in the batch training data set are regarded as negative samples.

[0067] S2-3, input the samples of the batch training data set into the feature extractor f(·), train the feature extractor f(·) to extract positive sample pairs and The characteristic representation is represented by h i and h j , the nonlinear projection head g(·) maps the feature representation to features in the contrast loss space, denoted as z i and z j ;

[0068] S2-3, use contrast loss to perform constrained training until the contrast training loss no longer decreases so that the same sample can achieve maximum consistency between two related samples after different data enhancements; in a specific embodiment, the normalized temperature scale cross entropy loss function Loss can be used to calculate the contrast loss, and the final loss is the sum of all positive sample pairs in a training batch. and A pair of positive samples The loss function is expressed as:

[0069]

[0070] Among them, 1 [k≠i] ∈{0,1} is the indicator function, which means it is 1 when k≠i and 0 otherwise. τ is the temperature parameter, N is the number of original samples in the training batch, i.e., the number of positive sample pairs, and sim is the cosine similarity function, i.e. ||*|| represents the norm. In a specific embodiment, τ is set to 0.1 and N is set to 1024. τ can appropriately scale the calculation of similarity. When computing resources and training time are fixed, a large training batch size should be set as much as possible, because a larger batch size can provide more negative examples and thus promote model convergence.

[0071] S3. Build a human-machine asynchronous recognition model based on the trained feature extractor. Specifically, discard the nonlinear projection head g(·), replace the last global average pooling layer in the trained feature extractor f(·) with a flattening layer, and then splice the gradient embedding layer and the multi-head classification layer to form a multi-task classification model, which is the human-machine asynchronous recognition model of the present invention. The gradient embedding layer and the multi-head classification layer are both composed of fully connected layers, and the activation functions are ReLU and Softmax respectively;

[0072] A pool-based mixed sampling active learning strategy is used to train a multi-task classification model and determine the diversity of the constructed training set and the coverage of different relevant objects in the test set, as follows:

[0073] S4. Fine-tune the human-machine asynchronous recognition model based on the training set X. Specifically, the model loads the weight of the feature extractor f(·) as initialization, sets the active learning iteration rounds T=20, the number of samples used for learning the human-machine asynchronous recognition model in each round M=50, and the candidate pool size B=5000 for selecting uncertainty and diversity samples; M samples are randomly selected from the unlabeled data pool U without duplication, and are annotated by professionals to form the training set X for fine-tuning the multi-task classification model;

[0074] S5. Fine-tune the trained multi-task classification model to predict the test data set Y and evaluate the coverage of each class of samples in the training set X for the test data set Y. Specifically, it includes the following sub-steps:

[0075] S5-1. Given a test data set Y, the test data set Y includes several original samples and corresponding labels annotated by professionals; use the fine-tuned multi-task classification model to predict the test data set Y and calculate the gradient weighted class activation signal L of each test sample. Grad-CAM ,Right now

[0076]

[0077] Channel Weight in Represents the maximum probability y of the possible categories output by the test sample model for the feature map f output by the last convolutional layer of the feature extractor on channel k k (t), ReLU(*) represents the ReLU function.

[0078] Among them, the constant Z = ∑ t 1, t is the feature map f k (t) dimension.

[0079] S5-2, evaluate the sample coverage of each category of the training set X to the test data set Y;

[0080] The constructed training set X covers each class of n samples of the test data set Y CVG(X,Y n ) means that for each test sample in the test data set Y, the most similar sample of the corresponding category in the training set X is found, and then the shortest distance of all samples in the test data set Y is averaged, that is,

[0081]

[0082] Where a∈X n , d∈Y n , n is one of the categories c, Y n represents the set of samples in the test set Y whose categories are all n, |Y n | indicates Y n The number of samples in the collection.

[0083] The distance D(a, b) between the breathing sequences of two samples in the model space feature is expressed as:

[0084]

[0085] Figure 2 The distance calculation between two samples of the same ineffective inspiratory effort class is shown, with the flow velocity waveform as an example. In this embodiment, the gradient weighted activation signal L Grad-CAM The activation domain is set to be greater than 0.5 as a valid feature. The larger the D value, the greater the distance between the two samples and the lower the similarity between the samples.

[0086] S6, first according to each type of sample coverage CVG (X, Y n ) Set the sample ratio in the candidate pool, and then randomly select B samples from the unlabeled data pool U in proportion without duplication based on the pseudo labels predicted by the model for the samples;

[0087] Obviously, the sample coverage of each class CVG(X,Y n), the higher the sample ratio should be, so that subsequent training can fully cover different related objects in the test set. In a specific embodiment, the sample ratio can be calculated as follows:

[0088]

[0089] In the formula, CVG(X,Y n ) represents the sample coverage of category n of the training set X to the test data set Y; max(CVG(X,Y n )) represents the sample coverage of the category with the largest coverage in the training set X to the test data set Y; W n Represents the proportion of samples in the nth category.

[0090] S7, clustering the B gradient embedding sequences and selecting the original sample corresponding to the first gradient embedding sequence of the cluster center, after being labeled by professionals, into the training set X and repeating steps S4 to S7 until the iteration round is satisfied. Specifically, it includes the following sub-steps:

[0091] S7-1. Based on the human-machine asynchronous recognition model, the prediction probability vector p and the output vector l of the model gradient embedding layer are calculated for B samples in the candidate pool, and the gradient embedding sequence S = [s1, s2, ..., s h ], where h is the number of classification heads. The gradient embedding sequence constructed by each classification head is represented as s:

[0092]

[0093] argmax(p) means finding the index of the position with the maximum probability in the predicted probability vector p.

[0094] In this embodiment, there are four classification tasks and corresponding classification heads. The constructed gradient embedding sequence S = [s1, s2, s3, s4] has a dimension of (128×4×1), which represents a 512-dimensional one-dimensional vector constructed by the probabilities corresponding to the output category labels of the four classification heads and the output vector of the gradient layer.

[0095] S7-2. Use a clustering algorithm to select M cluster centers for B gradient embedding sequences, and select the original sample corresponding to the first gradient embedding sequence of the cluster center as the data to be labeled; in a specific embodiment, the clustering algorithm can use KMeans++.

[0096] S7-3. After professionals have labeled M samples, they are included in the training set X and steps S4 to S7 are repeated until the active learning iteration round T is met.

[0097] Table 3 shows the classification performance of the human-machine asynchrony recognition model constructed after 20 rounds of active learning iterations in the embodiment, as well as the evaluation results of the training set and the test set. Figure 3 The changing trend of the classification performance of the network model in 20 iterations of the embodiment and the changing trend of the evaluation results of the training set and the test set are shown.

[0098] Table 3 Classification performance of the network model after 20 active learning iterations and evaluation of the training set and test set in the embodiment

[0099]

[0100]

[0101] The above results show that the method of the present invention can obtain a human-machine asynchronous recognition model with excellent generalization performance through fine-tuning training without the need for a large-scale labeled data set. The method provided by the present invention can effectively reduce the amount of labeled data required for model learning while meeting the preset standards, and has good generalization performance.

[0102] In the embodiment of the present invention, various types of human-machine asynchrony such as too long cycle, too short cycle, double triggering and ineffective inspiratory effort are detected and identified, and the diversity of the constructed training set and the coverage of the test set are evaluated based on the spatial feature similarity. However, the present invention is not limited to the detection of such human-machine asynchrony, and has general applicability to the identification and analysis of other types of human-machine asynchrony. The present invention can also be extended to the detection and analysis of various one-dimensional signals, such as electrocardiogram signals.

[0103] In addition, in a specific embodiment, step S5 further includes evaluating the diversity of the training set X itself. The diversity DIV(X) of the training set X is represented by the average distance between all original sample pairs of the same category in the training set X, that is,

[0104]

[0105] Where a,b∈X n , n is one of the categories c, X n represents the set of samples whose categories are all n in the training set X, |X n | indicates X n The number of samples in the collection.

[0106] Diversity can evaluate the complexity of each type of samples selected by active learning, measure the amount of information in the training set, and evaluate whether the training set is useful for the model.

[0107] Corresponding to the aforementioned embodiment of a method for constructing a human-computer asynchrony recognition model based on active contrastive learning, the present invention also provides an embodiment of a system for constructing a human-computer asynchrony recognition model based on active contrastive learning.

[0108] A system for constructing a human-computer asynchronous recognition model based on active contrastive learning, used to execute a method for constructing a human-computer asynchronous recognition model based on active contrastive learning, such as Figure 4 As shown, including:

[0109] A data acquisition module, used to acquire unlabeled mechanical ventilation data and construct an unlabeled data pool; the unlabeled data pool includes original samples and positive sample pairs constructed based on the original samples;

[0110] The feature extractor construction and training module is used to construct a feature extractor and a nonlinear projection head and train them based on an unlabeled data pool using unsupervised contrastive learning to obtain a trained feature extractor;

[0111] A human-machine asynchronous recognition model construction module is used to construct a human-machine asynchronous recognition model based on a trained feature extractor. The human-machine asynchronous recognition model is composed of replacing the last global average pooling layer in the feature extractor with a flattening layer and then splicing it with a gradient embedding layer and a multi-head classification layer;

[0112] The human-machine asynchronous recognition model training module is used to fine-tune the human-machine asynchronous recognition model based on the training set X; the training set X includes M original samples randomly selected from the unlabeled data pool U without duplication and the corresponding labels annotated by professionals;

[0113] Predict the test dataset Y based on the fine-tuned multi-task classification model and evaluate the sample coverage of each category of the training set X on the test dataset Y;

[0114] The sample ratio is set according to the sample coverage of each category, and then B original samples are randomly selected from the unlabeled data pool U in proportion and without duplication based on the pseudo labels obtained by the model's prediction of the samples; and the B original samples are predicted based on the human-computer asynchronous recognition model to construct a gradient embedding sequence;

[0115] The B gradient embedding sequences are clustered and the original sample corresponding to the first gradient embedding sequence of the cluster center is selected and labeled by professionals, and then included in the training set X and steps S4 to S7 are repeated until the iteration round is met.

[0116] As for the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The system embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0117] The embodiment of the present invention further provides a method for identifying human-machine asynchrony based on active contrast learning, which is specifically:

[0118] A breathing sequence to be identified is obtained, and the breathing sequence to be identified is input into a human-computer asynchrony recognition model constructed by the human-computer asynchrony recognition model construction method based on active contrast learning to obtain a human-computer asynchrony recognition result.

[0119] Furthermore, the embodiment also provides a human-machine asynchrony recognition system based on active contrast learning, comprising:

[0120] The recognition module is used to input the breathing sequence to be recognized into the human-computer asynchrony recognition model constructed by the human-computer asynchrony recognition model construction method based on active contrast learning to obtain the human-computer asynchrony recognition result.

[0121] In the above embodiments, the present invention is only exemplarily described, but those skilled in the art may make various modifications to the present invention without departing from the spirit and scope of the present invention after reading this patent application.

Claims

1. A method for constructing a human-machine asynchronous recognition model based on active contrastive learning, characterized in that: include: S1. Obtain unlabeled mechanical ventilation data and build an unlabeled data pool; The unlabeled data pool includes original samples and positive sample pairs constructed based on the original samples; S2, construct a feature extractor and a nonlinear projection head and train them based on an unlabeled data pool using unsupervised contrastive learning to obtain a trained feature extractor; S3. Building a human-machine asynchronous recognition model based on the trained feature extractor, wherein the human-machine asynchronous recognition model is composed of replacing the last global average pooling layer in the feature extractor with a flattening layer and then splicing it with a gradient embedding layer and a multi-head classification layer; S4. Fine-tune the human-machine asynchronous recognition model based on the training set X; the training set X includes M original samples randomly selected from the unlabeled data pool U without duplication and the corresponding labels annotated by professionals; S5. The fine-tuned multi-task classification model is used to predict the test data set Y, and the coverage of each class of samples of the training set X on the test data set Y is evaluated; the coverage of each class of samples of the training set X on the test data set Y is expressed as first finding the most similar sample of the corresponding class in the training set X for each test sample in the test data set Y, and then averaging the shortest distances of all samples in the test data set Y, as follows: Where a∈X n , d∈Y n , n is one of the categories c, X n represents the sample set of n categories in the training set X, Y n represents the set of samples in the test set Y whose categories are all n, |Y n | indicates Y n The number of samples in the set, D represents the distance between the breathing sequences of two samples in the model space feature; S6. Set the sample ratio according to the sample coverage of each category, and then randomly extract B original samples from the unlabeled data pool U in proportion without duplication based on the pseudo labels obtained by the model's prediction of the samples; and predict the B original samples based on the human-machine asynchronous recognition model to construct a gradient embedding sequence; wherein, the sample ratio is set according to the sample coverage of each category, as follows: In the formula, CVG(X,Y n ) represents the sample coverage of category n of the training set X to the test data set Y; max(CVG(X,Y n )) represents the sample coverage of the category with the largest coverage in the training set X to the test data set Y; W n Represents the proportion of samples in the nth category; S7, clustering the B gradient embedding sequences and selecting the original sample corresponding to the first gradient embedding sequence of the cluster center, after being labeled by professionals, into the training set X and repeatedly executing steps S4 to S7 until the iteration round is satisfied.

2. The method according to claim 1, characterized in that In step S1, the original sample includes a pressure-time and flow rate-time signal corresponding to a breath, and the positive sample pair is two related samples obtained by applying two independent random data enhancement operators based on the original sample.

3. The method according to claim 1, characterized in that In step S2, when unsupervised contrastive learning is used for training, a normalized temperature scale cross entropy loss function is used to calculate the contrast loss.

4. The method according to claim 1, characterized in that: The feature extractor is composed of several one-dimensional convolutional layers, batch normalization layers, one-dimensional maximum pooling layers, feature fusion layers and global average pooling layers.

5. The method according to claim 1, characterized in that The distance between the breathing sequences of two samples in the model space feature is represented by the gradient weighted class activation signal of the samples: Where, L Grad-CAM Represents the gradient weighted class activation signal of the sample, and the superscript represents the corresponding sample; L Grad-CAM =ReLU(∑ k to k f k (t)); Channel Weight in Represents the maximum probability y of the possible categories output by the sample model for the feature map f output by the last convolutional layer of the feature extractor on channel k k The gradient of (t); where the constant Z = ∑ t 1, t is the feature map f k (t) dimension.

6. A system for constructing a human-machine asynchronous recognition model based on active contrastive learning, used to execute a method for constructing a human-machine asynchronous recognition model based on active contrastive learning as described in any one of claims 1 to 5, characterized in that: include: A data acquisition module, used to acquire unlabeled mechanical ventilation data and construct an unlabeled data pool; The unlabeled data pool includes original samples and positive sample pairs constructed based on the original samples; The feature extractor construction and training module is used to construct a feature extractor and a nonlinear projection head and train them based on an unlabeled data pool using unsupervised contrastive learning to obtain a trained feature extractor; A human-machine asynchronous recognition model construction module is used to construct a human-machine asynchronous recognition model based on a trained feature extractor. The human-machine asynchronous recognition model is composed of replacing the last global average pooling layer in the feature extractor with a flattening layer and then splicing it with a gradient embedding layer and a multi-head classification layer; The human-machine asynchronous recognition model training module is used to fine-tune the human-machine asynchronous recognition model based on the training set X; the training set X includes M original samples randomly selected from the unlabeled data pool U without duplication and the corresponding labels annotated by professionals; The test dataset Y is predicted based on the fine-tuned multi-task classification model, and the sample coverage of each category of the training set X to the test dataset Y is evaluated; the sample coverage of each category of the training set X to the test dataset Y is expressed as first finding the most similar sample of the corresponding category in the training set X for each test sample in the test dataset Y, and then averaging the shortest distances of all samples in the test dataset Y, as follows: Where a∈X n , d∈Y n , n is one of the categories c, X n represents the sample set of n categories in the training set X, Y n represents the set of samples in the test set Y whose categories are all n, |Y n | indicates Y n The number of samples in the set, D represents the distance between the breathing sequences of two samples in the model space feature; The sample ratio is set according to the sample coverage of each category, and then B original samples are randomly selected from the unlabeled data pool U in proportion and without duplication based on the pseudo labels obtained by the model's prediction of the samples; and the B original samples are predicted based on the human-machine asynchronous recognition model to construct a gradient embedding sequence; wherein, the sample ratio is set according to the sample coverage of each category, as follows: In the formula, CVG(X,Y n ) represents the sample coverage of category n of the training set X to the test data set Y; max(CVG(X,Y n )) represents the sample coverage of the category with the largest coverage in the training set X to the test data set Y; W n Represents the proportion of samples in the nth category; The B gradient embedding sequences are clustered and the original sample corresponding to the first gradient embedding sequence of the cluster center is selected and labeled by professionals, and then included in the training set X and steps S4 to S7 are repeated until the iteration round is met.

7. A method for identifying human-machine asynchrony based on active contrastive learning, characterized in that: Specifically: Obtain a breathing sequence to be identified, input the breathing sequence to be identified into a human-machine asynchronous recognition model constructed by a method for constructing a human-machine asynchronous recognition model based on active contrast learning as described in any one of claims 1 to 5, and obtain a human-machine asynchronous recognition result.

8. A human-machine asynchronous recognition system based on active contrast learning, characterized in that: include: The recognition module is used to input the breathing sequence to be recognized into the human-computer asynchrony recognition model constructed by the human-computer asynchrony recognition model construction method based on active contrast learning as described in any one of claims 1 to 5 to obtain the human-computer asynchrony recognition result.