Method for identifying workload with label noise
By using entropy maximizing loss, gradient cropping and label smoothing methods in workload recognition, combined with time-frequency feature extraction network and classifier, the discrimination results are corrected by feature distribution, the label noise problem is solved and the model is improved. and generalization, achieving higher recognition accuracy.
Patent Information
- Application Number
- CN202410944264.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-07-15
AI Technical Summary
When processing workload data, the existing technology has ambiguity in the scale annotation and subject understanding deviation, resulting in label noise problems, affecting the generalization ability and accuracy of the model. The existing methods are difficult to fundamentally correct the data and improve the accuracy. limit.
A workload recognition method with label noise is proposed. Through pre-training model, entropy maximization loss, gradient cropping and label smoothing processing, combined with time-frequency feature extraction network and classifier, the discrimination results are corrected by feature distribution, and the noise-containing data is transferred. Semi-supervised learning as label-free data improves the robustness and generalization of the model.
Effectively reduce the impact of label noise on the model, improve the accuracy and generalization of workload recognition, enhance the robustness of the model, and ensure the accuracy of the recognition results.
Smart Images

Figure CN118964923A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of physiological signal recognition, and in particular to a workload recognition method in the presence of label noise. Background Art
[0002] Since most existing workload datasets use scales to label data, this labeling method has two main problems: (1) from the perspective of scale design: the description of the scale is ambiguous; (2) from the perspective of the subjects: the subjects' understanding of the scale is prone to bias, and different subjects' understanding of the scale may also be different. Therefore, the labels obtained by labeling the scale may be inaccurate, resulting in the problem of noisy labels in the dataset, which may cause the model to overfit on the noisy labels, thereby reducing the generalization ability of the entire model. Many researchers have tried to solve the problem of noisy dataset labels and proposed various algorithms, which have achieved good results. In particular, the algorithm based on semi-supervised learning first identifies clean samples and noisy samples, treats the noisy samples as unlabeled data, and then performs semi-supervised learning. It greatly reduces the impact of noisy data, so it has received more attention in the past two years and has high research value.
[0003] In order to solve the problem of insufficient generalization of models trained under label noise conditions, many scholars have tried to propose different methods. The first method is to use a robust loss function (Robust Loss). This type of method often designs some losses that are robust to label noise to reduce the impact of label noise. Ghosh et al. proved that for classification tasks under label noise, the mean absolute error (MAE) is more robust than the cross entropy loss (CE). In order to solve the problem that CE is not robust to label noise, Zhang et al. combined the robustness of MAE to noise and the stability of CE training, and proposed a generalized cross entropy loss GCE Loss.
[0004] The second type of method is based on sample importance weighting. The basic idea of this type of method is to assign lower weights to noisy samples and higher weights to clean samples, so that the model can reduce the impact of noise. Sukhbaatar et al. reduced the weight of noisy data and increased the weight of clean data, but this lost a lot of information and the upper limit of generalization ability was not high. Litany et al. regarded this problem as transfer learning, in which the source domain is the original data (the label is noisy) and the target domain is a subset of the clean data set. Then the data in the source domain is weighted to minimize the loss of the target domain.
[0005] In the past two years, many researchers have tried to solve this problem based on semi-supervised learning methods, and have achieved relatively good results. This type of method first divides the training set into noisy data and non-noisy data through certain methods, regards the noisy data as unlabeled data, and regards the clean data as labeled data, and then performs semi-supervised learning on the labeled data and unlabeled data respectively. Li et al. tried to use the loss size as an indicator to determine whether it is a noisy label. The Gaussian mixture model can separate noisy and clean labels. At the same time, during the training process, the entropy of the model is kept at a large level through loss, which can minimize the impact of noisy labels on the model. Further, the authors obtained better results by improving the semi-supervised learning method obtained by MixMatch.
[0006] Geoff et al. used another method to determine whether the label is noisy. They calculated the AUM (Area Under the Margin) during the training process, that is, the cumulative size of the logit output by the network, to determine whether the label of the corresponding sample is noisy. The authors discarded the data judged to be noisy, constructed a relatively clean training data set, and obtained a model with better generalization performance through training based on this data set.
[0007] Although the above algorithms have improved the generalization of label noise learning to a certain extent, there are still many problems. First of all, except for semi-supervised learning methods, other methods often only reduce the negative impact of label noise, and do not fundamentally correct the data. This feature of these algorithms makes it impossible to further improve their accuracy, and the upper limit is low. In addition, for some existing semi-supervised learning-based algorithms, although good results have been achieved, there are still some problems in the process of judging whether the sample label is noisy. These algorithms judge whether the data is noisy based on a certain indicator (for example, the DivideMix method in the literature is based on the size of the loss), and the results may still have a certain amount of noise. These methods do not correct the judgment results from the feature level, which may cause the judgment results to still have noise, thus affecting the subsequent semi-supervised learning results. Summary of the invention
[0008] Aiming at the problem of data label noise that may exist in workload data, the present invention proposes a workload identification method with label noise to reduce the influence of label noise on the workload identification model, increase its generalization, and ensure the identification accuracy.
[0009] The technical solution adopted by the present invention is:
[0010] A workload identification method in the presence of label noise, the method comprising the following steps;
[0011] Step 1: pre-training the workload recognition model based on the original data set;
[0012] During pre-training, the workload labels output by the workload identification model are smoothed, and the training loss of the samples is calculated using the entropy maximization loss. When the gradient is calculated based on the training loss, the gradient is clipped and the model parameters are tuned using the gradient descent algorithm.
[0013] Step 2: Perform label discrimination based on the labels of the recognition loss training data set of the workload recognition model to distinguish between a noisy data set and a clean data set (a data set without noise);
[0014] Then, the discrimination result is corrected based on the characteristic distribution of the sample: the contribution ratio of the sample during gradient update is configured based on the characteristic distribution of the sample. The contribution ratio of densely distributed samples is large, and the contribution ratio of sparsely distributed samples is small. For example, the learning weight of sparsely distributed samples is reduced, or the learning weight of densely distributed samples is increased, or the learning weight of densely distributed samples is increased and the distribution coefficient is reduced.
[0015] Step 3: Use the noisy dataset as an unlabeled dataset and the clean dataset as a labeled dataset, and perform supervised learning on the workload identification model based on the correction results to obtain the final workload identification model.
[0016] Furthermore, the workload identification model includes a time-frequency feature extraction network and a classifier, wherein the time-frequency feature extraction network is used to extract the time domain signal features and frequency domain signal features of the physiological signal, and then the time domain signal features and the frequency domain signal features are concatenated and fused to obtain the final signal features and input into the classifier; the classifier is used to output the classification probability of the workload category to which the current signal feature belongs.
[0017] Furthermore, the final signal features extracted by the time-frequency feature extraction network specifically include:
[0018] Based on a preset cutting window size, the physiological signal of the sample is cut and processed to cut the physiological signal into a plurality of signal segments;
[0019] Input each fragment into a convolutional neural network for dimensionality reduction to obtain the encoded representation of each fragment;
[0020] Then, the physiological signal encoding composed of the encoded representations of all the segments is input into the Transformer encoder for feature extraction to obtain the time domain features of the samples;
[0021] Extract frequency domain features of the physiological signals of the samples based on the attention mechanism to obtain the frequency domain features of the samples;
[0022] The time domain features and frequency domain features are spliced and fused to obtain the final signal characteristics of the sample.
[0023] Furthermore, the label smoothing process is specifically as follows:
[0024] The code p of each label encoding position of the sample i Set to:
[0025] If the predicted label of the sample is consistent with the true label, then p i Let τ be τ, where τ is a number less than 1 and greater than 0;
[0026] If the predicted label of the sample is inconsistent with the true label, then p i Set to Where C represents the number of identification categories of the workload identification model.
[0027] Furthermore, the expression of entropy maximization loss is:
[0028]
[0029] Where C represents the number of identification categories of the workload identification model, p(x i ) represents the probability distribution of the i-th class sample output by the workload identification model.
[0030] Furthermore, the gradient is clipped as follows: if the L2 norm of the calculated gradient is less than or equal to the first threshold clip_value, it remains unchanged; otherwise, the gradient is adjusted to Where norm(ω) represents the L2 norm of the gradient ω.
[0031] Furthermore, when the discrimination result is corrected based on the characteristic distribution of the sample, the density of the area where the sample is located is represented by the Euclidean distance of the feature.
[0032] Furthermore, in step 2, when the discrimination result is corrected based on the characteristic distribution of the sample, the entropy of the evolving K nearest neighbor classifier is calculated based on the entropy of the probability distribution obtained after the sample is input into the K nearest neighbor classifier;
[0033] If the K nearest neighbor entropy of the sample is greater than the preset second threshold, it is determined that the current sample is in a low-density area of the feature space; if the K nearest neighbor entropy of the sample is less than the preset third threshold, it is determined that the current sample is in a high-density area of the feature space; when correcting the discrimination result, the learning weight of the sample in the low-density area is reduced and / or the learning weight of the sample in the high-density area is increased.
[0034] The technical solution provided by the present invention brings at least the following beneficial effects:
[0035] The present invention adopts a semi-supervised learning workload identification mechanism of first discriminating and then correcting to solve the problem of label noise in workload identification. The present invention first uses the original data to preheat the neural network; then divides the original data into a clean data set and a label noisy data set; finally, the label noisy data set is regarded as unlabeled data for semi-supervised learning. The present invention proposes a new physiological signal feature extraction network, which uses three different means to reduce the overfitting of noise data in the preheating stage, increase the robustness of the model, and improve the subsequent accuracy of label noise discrimination; the label noise discrimination result is corrected using the information of feature distribution, thereby increasing the generalization of the recognition model and better ensuring the recognition accuracy of the workload recognition model. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0037] Figure 1 A schematic diagram of the processing process of a workload identification method with label noise provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0038] To make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be described in detail and completely in conjunction with the drawings in the implementation of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings can be arranged and designed using different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present invention.
[0039] The embodiment of the present invention provides a workload recognition method with label noise. The method proposes a First Discriminate Then Correct Semi-supervised Learning Workload Recognition (FDTC) algorithm to solve the problem of label noise in workload recognition. The overall idea is divided into three stages: first, the original data is used to preheat the neural network; then the original data is divided into a clean data set and a label noisy data set; finally, the label noisy data set is regarded as unlabeled data to perform semi-supervised learning on the recognition model. The main work is in the first two stages, which mainly include three points: (1) a new physiological signal time-frequency feature extraction network is proposed; (2) three different methods are used to reduce the overfitting of noisy data in the preheating stage; (3) the label noise discrimination result is corrected using the information of feature distribution. First, in the preheating stage, three methods are used to try to alleviate the overfitting of the model to the noisy label, increase the robustness of the model, and improve the accuracy of the subsequent discrimination of label noise. In addition, the discrimination result based on the loss size is corrected using the feature distribution of the data.
[0040] In the method proposed in the embodiment of the present invention, the workload identification model used includes: (physiological signal) time-frequency feature extraction network and classifier, wherein the time-frequency feature extraction network is used to extract the time domain signal features and frequency domain signal features of physiological signals (such as EEG signals), and then the two are spliced and fused to obtain the final signal features and input into the classifier; the classifier is used to output the classification probability of the workload category to which the current signal feature belongs. Figure 1 The specific implementation process of the workload identification method with label noise provided by the embodiment of the present invention includes:
[0041] Phase 1, the purpose of the warm-up phase is to train a model that is not overfitted to the noise data as much as possible. Therefore, the method proposed in the embodiment of the present invention uses three different methods in the entire pre-training phase to alleviate the overfitting of the model to increase the accuracy of subsequent noise label discrimination.
[0042] (1) Entropy maximization loss
[0043] Entropy can measure the degree of disorder of a system, and its expression is:
[0044]
[0045] Among them, p(·) represents the probability distribution of the recognition model output, x i represents the i-th class sample, and C represents the number of classes in the data set.
[0046] Using the traditional cross entropy loss to optimize the recognition model will make the output of the recognition model too confident, which will cause the recognition model to overfit to the noise data and reduce the generalization of the recognition model. Therefore, the output of the recognition model needs to be less confident in order to reduce the overfitting of the recognition model to the noise. Therefore, the embodiment of the present invention uses the definition of entropy to measure the degree of chaos of a system, calculates the negative entropy of the probability distribution of the final output of the recognition model, and adds it to the loss function, and then uses the optimizer to try to keep the entropy not too small, which can reduce the overfitting of the recognition model to the label noise. Among them, the entropy maximization loss can be expressed as:
[0047]
[0048] (2) Gradient Clipping
[0049] The optimization of deep neural networks often uses the gradient descent algorithm, which is based on the gradient (first-order derivative) of the network parameters. The larger the gradient, the larger the update amplitude of the neural network. The larger the update amplitude, the more likely the neural network is to overfit to noisy labels, thereby reducing the generalization of the neural network. Therefore, it is necessary to constrain the gradient during the optimization process of the neural network. Gradient clipping is a method to avoid excessive gradients during the optimization process. Its basic idea is that if the L2 norm of the gradient is greater than a certain threshold clip_value, the gradient will be scaled so that its scaled L2 norm is equal to clip_value. Gradient clipping can be expressed using the formula:
[0050]
[0051] in, represents the gradient after gradient clipping, ω represents the gradient, and norm(ω) represents the L2 norm of the gradient.
[0052] (3) Label smoothing
[0053] Generally, one-hot encoding (also called hard labeling) is used to train deep neural networks. One-hot encoding means encoding the correct category as 1 and the incorrect category as 0. Assuming that the correct label of a sample is medium load, and the data set contains three categories in total, then its corresponding One-hot encoding is (0,1,0). It can be seen that the One-hot label will make the trained neural network more confident and easily cause the model to overfit to the noise label. Therefore, an embodiment of the present invention uses label smoothing technology to reduce the overfitting of the neural network to noise. Label smoothing does not set the encoding of the true category to 1, but a number τ less than 1 and greater than 0. Nor will the encoding of other categories be set to 0, but rather C is the number of categories. The code p for each position iIt can be expressed using the following formula:
[0054]
[0055] Among them, i is the sample number and y is the true category (true label).
[0056] It should be noted that, when the embodiment of the present invention is implemented, the workload category is predicted based on the recognition model, wherein the granularity (number of categories) of the workload recognition result depends on the dataset used, for example, the workload level is divided into two categories in the STEW dataset, while three different levels of workload are stimulated in the CONTEST dataset. When alleviating the overfitting of the model, the labels are firstly smoothed, and then the entropy maximization loss is used as the classification loss of the model during the training process, and finally, when back-propagation is performed based on the calculated loss by the gradient descent method to optimize the model parameters, the gradient is clipped before back-propagation.
[0057] In stage 2, after the warm-up stage, it is necessary to discriminate the label noise of the original data based on the trained time-frequency feature extractor (time-frequency feature extraction network). In addition to using loss to discriminate the label, the method proposed in the embodiment of the present invention also corrects the discrimination result based on the feature distribution.
[0058] High-density region criterion: This criterion is mainly aimed at unlabeled data (labeled noisy data obtained by loss-based discrimination methods). It means that the true label of unlabeled data falling in the high-density region (the high-density region indicates the region where the density of labeled samples of the same category in the feature space approaches 1) is usually the same as the label of the corresponding high-density region. The characteristics of high-density regions are low sample mixing and high data density.
[0059] Low-density region criterion: This criterion is mainly aimed at labeled data (referring to the labeled noise-free data obtained by the loss-based discriminant method), and refers to the labeled data falling in the low-density area (the low-density area indicates the area where the density of labeled samples of the same category in the feature space approaches 0). The label corresponding to the labeled data is usually different from the true label. The characteristics of the low-density area are high degree of sample mixing and low data density.
[0060] The embodiment of the present invention uses the Euclidean distance of the features of the K nearest neighbors from the sample point to represent the density of the area where the sample is located. The larger the average distance, the lower the sample density in the area, and the smaller the average distance, the higher the sample density in the area. The entropy of the category distribution of these K nearest neighbors is used to measure the degree of mixing of the samples. The larger the entropy, the higher the degree of mixing in the area, and the smaller the entropy, the lower the degree of mixing in the area. Therefore, considering these two points comprehensively, the embodiment of the present invention is based on the idea of the K nearest neighbor algorithm, considers the distance factor of the K nearest neighbors, and calculates the entropy of the classification probability distribution to measure the sample distribution density near a sample. Using p i To represent the probability distribution obtained after the i-th sample is input into the K nearest neighbor classifier, then the probability distribution p can be calculated i The entropy of a sample is called K Nearest Neighbors Entropy (KNNE). If the K nearest neighbor entropy of a sample is greater than a threshold, then the sample can be considered to be in a low-density area of the feature space. If the K nearest neighbor entropy of a sample is lower than a threshold, then the sample can be considered to be in a high-density area of the feature space. Based on the high-density area criterion, it can be seen that when the unlabeled data feature falls in a high-density area of a class, the label of the unlabeled data should be the same as that of the class. Therefore, the embodiment of the present invention proposes a first correction rule: for a certain unlabeled data, if its K nearest neighbor entropy is less than a threshold α, then the unlabeled data can be labeled with a label corresponding to the output of the K nearest neighbor algorithm. Use u i represents the unlabeled data feature, then the calculation method of its K nearest neighbor entropy is:
[0061] KNNE(u i )=Entropy(KC(u i )),
[0062] Here, KC(·) represents the K nearest neighbor classifier, and KNNE(·) represents the K nearest neighbor entropy.
[0063] In addition, based on the low-density area criterion, it can be used to determine whether certain labeled samples are credible. If the K nearest neighbor entropy of a labeled sample is higher than the threshold β, then the sample is considered to be in a low-density area. If some labeled samples are in a low-density area, then the actual label of the labeled sample may be wrong. Based on this, the embodiment of the present invention will reduce the actual learning weight of all samples in a low-density area.
[0064] The embodiments of the present invention are based on the high-density area criterion and the low-density area criterion. It can be seen that according to the different distributions of different sample features, their contribution ratios during gradient updating are different. Therefore, the learning weight can be adjusted. For example, the actual learning weight of all samples in the low-density area is reduced, and the specific degree of reduction can be set based on the actual application scenario.
[0065] In stage 3, after loss-based label noise discrimination processing, labeled data sets and unlabeled data sets are obtained. Based on the selected semi-supervised learning algorithm, the label-noisy data set is regarded as unlabeled data for semi-supervised learning. A recognition model for workload identification is obtained. For example, the semi-supervised learning algorithm adopts the FixMatch algorithm. FixMatch requires strong and weak enhancement of the data. In an embodiment of the present invention, the enhancement methods used for the physiological signals used include: adding Gaussian white noise (Jitter), scaling (Scale), random permutation (Random Permutation), random clipping (Random Clipping) and random masking (Random Masking).
[0066] Transformer has shown great advantages in processing time series data. However, directly inputting physiological signals into Transformer Encoder for feature extraction will cause overfitting problems. In order to solve this problem, a series of processing of physiological signals is required. The input physiological signals are divided into multiple time windows, each time window is reduced in dimension by a convolutional neural network, and finally the encoding of the signal in this window is output to reduce the parameters of the entire model, thereby reducing the overfitting of the model.
[0067] In the embodiment of the present invention, the size of the cutting window of the physiological signal is defined as W, and the step size is S. Based on the cutting window, the entire physiological signal x i Divided into several segments:
[0068]
[0069] Among them, K represents the number of divided segments, S i represents the cutting result of sample i, i.e., the set of fragments, represents the j-th fragment, j=1,2,…,K.
[0070] Then each fragment Input to the encoding layer E m , get the encoding representation of each fragment: Thus, the encoding representation based on K fragments Get the encoding of this physiological signal
[0071] Then the obtained physiological signal encoding is input into Transformer Encoder for feature extraction. The formula is expressed as:
[0072]
[0073] Where En(·) is the Transformer Encoder, is the obtained time domain feature.
[0074] The entire feature extraction process of the time domain signal can be expressed as:
[0075]
[0076] in, Represents the entire time domain feature extraction process, including the division of physiological signal time windows and feature extraction based on Em and En.
[0077] Different physiological signals have different main frequencies. For example, the main frequency of the ECG signal is 0.5-45HZ, and the main frequency of the EEG is 0.5-80HZ. Therefore, the noise outside the main frequency should be discarded, and the model should focus more on the main frequency. In addition, the importance of the main frequencies of physiological signals is not the same. For example, the workload focuses more on the α band (8-12HZ) and θ band (4-8HZ) of the EEG. Based on this, this chapter uses the attention mechanism for the frequency domain feature extraction network to make the model focus on those frequency bands that are more important to the workload, thereby improving the generalization ability of the network model. The entire feature extraction process of the entire frequency domain signal is expressed as follows:
[0078]
[0079] in represents the frequency domain feature extractor, Represents the extracted frequency domain features.
[0080] The time domain feature extractor and the frequency domain feature extractor constitute the time-frequency feature extractor (i.e., the time-frequency feature extraction network). After extracting the time domain features and the frequency domain features, the two features are fused by splicing to obtain the final physiological signal features. The formula is as follows:
[0081]
[0082] Then the feature fea of each sample iThe input is to the classifier (such as at least one fully connected layer and one fully connected layer with a softmax function in a cascade), and the time-frequency feature extraction network and the classifier are trained based on their output and label smoothing, entropy maximization loss, and gradient clipping; the loss of each sample is judged based on the trained model, and then the judgment result is corrected through the feature distribution of noisy samples and non-noisy samples (clean samples); finally, the samples judged as noisy are regarded as unlabeled samples, and the clean samples are regarded as labeled samples, and the recognition model (or only its classifier) is semi-supervised learning is performed based on the unlabeled and labeled samples to obtain the final workload recognition model.
[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
[0084] The above are only some embodiments of the present invention. For those skilled in the art, several modifications and improvements can be made without departing from the creative concept of the present invention, which all belong to the protection scope of the present invention.
Claims
1. A workload identification method in the presence of label noise, characterized in that: The following steps are involved: Step 1: pre-training the workload recognition model based on the original data set; During pre-training, the workload labels output by the workload identification model are smoothed, and the training loss of the samples is calculated using the entropy maximization loss. When the gradient is calculated based on the training loss, the gradient is clipped and the model parameters are tuned using the gradient descent algorithm. Step 2, label discrimination is performed based on the labels of the recognition loss training data set of the workload recognition model to distinguish between the noisy data set and the clean data set; Then, the discrimination result is corrected based on the characteristic distribution of the sample: the contribution ratio of the sample during gradient update is configured based on the characteristic distribution of the sample. The contribution ratio of densely distributed samples is large, and the contribution ratio of sparsely distributed samples is small. Step 3: Use the noisy dataset as an unlabeled dataset and the clean dataset as a labeled dataset, and perform supervised learning on the workload identification model based on the correction results to obtain the final workload identification model.
2. The method according to claim 1, characterized in that The workload identification model includes a time-frequency feature extraction network and a classifier, wherein the time-frequency feature extraction network is used to extract the time domain signal features and frequency domain signal features of the physiological signal, and then the time domain signal features and the frequency domain signal features are spliced and fused to obtain the final signal features and input into the classifier; the classifier is used to output the classification probability of the workload category to which the current signal feature belongs.
3. The method according to claim 2, characterized in that The final signal features extracted by the time-frequency feature extraction network include: Based on a preset cutting window size, the physiological signal of the sample is cut and processed to cut the physiological signal into a plurality of signal segments; Input each fragment into a convolutional neural network for dimensionality reduction to obtain the encoded representation of each fragment; Then, the physiological signal encoding composed of the encoded representations of all the segments is input into the Transformer encoder for feature extraction to obtain the time domain features of the samples; Extract frequency domain features of the physiological signals of the samples based on the attention mechanism to obtain the frequency domain features of the samples; The time domain features and frequency domain features are spliced and fused to obtain the final signal characteristics of the sample.
4. The method according to claim 1, characterized in that The label smoothing process is as follows: The code p of each label encoding position of the sample i Set to: If the predicted label of the sample is consistent with the true label, then p i Let τ be τ, where τ is a number less than 1 and greater than 0; If the predicted label of the sample is inconsistent with the true label, then p i Set to Where C represents the number of identification categories of the workload identification model.
5. The method according to claim 1, characterized in that The expression of entropy maximization loss is: Where C represents the number of identification categories of the workload identification model, p(x i ) represents the probability distribution of the i-th class sample output by the workload identification model.
6. The method according to claim 1, characterized in that The gradient clipping process is as follows: if the calculated L2 norm of the gradient is less than or equal to the first threshold clip_value, it remains unchanged; otherwise, the gradient is adjusted to Where norm(ω) represents the L2 norm of the gradient ω.
7. The method according to claim 1, characterized in that When correcting the discrimination result based on the feature distribution of the sample, the density of the area where the sample is located is represented by the Euclidean distance of the feature.
8. The method according to claim 1, characterized in that In step 2, when the discrimination result is corrected based on the characteristic distribution of the sample, the entropy of the evolving K nearest neighbor classifier is calculated based on the entropy of the probability distribution obtained after the sample is input into the K nearest neighbor classifier; If the K nearest neighbor entropy of the sample is greater than the preset second threshold, it is determined that the current sample is in a low-density area of the feature space; If the K nearest neighbor entropy of the sample is less than the preset third preset, it is determined that the current sample is in a high-density area of the feature space; When correcting the discrimination result, the learning weight of the samples in the low-density area is reduced and / or the learning weight of the samples in the high-density area is increased.
Citation Information
Patent Citations
Label anti-noise text classification method based on semi-supervised learning
CN116521863A
Similarity-based point query weak supervision point cloud semantic segmentation method and system
CN116894946A
Multi-label image recognition method, device and equipment based on hierarchical prompt learning
CN117079009A
Noise-containing label image identification method and system fusing partial label and negative learning
CN117557886A
Cited By
Intrusion detection model training method and intrusion detection method
CN119922022A
