Debiasing consistent semi-supervised action recognition method

Abstract

The application provides a de-biasing consistent semi-supervised action recognition method, global average sampling is used to obtain global information of a video, local continuous sampling is used to obtain local information of the video, model training is performed according to the fact that the local information and the global information of the video are consistent in semantics, the global information and the local information of the video are fully utilized, the time sequence of an action is more fully modeled, and the time sequence modeling capability and the robustness of the model are enhanced; and the long-tail distribution problem of pseudo labels which is not paid attention to by a traditional method is solved, adaptive de-biasing processing is performed on the output of a teacher model, dynamic de-biasing is performed on the output of the teacher model without increasing training parameters, and more reasonable pseudo labels are obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human action recognition, in particular to a bias-removed consistent semi-supervised human action recognition method. BACKGROUND

[0002] Action Recognition plays an important role in the field of computer vision, it is not only an important development direction of the field, but also a key component of video understanding. The latest development in action recognition has opened the door to a series of real-world applications, including but not limited to visual security systems, behavior research, sports analysis and elderly fall detection systems.

[0003] In the field of action recognition, a common practice is to first pre-train the model on a large-scale dataset (for example, Kinetics contains up to 650,000 video clips), and then fine-tune on a small downstream dataset to obtain better performance. However, due to the time-consuming and expensive nature of annotating large-scale datasets, training models on large-scale datasets with complete annotations is hindered; on the other hand, the global upload of about 500 hours of video to YouTube every minute naturally provides a large amount of unlabeled data; at the same time, semi-supervised learning and self-supervised learning have achieved good results in traditional image tasks. Therefore, it is a natural idea to use semi-supervised models to make full use of large-scale unlabeled datasets with acceptable costs.

[0004] The field of semi-supervised action recognition has achieved many fruitful research results, which can be mainly divided into three categories: semi-supervised action recognition based on pseudo-labels, semi-supervised action recognition combined with self-supervision, and semi-supervised action recognition based on multi-modal. By summarizing the current research status of semi-supervised models, it can be found that there are mainly two problems:

[0005] 1. Most of the current semi-supervised action recognition methods are obtained by twice continuous sampling of two clips as input, for example, the SVFormer and TimeBalance methods which have achieved good experimental results, but they ignore the integrity of the video and ignore the importance of global information of the video; in addition, a small part of methods use global sampling, such as the TCL method which uses fast and slow global sampling by using the fact that the video playback speed does not change the semantic information of the video, but the method inevitably ignores the role of local information. It can be seen that as a typical video task, the current semi-supervised action recognition methods generally do not consider the essential properties of the video, and do not fully utilize the dynamic spatio-temporal information of the video.

[0006] 2. The current research in the field of semi-supervised action recognition does not focus on the severe bias of the model during the training process, which makes the pseudo labels predicted by the model present a very obvious long-tail distribution. The applicant finds through a large number of experiments that in the field of semi-supervised action recognition, even if the data set is evenly distributed, the distribution of the pseudo labels will appear significantly uneven in the training process based on the semi-supervised framework FixMatch. SUMMARY

[0007] In view of the problems in the prior art, the application provides a debiased consistent semi-supervised action recognition method, which specifically comprises the following steps:

[0008] Step 1: sampling the video to be recognized; the sampling here can be any kind of sampling, such as average sampling or continuous sampling.

[0009] Step 2: inputting the frame sequence sampled in step 1 into the trained recognition model to obtain the recognition result;

[0010] The recognition model is trained by the following process:

[0011] Step 2.1: obtaining training samples, the training samples including a small amount of labeled videos and a large amount of unlabeled videos; specifically, including N L labeled videos (xl, yl) ∈ D L and N U unlabeled videos xu ∈ D U , wherein xl is a labeled video sample, yl is the label of xl, xu is an unlabeled video sample, D L is a labeled video set, D U is an unlabeled video set, D U > D L ;

[0012] Step 2.2: sampling the labeled videos and training by using the labels to obtain the trained model as the initial value of the teacher model and the student model; specifically, sampling the labeled videos xl twice, obtaining global input xl g by global average sampling, obtaining local input xl l by local continuous sampling, and training by using the label yl;

[0013] Step 2.3: globally averaging and locally continuously sampling the unlabeled videos, and performing weak enhancement and strong enhancement on the segments obtained by global average sampling, and performing weak enhancement on the segments obtained by local continuous sampling; specifically, for the unlabeled video xu, obtaining the segment xu g by global average sampling, and obtaining the segment xu lFor the fragment xu g Two views are generated using weak enhancement and strong enhancement respectively. gw =A weak (xu g ),xu gs =A strong (xu g ), A weak (.) indicates weak enhancement, A strong (.) indicates strong enhancement; for the fragment xu l Use weak enhancement to generate view xu lw =A weak (xu l );

[0014] Step 2.4: Use the weakly enhanced view of the globally averaged sampled segment as input to the teacher model, obtain the output of the teacher model, and then apply the formula...

[0015]

[0016] The output of the teacher model is debiased; where p k The output of the teacher model for the k-th input sample. The output of the teacher model after bias removal. Let λ be the bias vector, and λ be the debiasing parameter.

[0017] Furthermore, bias vectors The initial value is a vector of all 1s with the same dimension as the number of categories to be identified. During training, according to the formula...

[0018]

[0019] Momentum updates are performed, where m2 is the second momentum coefficient and μB is the number of samples in a training batch.

[0020] Furthermore, the bias removal parameter λ is an adaptive bias removal parameter, determined jointly by the prediction and bias vector of the current sample, specifically as follows:

[0021]

[0022] in Indicates taking the bias vector The mean, argmax(p) k ) indicates taking p k The category with the highest probability of being identified. Indicates taking the bias vector argmax(p) k The corresponding element value;

[0023] Step 2.5: learning the student model by using the de-biased teacher model output as pseudo label pairs, wherein the input of the student model is the globally average sampled enhanced view and the locally continuous sampled weak enhanced view;

[0024] Further, the student model is updated by the pseudo label predicted by the teacher model, wherein the parameter of the teacher model is θ t The student model parameter is updated by the exponential moving average as follows: s

[0025] θ t ←m1θ t +(1-m1)θ s

[0026] wherein m1 is the first momentum coefficient, θ t and θ s are the parameters of the teacher model and the student model respectively;

[0027] Step 2.6: obtaining the trained recognition model after reaching the set number of training iterations.

[0028] Advantages

[0029] The de-biased consistency semi-supervised human action recognition method provided by the application uses global average sampling to obtain global information of the video, uses local continuous sampling to obtain local information of the video, and performs model training according to the semantic consistency between the local information and the global information of the video, fully utilizes the global information and the local information of the video, more fully models the time sequence of the action, and enhances the time sequence modeling capability and robustness of the model; and the long-tail distribution problem of the pseudo label not concerned by the traditional method is solved, the output of the teacher model is adaptively de-biased, the output of the teacher model is dynamically de-biased without increasing the training parameters, and more reasonable pseudo labels are obtained.

[0030] Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and / or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0031] The above and / or additional aspects and advantages of the application will become apparent and be readily appreciated from the description of the embodiments, taken in conjunction with the following drawings in which:

[0032] Figure 1 : the de-biased consistency method provided by the application has a whole framework;

[0033] Figure 2 : a global-local consistency framework;

[0034] Figure 3 ​Adaptive de-biasing module schematic diagram

[0035] Figure 4 HMDB51 and Kinetics400 distribution basic balanced dataset, the pseudo label distribution situation diagram in the training process. DETAILED DESCRIPTION

[0036] Embodiments of the present application are described in detail below, which are exemplary and intended to explain the present application, and cannot be understood as a limitation of the present application.

[0037] (I) global-local consistency

[0038] Action recognition as a representative task in the field of video understanding, video contains rich information, but also brings the problem of long sequence length, a few hundred frames of video, want to input to the model is not realistic at the same time, the efficiency is extremely low, not all information is key information, there are quite a lot of redundancy. So how to sample a long video to make efficient use of key information has also become a big difficulty. The main sampling methods in the field of video understanding are as follows:

[0039] 1. Continuous sampling

[0040] Continuous sampling is as its name implies, sampling continuous segments in a video as input information of the video. The sampling steps are: first, randomly determine a starting position, then randomly sample N frames, the time step is ts, and the fixed input quantity is N frames. The biggest advantage of this continuous sampling method is simple and efficient, and it is also the most commonly used method in semi-supervised action recognition and even video understanding.

[0041] 2. Average sampling

[0042] Average sampling is a global information acquisition method compared with continuous sampling. If a video contains T frames in total and the input segment to be sampled is N frames, the average sampling strategy is to uniformly sample N frames across the entire video, where the time step s is proportional to the actual length T of the video, i.e. Since global sampling is uniformly distributed along the time dimension, the sampled segment can cover the visual content of the entire video regardless of the duration of the video action. In contrast, the number of frames of the original video input is not fixed, and continuous sampling is limited by the fixed sampling interval. From the modeling point of view, its time coverage is still local, and it cannot capture the visual content of the entire video. For example, in the case where the time step ts and the number of sampled frames N are both 8, the sampled segment of 64 frames only accounts for a small part of a 10-second video (about 300 frames).

[0043] Current semi-supervised action recognition methods primarily use two clips obtained through two consecutive samplings as input, neglecting the overall nature of the video and the importance of global information. Therefore, in this embodiment, to enhance the spatiotemporal modeling capability of the action recognition model and enable it to simultaneously consider both global and local information, a combination of global average sampling and local continuous sampling is used for the input during model training. This results in global segments that roughly cover the entire visual content and local segments with richer details.

[0044] For a typical semi-supervised action recognition model training task, assuming there are N training video samples in total, including N... L Each labeled video (xl, yl) ∈ D L and N U An unlabeled video xu∈D U , where xl represents labeled video samples with the label yl, and xu represents unlabeled videos. D U >>D L The goal of semi-supervised action recognition model training is to fully utilize D L and D U We train the model in the hope that it can achieve better results under the condition of limited labels.

[0045] like Figure 2 As shown, the global-local consistency method used in this embodiment is based on FixMatch, a classic framework in the field of semi-supervised learning. Unlike FixMatch, which uses consistency loss between two different views, this embodiment incorporates the idea of ​​global-local consistency. In the teacher-student model, global information, which is more complete overall information, is used as the input of the teacher branch to obtain pseudo-labels, which are then used to supervise the learning of local information in the student branch. Figure 2 It can be seen that the training paradigm consists of two parts, for the labeled video set... The model is optimized under label supervision. Specifically, the labeled video xl is first sampled twice, and the global input xl is obtained by global average sampling. g The local input xl is obtained through local continuous sampling. l Similar to the usual labeled video learning method, the model is trained using the label yl, and the supervised loss Ls is defined as follows:

[0046]

[0047] Where F(.) represents the model's predicted value, and H(.) is the standard cross-entropy loss function. The trained model is used as the initial value for both the teacher and student models.

[0048] For unlabeled video xu, global average sampling is first used to obtain the segment xu. g The fragment xu is obtained through local continuous sampling. l For the fragment xu g Two views are generated using weak enhancements (e.g., random horizontal flipping, random scaling, and random cropping) and strong enhancements (e.g., AutoAugment or Dropout), respectively. gw =A weak (xu g ),xu gs =A strong (xu g A weak (.) indicates weak enhancement, A strong (.) indicates strong enhancement; while for the fragment xu l Use weak enhancement to generate view xu lw =A weak (xu l Next, use the weakly enhanced view xu of the global fragment. gw Used as input to the teacher model branch to generate pseudo-labels. Among them, F(xu) gw The output of the teacher model is represented by , which is the probability distribution of each recognition category. `argmax(.)` indicates selecting the recognition category with the highest probability. For the student model branch, the input includes a strongly augmented view of the global segment. gs And weakly enhanced view of local fragments xu lw Pseudo-labels obtained using global information This is used to supervise the student model's learning. The semi-supervised loss for unlabeled data is defined as follows:

[0049]

[0050] Where δ is a predefined threshold. Let f(x) be the indicator function, representing the maximum class probability max(F(x)) when the teacher model outputs the maximum class probability. gw When )) is greater than δ, A value of 1 indicates that the model has a certain level of confidence in the predicted value for this class, and a pseudo-label has been selected for the unlabeled data; otherwise, a value of 0 indicates a value of 0.

[0051] In the classic FixMatch framework, strong and weak augmented views share the same model. This approach is prone to model training instability and can even lead to model collapse. Therefore, this embodiment uses the exponential moving average (EMA)-Teacher within the global-local consistency framework. The teacher model predicts pseudo-labels to update the student model, where the teacher model's parameters θ...t by the student model parameters θ s is updated by an exponential moving average, i.e.

[0052] θ t ←m1θ t +(1-m1)θ s

[0053] where m1 is the first momentum coefficient, θ t and θ s are the parameters of the teacher model and the student model, respectively.

[0054] It can be seen that in the global-local consistency method, the training is divided into two stages. First, in the preheating stage, the labeled data part is used for preheating, and the label is used for supervised learning. The loss is

[0055]

[0056] In the second stage, the labeled data and the unlabeled data participate in the training at the same time in a certain proportion. The unsupervised pseudo-label consistency loss is

[0057]

[0058] The total loss is

[0059] L all =L s +γL un

[0060] where γ is the hyperparameter balancing the loss terms.

[0061] (II) Confidence-based adaptive debiasing strategy

[0062] Long-Tail Distribution is a common probability distribution pattern in statistics and data analysis. In this distribution, most samples are concentrated in a few categories (head), while the remaining large number of categories (tail) have low individual frequency but large quantity, occupying the overall "long tail".

[0063] Applicant found an interesting phenomenon through a large number of experiments: even if the current adopted dataset is class-balanced, that is, the number of samples of each class is roughly the same, and the labeled data is also balanced, and at the same time the pre-training dataset of the model is also balanced data, in the process of semi-supervised training based on pseudo-labels, the model shows a serious bias problem for high-confidence predictions of unlabeled data, that is, in the training process, the model's predicted pseudo-labels show a classic long-tail distribution, as shown in Figure 4 Under the condition that the pre-training dataset and the labeled data are roughly balanced, in the process of semi-supervised training based on pseudo-labels, the model shows a very obvious bias state for high-confidence predictions of unlabeled data. That is, due to the similarity of the internal data, the pseudo-labels obtained by the semi-supervised model are inherently unbalanced. Since the student model will implicitly inherit the unbalanced pseudo-labels during training, it will further strengthen the bias of the teacher model. Once the confused samples are incorrectly labeled as pseudo-labels, this error is almost impossible to correct itself. It can even mislead the model's learning, further amplifying the existing bias, and thus producing more incorrect predictions. For this pseudo-label long-tail distribution problem, the most intuitive solution is resampling or undersampling, but resampling often exacerbates model overfitting, leading to insufficient generalization ability of the model; while undersampling method will lead to the loss of information of the head class, thus affecting the original high-precision class and reducing the precision.

[0064] To solve the above problems, the embodiment introduces a debiasing algorithm to alleviate the bias in the model training process, which can improve the accuracy of the tail class to some extent while ensuring the high-precision class. As shown in Figure 3 The bias vector is introduced to debias the output of the teacher model:

[0065] An all-1 bias vector with a dimension equal to the number of recognition classes is initialized In the training process, the following formula is used:

[0066]

[0067] The bias vector is updated with momentum, where m2 is the second momentum coefficient, μB is the number of samples in a training batch, and p k is the output of the teacher model for the kth sample, which is the probability distribution of each recognition class.

[0068] In the process of using the bias vector Debiasing the output of the teacher model:

[0069]

[0070] where λ is the debiasing parameter.

[0071] Using the bias vector The principle of debiasing the output of the teacher model is that due to the large number of tail classes being misclassified into the head classes, the number distribution of pseudo labels presents a long-tail distribution. If the bias of the model is abstracted as a vector, then this bias vector must be positively correlated with the number of pseudo labels of each class, which explains the above Through the momentum update process, the bias vector will present a distribution positively correlated with the pseudo labels, so through the debiasing module, the model's original prediction is fine-tuned through to make the model more biased towards the tail classes to some extent.

[0072] As for the debiasing parameter λ, it controls the strength of the indirect effect, that is, it controls the degree of debiasing. If the debiasing parameter is too strong, the model will be difficult to fit the data, and too small a debiasing parameter can hardly eliminate the bias, which will ultimately damage the generalization ability. In the process of experiment, we found that the model is very sensitive to the choice of debiasing parameter, that is, the effect of the debiasing module is greatly affected by the hyperparameter λ, but it is difficult to tune the parameter. Therefore, the debiasing algorithm is further improved in this embodiment, and an adaptive debiasing parameter is introduced. Specifically, the debiasing parameter is determined by the prediction of the current sample and the bias vector. Since the prediction of the head class is filled with a large number of misclassified tail samples, this embodiment hopes to introduce a greater debiasing degree when predicting the head class. The specific design is as follows:

[0073]

[0074] where represents the mean of the bias vector , argmax(p k ) represents the recognition class with the maximum probability in p k , and represents the element value corresponding to argmax(p k ) in the bias vector .

[0075] For the current calculation and analysis of the bias parameter, the ratio of the bias of the current predicted class to the average bias represents the number of the current predicted class. The larger the value, the closer the current predicted class is to the head class. The smaller the value, the closer the current predicted class is to the tail class. The introduction of the adaptive debiasing degree λ makes the current prediction more debiased when predicting the head class, so as to obtain a more reasonable label.

[0076] (III) Overall framework of the debiasing consistency method

[0077] Based on the above two parts, as shown in Figure 1 The overall framework of the model training of the debiasing consistency method of the embodiment is as follows:

[0078] In the warm-up stage, the model is fitted using all the labeled data. In the later semi-supervised training stage, the labeled data and the unlabeled data participate in the training in the ratio of N l :N u The framework uses a student-teacher model to momentum update the model parameters to ensure the stability of the training process. In the unlabeled data part, the weakly enhanced global information is used as the input of the teacher model, and the input of the student model is composed of two parts, which are the strongly enhanced global input and the weakly enhanced local input. The consistency of the global semantic information and the local semantic information is used to complete the training of the unlabeled data. And the debiasing strategy based on confidence is used to debias the output of the teacher model branch: due to the existence of model bias, the bias of the model is abstracted into a vector, and the output of the teacher model is used for momentum update, and an adaptive debiasing module is introduced to dynamically obtain a "unbiased" prediction, so as to obtain a fine-tuned pseudo label to train the student model.

[0079] Based on this, the debiasing consistency semi-supervised human action recognition method proposed in the embodiment finally includes the following steps:

[0080] Step 1: sampling the video to be recognized; the sampling here can be any sampling method, such as average sampling or continuous sampling.

[0081] Step 2: inputting the frame sequence obtained by sampling in step 1 into the trained recognition model to obtain the recognition result;

[0082] The recognition model is trained by the following process:

[0083] Step 2.1: obtaining training samples, the training samples including a small amount of labeled videos and a large amount of unlabeled videos; specifically, including N L labeled videos (xl,yl)∈D L and N U unlabeled videos xu∈D Uwhere xl is the labeled video sample with label yl, xu is the unlabeled video, D L is the set of labeled videos, D U is the set of unlabeled videos, D U > D L ;

[0084] Step 2.2: Sample the labeled videos and train using the labels to obtain a trained model as the initial value of the teacher model and the student model; specifically, sample the labeled video xl twice, obtain global input xl g by global average sampling, and obtain local input xl l by local continuous sampling, and train using the label yl;

[0085] Step 2.3: Perform global average sampling and local continuous sampling on the unlabeled videos, and perform weak enhancement and strong enhancement on the segments obtained by global average sampling, and perform weak enhancement on the segments obtained by local continuous sampling; specifically, for the unlabeled video xu, obtain segment xu g by global average sampling, and obtain segment xu l by local continuous sampling; for segment xu g , generate two views xu gw = A weak (xu g ), xu gs = A strong (xu g ) using weak enhancement and strong enhancement respectively, where A weak (.) represents weak enhancement, and A strong (.) represents strong enhancement; for segment xu l , generate view xu lw = A weak (xu l ) using weak enhancement;

[0086] Step 2.4: Use the view after weak enhancement of the global average sampling segment as the input of the teacher model to obtain the output of the teacher model, and de-bias the output of the teacher model according to the formula

[0087]

[0088] ; where p k is the output of the teacher model for the kth input sample, p is the de-biased output of the teacher model, b is the bias vector, and λ is the de-biasing parameter;

[0089] Further, the bias vector b is ​The initial value is a full 1 vector with the same dimension as the number of identification categories, and in the training process, the formula

[0090]

[0091] Momentum update is performed, where m2 is a second momentum coefficient, and μB is the number of samples in a training batch;

[0092] Further, the debiasing parameter λ is an adaptive debiasing parameter determined by the prediction of the current sample and the bias vector, specifically:

[0093]

[0094] wherein represents taking the mean of the bias vector argmax(p k ) represents taking the identification category with the maximum probability in p k represents the element value corresponding to argmax(p k ) in the bias vector

[0095] Step 2.5: Use the output of the debiased teacher model as pseudo-label pairs to supervise the learning of the student model, wherein the input of the student model is the globally averaged sampling segment strongly enhanced view and the locally continuous sampling segment weakly enhanced view;

[0096] Further, the student model is updated by the pseudo-label predicted by the teacher model, wherein the parameter θ t of the teacher model is updated by the exponential moving average of the parameter θ s of the student model:

[0097] θ t ←m1θ t +(1-m1)θ s

[0098] wherein m1 is a first momentum coefficient, and θ t and θ s are the parameters of the teacher model and the student model, respectively;

[0099] Step 2.6: After reaching the set number of training iterations, the trained identification model is obtained.

[0100] Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments without departing from the principles and purposes of the present application within the scope of the present application.​​

Claims

1. A semi-supervised action recognition method for biased consistency removal, characterized in that: Includes the following steps: Step 1: Sample the video to be identified; Step 2: Input the frame sequence obtained from Step 1 into the trained recognition model to obtain the recognition result; The recognition model is trained through the following process: Step 2.1: Obtain training samples, which include a labeled video set D. L and unlabeled video collection D U ; where D U >>D L ; Step 2.2: Sample the labeled videos and use the labels to train the model, and use the trained model as the initial value for the teacher model and the student model; Step 2.3: Perform global average sampling and local continuous sampling on the unlabeled video, and perform weak enhancement and strong enhancement on the segments obtained by global average sampling, and weak enhancement on the segments obtained by local continuous sampling; Step 2.4: Use the weakly enhanced view of the globally averaged sampled segment as input to the teacher model, obtain the output of the teacher model, and then apply the formula... The output of the teacher model is debiased; where p k The output of the teacher model for the k-th input sample. The output of the teacher model after bias removal. Let λ be the bias vector, and λ be the debiasing parameter. Step 2.5: Use the debiased teacher model output as pseudo-label pairs to supervise the student model's learning. The student model's input consists of a strongly enhanced view of a globally averaged sampled segment and a weakly enhanced view of a locally continuous sampled segment. Step 2.6: After reaching the set number of training iterations, the trained recognition model is obtained.

2. The semi-supervised action recognition method for biased consistency removal according to claim 1, characterized in that: In step 1, average sampling or continuous sampling is used to sample the video to be identified.

3. The semi-supervised action recognition method for biased consistency removal according to claim 1, characterized in that: In step 2.1, there is a tagged video set D. L Including N L Each labeled video (xl, yl) ∈ D L Unlabeled video collection D U Including N U An unlabeled video xu∈D U , where xl is a labeled video sample with the label yl, and xu is an unlabeled video sample.

4. The semi-supervised action recognition method for biased consistency removal according to claim 1, characterized in that: In step 2.2, the labeled video xl is sampled twice, and the global input xl is obtained by global average sampling. g The local input xl is obtained through local continuous sampling. l Training is performed using the label yl.

5. The semi-supervised action recognition method for biased consistency removal according to claim 1, characterized in that: In step 2.3, for the unlabeled video xu, global average sampling is used to obtain the segment xu. g The fragment xu is obtained through local continuous sampling. l For the fragment xu g Two views are generated using weak enhancement and strong enhancement respectively. gw =A weak (xu g ),xu gs =A strong (xu g A weak (.) indicates weak enhancement, A strong (.) indicates strong enhancement; For the fragment xu l Use weak enhancement to generate view xu lw =A weak (xu l ).

6. The semi-supervised action recognition method for biased consistency removal according to claim 1, characterized in that: In step 2.4, the bias vector The initial value is a vector of all 1s with the same dimension as the number of categories to be identified. During training, according to the formula... Momentum updates are performed, where m2 is the second momentum coefficient and μB is the number of samples in a training batch.

7. The semi-supervised action recognition method for biased consistency removal according to claim 1, characterized in that: In step 2.4, the bias removal parameter λ is an adaptive bias removal parameter, which is jointly determined by the prediction and bias vector of the current sample, specifically as follows: in Indicates taking the bias vector The mean, argmax(p) k ) indicates taking p k The category with the highest probability of being identified. Indicates taking the bias vector argmax(p) k The element value corresponding to ).

8. The semi-supervised action recognition method for biased consistency removal according to claim 1, characterized in that: In step 2.5, the pseudo-labels predicted by the teacher model are used to update the student model, where the parameter θ of the teacher model... t Through student model parameters θ s Exponential moving average update: i t ←m1θ t +(1-m1)θ s Where m1 is the first momentum coefficient, θ t and θ s These are the parameters for the teacher model and the student model, respectively.

Images