Depolarization consistency semi-supervised action recognition method

Abstract

The invention provides a depolarization consistency semi-supervised action recognition method, which comprises the following steps of: acquiring global information of a video by using global average sampling, acquiring local information of the video by using local continuous sampling, and performing model training according to the fact that the local information and the global information of the video are kept consistent semantically; global information and local information of the video are fully utilized, more sufficient modeling is performed on the time sequence of the action, and the time sequence modeling capability and robustness of the model are enhanced; and aiming at the problem of long tail distribution of pseudo labels which are not concerned by a traditional method, adaptive depolarization processing is carried out on the output of the teacher model, and dynamic depolarization is carried out on the output of the teacher model on the premise of not increasing training parameters, so that more reasonable pseudo labels are obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of human action recognition, and in particular to a debiased consistency semi-supervised human action recognition method. Background Art

[0002] Action recognition plays an important role in the field of computer vision. It is not only an important development direction in the field, but also a key component of video understanding. Recent developments in action recognition have opened the door to a range of real-world applications, including but not limited to visual security systems, behavioral research, sports analysis, and elderly fall detection systems.

[0003] In the field of action recognition, one of the most common practices is to first pre-train the model on a large-scale dataset (for example, Kinetics contains up to 650,000 video clips), and then fine-tune it on a small downstream dataset to achieve better performance. However, since it is time-consuming and expensive to annotate large-scale datasets, training models on large-scale datasets with complete annotations is hindered; on the other hand, about 500 hours of video are uploaded to YouTube every minute around the world, which naturally provides a large amount of unlabeled data; at the same time, semi-supervised learning and self-supervised learning have achieved good results in traditional image tasks. So a natural idea is to use a semi-supervised model to make full use of large-scale unlabeled datasets with acceptable costs.

[0004] Many fruitful research results have been achieved in the field of semi-supervised action recognition, which can be divided into three categories: semi-supervised action recognition based on pseudo labels, semi-supervised action recognition combined with self-supervision, and semi-supervised action recognition based on multimodality. Summarizing the current status of semi-supervised model research, we can find that there are mainly the following two problems:

[0005] 1. Most of the current semi-supervised action recognition methods use two clips as input through two consecutive samplings, such as the SVFormer and TimeBalance methods that have achieved good experimental results, but they all ignore the integrity of the video and the importance of the global information of the video; in addition, a small number of methods use global sampling, such as the TCL method that uses the video playback speed to perform fast and slow global sampling without changing the semantic information of the video, but the method inevitably ignores the role of local information. It can be seen that action recognition is a typical video task, and the current semi-supervised action recognition methods generally do not take into account the essential attributes of the video and do not make full use of the dynamic spatiotemporal information of the video.

[0006] 2. Current research in the field of semi-supervised action recognition has not paid attention to the serious bias tendency of the model during training, which makes the pseudo-labels predicted by the model show a very obvious long-tail distribution. The applicant has found through a large number of experiments that in the field of semi-supervised action recognition, even if the data set is evenly distributed, the distribution of pseudo-labels will show significant uneven distribution during the training process based on the semi-supervised framework FixMatch. Summary of the invention

[0007] In view of the problems existing in the prior art, the present invention proposes a debiased consistency semi-supervised action recognition method, which specifically includes the following steps:

[0008] Step 1: Sample the video to be identified; the sampling here can be any sampling method, such as average sampling and continuous sampling.

[0009] Step 2: Input the frame sequence sampled in step 1 into the trained recognition model to obtain the recognition result;

[0010] The recognition model is trained through the following process:

[0011] Step 2.1: Obtain training samples, which include a small number of labeled videos and a large number of unlabeled videos; specifically: including N L There are labeled videos (xl,yl)∈D L and N U There are unlabeled videos xu∈D U , where xl is a labeled video sample, its label is yl, xu is an unlabeled video sample, D L is a labeled video set, D U is the unlabeled video set, D U >>D L ;

[0012] Step 2.2: Sample the labeled videos and use the labels for training to obtain the trained model as the initial value of the teacher model and the student model; specifically, sample the labeled video xl twice, and obtain the global input xl through global average sampling g , local input xl is obtained by local continuous sampling l , use label yl for training;

[0013] Step 2.3: Perform global average sampling and local continuous sampling on the unlabeled video, and perform weak and strong enhancement on the segments obtained by global average sampling, and perform weak enhancement on the segments obtained by local continuous sampling; specifically, for the unlabeled video xu, global average sampling is used to obtain the segment xu g , the fragment xu is obtained by local continuous sampling l; For the fragment xu g , using weak enhancement and strong enhancement to generate two views respectively, xu gw =A weak (xu g ),xu gs =A strong (xu g ), A weak (.) indicates weak enhancement, A strong (.) indicates strong enhancement; for the fragment xu l , using weak enhancement to generate view xu lw =A weak (xu l );

[0014] Step 2.4: Use the weakly enhanced view of the global average sampled fragment as the input of the teacher model, get the output of the teacher model, and use the formula

[0015]

[0016] Debias the output of the teacher model; where p k is the output of the teacher model for the kth input sample, is the teacher model output after debiasing, is the bias vector, λ is the debiasing parameter;

[0017] Furthermore, the bias vector The initial value is a vector of all 1s with the same dimension as the number of recognized categories. During the training process, according to the formula

[0018]

[0019] Perform momentum update, where m2 is the second momentum coefficient and μB is the number of samples in a training batch;

[0020] Furthermore, the debiasing parameter λ adopts an adaptive debiasing parameter, which is determined by the prediction of the current sample and the bias vector, specifically:

[0021]

[0022] in Represents the bias vector The mean of argmax(p k ) means taking p k The most likely identification category, Represents the bias vector argmax(p k ) corresponding element value;

[0023] Step 2.5: Use the debiased teacher model output as a pseudo-label pair to supervise the student model for learning, where the input of the student model is the strongly enhanced view of the global average sampled fragment and the weakly enhanced view of the local continuous sampled fragment;

[0024] Furthermore, the teacher model predicts pseudo labels to update the student model, where the parameters θ of the teacher model are t The student model parameters θ s Exponential moving average update:

[0025] θ t ←m1θ t +(1-m1)θ s

[0026] Where m1 is the first momentum coefficient, θ t and θ s are the parameters of the teacher model and the student model respectively;

[0027] Step 2.6: After reaching the set number of training iterations, the trained recognition model is obtained.

[0028] Beneficial Effects

[0029] The debiasing consistency semi-supervised human action recognition method proposed in the present invention uses global average sampling to obtain the global information of the video, and uses local continuous sampling to obtain the local information of the video. Model training is performed based on the semantic consistency between the local information of the video and the global information, and the global and local information of the video are fully utilized to more fully model the timing of the action, thereby enhancing the timing modeling capability and robustness of the model. In addition, in view of the long-tail distribution problem of pseudo-labels that traditional methods have not paid attention to, the output of the teacher model is adaptively debiased, and the output of the teacher model is dynamically debiased without increasing training parameters to obtain more reasonable pseudo-labels.

[0030] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0032] Figure 1 : The overall framework of the debiasing consistency method proposed in this invention;

[0033] Figure 2 : Global-local consistency framework;

[0034] Figure 3: Schematic diagram of adaptive debiasing module;

[0035] Figure 4 : Pseudo-label distribution diagram of HMDB51 and Kinetics400 during training for the basically balanced datasets. DETAILED DESCRIPTION

[0036] Embodiments of the present invention are described in detail below. The embodiments are exemplary and intended to be used to explain the present invention, but should not be construed as limiting the present invention.

[0037] 1. Global-local consistency

[0038] Action recognition is a representative task in the field of video understanding. While videos contain rich information, they also bring the problem of long sequence length. It is not realistic to input all of a video with hundreds of frames into the model, and the efficiency is extremely low. Not all information is important information, and there is a lot of redundancy. Therefore, how to sample a long video so that it can efficiently use the key information has become a major difficulty. There are several main sampling methods in the current field of video understanding:

[0039] 1. Continuous sampling

[0040] Continuous sampling, as the name implies, is sampling continuous segments in a video as the video input information. The sampling steps are: first randomly determine a starting position, then randomly sample N frames, the sampling time step is ts, and the fixed input number is N frames. The biggest advantage of this continuous sampling method is that it is simple and efficient. It is also the most used method in semi-supervised action recognition and even video understanding.

[0041] 2. Average sampling

[0042] Compared with continuous sampling, average sampling is a global information acquisition. If a video contains a total of T frames, and the input segments to be sampled are N frames, the average sampling strategy is to uniformly sample N frames across the entire video span, where the time step s is proportional to the actual length T of the video, that is, Since global sampling is uniformly distributed along the time dimension, the sampled segments always roughly cover the visual content of the entire video, no matter how long the video action lasts. In contrast, the number of frames in the original video input is not fixed, and continuous sampling is limited by the fixed sampling interval. From a modeling perspective, its temporal coverage is still partial and cannot capture the visual content of the entire video. For example, when the time step ts and the number of sampled frames N are both 8, a sampled segment that covers 64 frames only occupies a small part of a 10-second video (about 300 frames).

[0043] The main semi-supervised action recognition methods currently use two clips as inputs through two consecutive samplings, which ignore the integrity of the video and the importance of the global information of the video. Therefore, in this embodiment, in order to increase the spatiotemporal modeling capability in the action recognition model so that it can pay attention to both global information and local information, a combination of global average sampling and local continuous sampling is used for the input of the model training, so as to obtain a global clip that roughly covers the entire visual content and a local clip with richer local detail information.

[0044] For a typical semi-supervised action recognition model training task, assume that there are N training video samples, including N L There are labeled videos (xl,yl)∈D L and N U There are unlabeled videos xu∈D U , where xl is a labeled video sample, its label is yl, and xu is an unlabeled video. U >>D L The goal of the semi-supervised action recognition model training task is to make full use of D L and D U To train the model, we hope that the model can achieve better results under the condition of limited labels.

[0045] like Figure 2 As shown, the global-local consistency method used in this embodiment is based on FixMatch, a classic framework in the field of semi-supervised learning. Unlike FixMatch, which uses consistency loss between two different views, the method of this embodiment incorporates the idea of ​​global-local consistency. In the teacher-student model, the global information with more complete overall information is used as the input of the teacher branch to obtain pseudo labels to supervise the local information of the student branch for learning. Figure 2 It can be seen that the training paradigm is divided into two parts. For the labeled video set The model is supervised and optimized by labels. Specifically, the labeled video xl is sampled twice, and the global input xl is obtained by global average sampling. g , local input xl is obtained by local continuous sampling l , which is consistent with the common labeled video learning method, the model is trained using the label yl, so the supervision loss Ls is defined as follows:

[0046]

[0047] Where F(.) is the predicted value of the model, and H(.) is the standard cross entropy loss function. The trained model is used as the initial value of the teacher model and the student model.

[0048] For unlabeled video xu, firstly, global average sampling is used to obtain the segment xu g , the fragment xu is obtained by local continuous sampling l For the fragment xu g , two views are generated using weak enhancement (e.g., random horizontal flipping, random scaling, and random cropping) and strong enhancement (e.g., AutoAugment or Dropout), respectively. gw =A weak (xu g ),xu gs =A strong (xu g ), A weak (.) indicates weak enhancement, A strong (.) indicates strong enhancement; while for the fragment xu l , using weak enhancement to generate view xu lw =A weak (xu l ). Then, the weakly enhanced view xu of the global fragment is used gw As input to the teacher model branch to generate pseudo labels Among them, F(xu gw ) represents the output of the teacher model, which is the probability distribution of each recognition category, and argmax(.) represents the recognition category with the largest probability. For the student model branch, the input includes the strongly enhanced view xu of the global fragment gs and the weakly enhanced view xu of the local fragment lw , pseudo labels obtained using global information To supervise the student model to learn. The semi-supervised loss for unlabeled data is defined as follows:

[0049]

[0050] Where δ is a predefined threshold, is the indicator function, indicating that when the teacher model outputs the maximum class probability max(F(x gw )) is greater than δ, The value is 1, which means that the model has a certain degree of confidence in the predicted value of this class and selects a pseudo label for the unlabeled data; otherwise, it is 0.

[0051] In the classic FixMatch framework, the strong augmentation view and the weak augmentation view share the same model. This method is prone to unstable model training and even model collapse. Therefore, this embodiment uses the exponential moving average (EMA)-Teacher in the global local consistency method framework. The teacher model predicts the pseudo label to update the student model, where the parameter θ of the teacher model ist The student model parameters θ s The exponential moving average update of

[0052] θ t ←m1θ t +(1-m1)θ s

[0053] Where m1 is the first momentum coefficient, θ t and θ s are the parameters of the teacher model and the student model respectively.

[0054] It can be seen that in the global-local consistency method, the training is divided into two stages. First, in the warm-up stage, the labeled data is used for warm-up, and the labels are used for supervised learning. The loss is

[0055]

[0056] In the second stage, labeled data and unlabeled data are trained simultaneously in a certain ratio, and the unsupervised pseudo-label consistency loss is

[0057]

[0058] The total loss is

[0059] L all =L s +γL un

[0060] Where γ is a hyperparameter of the balanced loss term.

[0061] 2. Confidence-based Adaptive Debiasing Strategy

[0062] Long-Tail Distribution is a common probability distribution pattern in statistics and data analysis. In this distribution, most of the samples are concentrated in a few categories (head), while the remaining large number of categories (tail) have a low frequency of a single category, but their number is huge, occupying the overall "long tail".

[0063] The applicant used common datasets and common settings for semi-supervised action recognition tasks, with 40%, 50% and 60% labeled data for HMDB51, 1% and 10% labeled data for UCF101, and 1% and 10% labeled data for Kinetics400. After a large number of experiments, an interesting phenomenon was discovered: even if the dataset currently used is class-balanced, that is, the number of samples in each category is roughly the same, the labeled data is also balanced, and the pre-training dataset of the model is also balanced data, in the semi-supervised training process based on pseudo-labels, the model's high-confidence prediction of unlabeled data shows a serious bias problem, that is, during the training process, the pseudo-labels predicted by the model show a classic long-tail distribution pattern, such as Figure 4 As shown in the figure, when the pre-training data set and the labeled data are roughly balanced, the model's high-confidence prediction of the unlabeled data in the semi-supervised training process based on pseudo-labels shows a very obvious biased state. In other words, due to the similarity of the intrinsic data, the pseudo-labels obtained by the semi-supervised model are inherently unbalanced. Since the student model will implicitly inherit the unbalanced pseudo-labels during the training process, thereby reinforcing the bias of the teacher model, once the confused samples are mistakenly labeled as pseudo-labels, this error is almost impossible to correct itself. It may even mislead the learning of the model, further amplify the existing bias, and produce more wrong predictions. For this long-tail distribution problem of pseudo-labels, the first and most intuitive solution is resampling or undersampling methods, but resampling often aggravates the overfitting of the model, resulting in insufficient generalization of the model; and the undersampling method will cause the loss of information of the head class, thereby affecting the original high-precision class and reducing the accuracy.

[0064] In view of the above problems, this embodiment introduces a debiasing algorithm to alleviate the deviation in the model training process, which can improve the accuracy of the tail class to a certain extent while ensuring the high-precision class. Figure 3 As shown, by introducing the bias vector Debias the output of the teacher model:

[0065] Initialize a bias vector of all 1s with the same dimension as the number of recognition categories During training, the following formula is used:

[0066]

[0067] Bias vector Perform momentum update, where m2 is the second momentum coefficient, μB is the number of samples in a training batch, and p k is the output of the teacher model for the kth sample, which is the probability distribution of each recognition category.

[0068] Using bias vectors Debias the output of the teacher model:

[0069]

[0070] Where λ is the debiasing parameter.

[0071] Exploiting bias vectors The principle of debiasing the output of the teacher model is that a large number of tail classes are misclassified as head classes, resulting in a long-tail distribution of pseudo labels. If the bias of the model is abstracted into a vector, then this bias vector must be positively correlated with the number of pseudo labels of each category, which explains the above Through the momentum update process, the bias vector It will show a distribution that is positively correlated with the pseudo-label, so through the debiasing module, the original prediction of the model is Fine-tune the model to be more biased towards the tail class to some extent.

[0072] As for the debiasing parameter λ, it controls the strength of the indirect effect, that is, it controls the degree of debiasing. If the debiasing parameter is too strong, it is difficult for the model to fit the data, and a debiasing parameter that is too small can hardly eliminate the bias, which will eventually damage the generalization ability. In the process of conducting experiments, we found that the model is very sensitive to the choice of debiasing parameters, that is, the effect of the debiasing module is greatly affected by the hyperparameter λ, but it is difficult to adjust its parameters. Therefore, this embodiment further improves the debiasing algorithm and introduces an adaptive debiasing parameter. Specifically, the debiasing parameter is determined by the prediction of the current sample and the bias vector. Since the prediction of the head class is full of a large number of misclassified tail samples, this embodiment hopes to introduce a greater degree of debiasing when predicting the head class. The specific design is as follows:

[0073]

[0074] in Represents the bias vector The mean of argmax(p k ) means taking p k The most likely identification category, Represents the bias vector argmax(p k ) corresponding to the element value.

[0075] From the calculation and analysis of the current debiasing parameters, we can see that the ratio of the bias corresponding to the current predicted category to the average bias reflects the number of the current predicted categories. The larger the value, the closer the current predicted category is to the head category, and the smaller the value, the closer the current predicted category is to the tail category. The introduction of the adaptive debiasing degree λ allows the current prediction to be debiased to a greater extent when the prediction is the head category, thereby obtaining a more reasonable label.

[0076] 3. Overall framework of the debiasing consistency method

[0077] Based on the above two parts, Figure 1 The overall framework of model training of the debiasing consistency method of this embodiment is shown as follows:

[0078] In the warm-up phase, the model uses all labeled data for fitting. In the later semi-supervised training phase, labeled data and unlabeled data are divided into N l :N u The framework uses a student-teacher model to momentum update the model parameters to ensure the stability of the training process. In the unlabeled data part, the weakly enhanced global information is used as the input of the teacher model. The input of the student model consists of two parts, namely the global input after strong enhancement and the local input after weak enhancement. The consistency of global semantic information and local semantic information is used to complete the training of unlabeled data. And the confidence-based debiasing strategy is used to debias the output of the teacher model branch: due to the existence of model bias, the bias of the model is abstracted into a vector, and the output of the teacher model is used for momentum update. At the same time, an adaptive debiasing module is introduced to dynamically obtain "unbiased" predictions, so as to obtain a fine-tuned pseudo-label to train the student model.

[0079] Based on this, the debiased consistency semi-supervised human action recognition method proposed in this embodiment finally includes the following steps:

[0080] Step 1: Sample the video to be identified; the sampling here can be any sampling method, such as average sampling and continuous sampling.

[0081] Step 2: Input the frame sequence sampled in step 1 into the trained recognition model to obtain the recognition result;

[0082] The recognition model is trained through the following process:

[0083] Step 2.1: Obtain training samples, which include a small number of labeled videos and a large number of unlabeled videos; specifically: including N L There are labeled videos (xl,yl)∈D L and N U There are unlabeled videos xu∈D U, where xl is a labeled video sample, its label is yl, xu is an unlabeled video, D L is a labeled video set, D U is the unlabeled video set, D U >>D L ;

[0084] Step 2.2: Sample the labeled videos and use the labels for training to obtain the trained model as the initial value of the teacher model and the student model; specifically, sample the labeled video xl twice, and obtain the global input xl through global average sampling g , local input xl is obtained by local continuous sampling l , use label yl for training;

[0085] Step 2.3: Perform global average sampling and local continuous sampling on the unlabeled video, and perform weak and strong enhancement on the segments obtained by global average sampling, and perform weak enhancement on the segments obtained by local continuous sampling; specifically, for the unlabeled video xu, global average sampling is used to obtain the segment xu g , the fragment xu is obtained by local continuous sampling l ; For the fragment xu g , using weak enhancement and strong enhancement to generate two views respectively, xu gw =A weak (xu g ),xu gs =A strong (xu g ), A weak (.) indicates weak enhancement, A strong (.) indicates strong enhancement; for the fragment xu l , using weak enhancement to generate view xu lw =A weak (xu l );

[0086] Step 2.4: Use the weakly enhanced view of the global average sampled fragment as the input of the teacher model, get the output of the teacher model, and use the formula

[0087]

[0088] Debias the output of the teacher model; where p k is the output of the teacher model for the kth input sample, is the teacher model output after debiasing, is the bias vector, λ is the debiasing parameter;

[0089] Furthermore, the bias vector The initial value is a vector of all 1s with the same dimension as the number of recognized categories. During the training process, according to the formula

[0090]

[0091] Perform momentum update, where m2 is the second momentum coefficient and μB is the number of samples in a training batch;

[0092] Furthermore, the debiasing parameter λ adopts an adaptive debiasing parameter, which is determined by the prediction of the current sample and the bias vector, specifically:

[0093]

[0094] in Represents the bias vector The mean of argmax(p k ) means taking p k The most likely identification category, Represents the bias vector argmax(p k ) corresponding element value;

[0095] Step 2.5: Use the debiased teacher model output as a pseudo-label pair to supervise the student model for learning, where the input of the student model is the strongly enhanced view of the global average sampled fragment and the weakly enhanced view of the local continuous sampled fragment;

[0096] Furthermore, the teacher model predicts pseudo labels to update the student model, where the parameters θ of the teacher model are t The student model parameters θ s Exponential moving average update:

[0097] θ t ←m1θ t +(1-m1)θ s

[0098] Where m1 is the first momentum coefficient, θ t and θ s are the parameters of the teacher model and the student model respectively;

[0099] Step 2.6: After reaching the set number of training iterations, the trained recognition model is obtained.

[0100] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention without departing from the principles and intent of the present invention.

Claims

1. A debiased consistency semi-supervised action recognition method, characterized by: The following steps are involved: Step 1: Sample the video to be identified; Step 2: Input the frame sequence sampled in step 1 into the trained recognition model to obtain the recognition result; The recognition model is trained through the following process: Step 2.1: Obtain training samples, which include a labeled video set D L and the unlabeled video set D U ; where D U >>D L ; Step 2.2: Sample the labeled videos and use the labels for training to obtain the trained models as the initial values ​​of the teacher model and the student model; Step 2.3: Perform global average sampling and local continuous sampling on the unlabeled video, and perform weak enhancement and strong enhancement on the segments obtained by global average sampling, and perform weak enhancement on the segments obtained by local continuous sampling; Step 2.4: Use the weakly enhanced view of the global average sampled fragment as the input of the teacher model, get the output of the teacher model, and use the formula Debias the output of the teacher model; where p k is the output of the teacher model for the kth input sample, is the teacher model output after debiasing, is the bias vector, λ is the debiasing parameter; Step 2.5: Use the debiased teacher model output as a pseudo-label pair to supervise the student model for learning, where the input of the student model is the strongly enhanced view of the global average sampled fragment and the weakly enhanced view of the local continuous sampled fragment; Step 2.6: After reaching the set number of training iterations, the trained recognition model is obtained.

2. A method for debiasing consistency semi-supervised action recognition according to claim 1, characterized in that: In step 1, the video to be identified is sampled using average sampling or continuous sampling.

3. A method for debiasing consistency semi-supervised action recognition according to claim 1, characterized in that: In step 2.1, there is a labeled video set D L Including N L There are labeled videos (xl,yl)∈D L , unlabeled video set D U Including N U There are unlabeled videos xu∈D U , where xl is a labeled video sample, its label is yl, and xu is an unlabeled video sample.

4. A method for debiasing consistency semi-supervised action recognition according to claim 1, characterized in that: In step 2.2, the labeled video xl is sampled twice, and the global input xl is obtained by global average sampling. g , local input xl is obtained by local continuous sampling l , trained using label yl.

5. The method for debiasing consistency semi-supervised action recognition according to claim 1, characterized in that: In step 2.3, for the unlabeled video xu, global average sampling is used to obtain the segment xu g , the fragment xu is obtained by local continuous sampling l ; For the fragment xu g , using weak enhancement and strong enhancement to generate two views respectively, xu gw =A weak (xu g ),xu gs =A strong (xu g ), A weak (.) indicates weak enhancement, A strong (.) indicates strong enhancement; For the fragment l , using weak enhancement to generate view xu lw =A weak (xu l ).

6. According to claim 1, a method for semi-supervised action recognition based on bias-free consistency is characterized by: In step 2.4, the bias vector The initial value is a vector of all 1s with the same dimension as the number of recognized categories. During the training process, according to the formula Perform momentum update, where m2 is the second momentum coefficient and μB is the number of samples in a training batch.

7. The method for debiasing consistency semi-supervised action recognition according to claim 1, characterized in that: In step 2.4, the debiasing parameter λ adopts an adaptive debiasing parameter, which is determined by the prediction of the current sample and the bias vector, specifically: in Represents the bias vector The mean of argmax(p k ) means taking p k The most likely identification category, Represents the bias vector argmax(p k ) corresponding to the element value.

8. The method for debiasing consistency semi-supervised action recognition according to claim 1, characterized in that: In step 2.5, the teacher model predicts the pseudo-label to update the student model, where the parameter θ of the teacher model is t The student model parameters θ s Exponential moving average update: i t ←m1θ t +(1-m1)θ s Where m1 is the first momentum coefficient, θ t and θ s are the parameters of the teacher model and the student model respectively.

Images