A video classification method, device, terminal and storage medium

Abstract

The present application is applicable to the field of computer technology, and provides a method, device, terminal and storage medium for video classification. The video classification method includes: obtaining multiple video frames corresponding to the video to be processed; inputting the multiple video frames into the video classification model for processing, and obtaining the classification results corresponding to the video to be processed. The video classification model is obtained by training the initial classifier network based on multiple labeled videos and a large number of pseudo-labeled videos, and there will be no overfitting phenomenon. Therefore, when the video classification model is used to classify the video, the classification result is more accurate, thereby improving the accuracy of the video classification. Among them, a large number of pseudo-labeled videos are generated based on multiple unlabeled videos, an initial generator network, an initial classifier network and an initial discriminator network.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular, relates to a method, device, terminal and storage medium for video classification. Background Art

[0002] Video classification refers to the identification of video content to determine the category to which the video belongs. It has important applications in video search, online video viewing, video content review, video recommendation and other service scenarios.

[0003] Traditional video classification models are trained based on sample videos labeled with video categories. Due to the limited sample videos available for training video classification models, the trained video classification models are prone to overfitting. When using an overfitted video classification model to classify videos, the video classification results will be inaccurate.

[0004] The overfitting of the video classification model refers to the overfitting of the classification results predicted by the video classification model with the classification results of the sample video. When the video classification model processes a video similar to the sample video, the processing result is more accurate, but when it processes a video that is not similar to the sample video, the processing result is inaccurate. Summary of the invention

[0005] In view of this, embodiments of the present application provide a method, device, terminal and storage medium for video classification to solve the problem of inaccurate video classification results of traditional video classification models.

[0006] In a first aspect, a method for video classification is provided, wherein the execution subject of the method for video classification is a terminal, and the terminal includes but is not limited to mobile terminals such as smart phones, tablet computers, computers, personal digital assistants (PDAs), and may also include terminals such as desktop computers. The method includes: obtaining multiple video frames corresponding to the video to be processed; inputting the multiple video frames into a video classification model for processing to obtain a classification result corresponding to the video to be processed; wherein the video classification model is obtained by training a sample video set based on an initial generator network, an initial discriminator network, and an initial classifier network; the sample video set includes multiple labeled videos, multiple pseudo-labeled videos, and multiple unlabeled videos; the multiple pseudo-labeled videos are generated based on multiple unlabeled videos, an initial generator network, an initial classifier network, and an initial discriminator network.

[0007] Among them, a labeled video is a video with a video classification label, and a labeled video may include a first video and a first classification label corresponding to the first video. A labeled video can be manually labeled with a classification label, or a video with a classification label can be obtained from the network, and there is no limitation on this. A pseudo-labeled video is a video with a classification label that can be used to train a video classification model, which is generated by an unlabeled video, an initial generator network, an initial classifier network, and an initial discriminator network. A pseudo-labeled video includes a second video and a second classification label corresponding to the second video.

[0008] The video classification method provided in the first aspect generates a large number of pseudo-labeled videos through multiple unlabeled videos, an initial generator network, an initial classifier network, and an initial discriminator network, and trains the initial classifier network based on multiple labeled videos and a large number of pseudo-labeled videos. Since the labeled videos and pseudo-labeled videos used for training are rich in variety and large in number, the video classification model obtained by training will not be overfitted. Therefore, when the video classification model is used to classify videos, the classification results are more accurate, thereby improving the accuracy of video classification.

[0009] In a possible implementation of the first aspect, multiple video frames are input into a video classification model for processing to obtain a classification result corresponding to the video to be processed, including:

[0010] The video classification model is used to extract feature vectors corresponding to each of the multiple video frames;

[0011] Generate video features corresponding to the video to be processed based on feature vectors corresponding to each of the multiple video frames;

[0012] The video features are classified to obtain a classification result corresponding to the video to be processed.

[0013] In this implementation, since the video classification model obtained by training the initial classifier network through multiple labeled videos and a large number of pseudo-labeled videos is used to process multiple video frames corresponding to the video to be processed, the extracted feature vectors are rich and comprehensive, and the video features generated based on the feature vectors are also richer and more comprehensive. And since the video classification model does not have the phenomenon of overfitting, when it classifies the rich and comprehensive video features, it can accurately match the classification results corresponding to the video features, thereby accurately obtaining the classification results corresponding to the video to be processed, and realizing accurate classification of the video to be processed.

[0014] In a possible implementation of the first aspect, before inputting the multiple video frames into the video classification model for processing and obtaining the classification result corresponding to the video to be processed, the method further includes:

[0015] Training a plurality of unlabeled videos based on the initial generator network, the initial classifier network, and the initial discriminator network to obtain a trained generator network;

[0016] Generate multiple pseudo-label videos based on the trained generator network and multiple unlabeled videos;

[0017] The initial classifier network is trained based on multiple labeled videos and multiple pseudo-labeled videos to obtain the video classification model.

[0018] In a possible implementation of the first aspect, a plurality of unlabeled videos are trained based on the initial generator network, the initial classifier network, and the initial discriminator network to obtain a trained generator network, including:

[0019] Based on the initial generator network and the initial classifier network, a target video is obtained from the multiple unlabeled videos; the target video is an unlabeled video whose confidence is greater than a confidence threshold; the confidence is used to preliminarily determine whether the unlabeled video can be used to generate a pseudo-labeled video;

[0020] Classify the target video based on the initial classifier network to obtain a classification result corresponding to the target video;

[0021] Inputting the labeled video, the target video, and the classification result corresponding to the target video into the initial discriminator network for discriminant processing to obtain a first discriminant result corresponding to the labeled video and a second discriminant result corresponding to the target video;

[0022] Based on the first discrimination result and the second discrimination result, determining a first loss value corresponding to the initial generator network and a second loss value corresponding to the initial discriminator network;

[0023] When the first loss value is less than or equal to the first preset threshold and the second loss value is less than or equal to the second preset threshold, stop training the initial generator network and use the trained initial generator network as the trained generator network.

[0024] In a possible implementation of the first aspect, the labeled video includes a first video and a first classification label corresponding to the first video; the pseudo-label video includes a second video and a second classification label corresponding to the second video; and the initial classifier network is trained based on multiple labeled videos and multiple pseudo-label videos to obtain the video classification model, including:

[0025] Inputting a plurality of video frames corresponding to the first video and a plurality of video frames corresponding to the second video into the initial classifier network for processing, and obtaining classification results corresponding to the first video and the second video respectively;

[0026] Calculate a third loss value based on the classification results corresponding to the first video and the second video, the first classification label, and the second classification label;

[0027] When the third loss value is greater than the third preset threshold, the parameters of the initial classifier network are adjusted, and the step of returning to execute is to input the multiple video frames corresponding to the first video and the multiple video frames corresponding to the second video into the initial classifier network for processing to obtain the classification results corresponding to the first video and the second video respectively.

[0028] In a possible implementation manner of the first aspect, after calculating the third loss value based on the classification results corresponding to the first video and the second video, the first classification label, and the second classification label, the method further includes:

[0029] When the third loss value is less than or equal to the third preset threshold, the training of the initial classifier network is stopped, and the trained initial classifier network is used as the video classification model.

[0030] In a second aspect, a video classification device is provided, the device comprising a processing unit (processing module), the processing unit being configured to:

[0031] Obtain multiple video frames corresponding to the video to be processed;

[0032] Input multiple video frames into a video classification model for processing to obtain a classification result corresponding to the video to be processed; wherein the video classification model is obtained by training a sample video set based on an initial generator network, an initial discriminator network, and an initial classifier network; the sample video set includes multiple labeled videos, multiple pseudo-labeled videos, and multiple unlabeled videos; the multiple pseudo-labeled videos are generated based on the multiple unlabeled videos, the initial generator network, the initial classifier network, and the initial discriminator network.

[0033] In a possible implementation manner of the second aspect, the processing unit is further configured to:

[0034] The video classification model is used to extract feature vectors corresponding to each of the multiple video frames;

[0035] Generating video features corresponding to the video to be processed based on feature vectors corresponding to each of the multiple video frames;

[0036] The video feature is classified to obtain the classification result.

[0037] In a possible implementation manner of the second aspect, the processing unit is further configured to:

[0038] Training a plurality of unlabeled videos based on the initial generator network, the initial classifier network, and the initial discriminator network to obtain a trained generator network;

[0039] Generate multiple pseudo-label videos based on the trained generator network and multiple unlabeled videos;

[0040] The initial classifier network is trained based on multiple labeled videos and multiple pseudo-labeled videos to obtain the video classification model.

[0041] In a possible implementation manner of the second aspect, the processing unit is further configured to:

[0042] Based on the initial generator network and the initial classifier network, a target video is obtained from the multiple unlabeled videos; the target video is an unlabeled video whose confidence is greater than a confidence threshold; the confidence is used to preliminarily determine whether the unlabeled video can be used to generate a pseudo-labeled video;

[0043] Classify the target video based on the initial classifier network to obtain a classification result corresponding to the target video;

[0044] Inputting the labeled video, the target video, and the classification result corresponding to the target video into the initial discriminator network for discriminant processing to obtain a first discriminant result corresponding to the labeled video and a second discriminant result corresponding to the target video;

[0045] Based on the first discrimination result and the second discrimination result, determining a first loss value corresponding to the initial generator network and a second loss value corresponding to the initial discriminator network;

[0046] When the first loss value is less than or equal to the first preset threshold and the second loss value is less than or equal to the second preset threshold, stop training the initial generator network and use the trained initial generator network as the trained generator network.

[0047] In a possible implementation manner of the second aspect, the labeled video includes a first video and a first classification label corresponding to the first video; the pseudo-label video includes a second video and a second classification label corresponding to the second video; and the processing unit is further configured to:

[0048] Inputting a plurality of video frames corresponding to the first video and a plurality of video frames corresponding to the second video into the initial classifier network for processing, and obtaining classification results corresponding to the first video and the second video respectively;

[0049] Calculate a third loss value based on the classification results corresponding to the first video and the second video, the first classification label, and the second classification label;

[0050] When the third loss value is greater than the third preset threshold, the parameters of the initial classifier network are adjusted, and the step of inputting the multiple video frames corresponding to the first video and the multiple video frames corresponding to the second video into the initial classifier network for processing is returned to obtain the classification results corresponding to the first video and the second video respectively.

[0051] In a possible implementation manner of the second aspect, when the third loss value is less than or equal to the third preset threshold, the processing unit is further used to:

[0052] When the third loss value is less than or equal to the third preset threshold, the training of the initial classifier network is stopped, and the trained initial classifier network is used as the video classification model.

[0053] According to a third aspect, a video classification apparatus is provided, the apparatus comprising at least one processor and an interface circuit, the at least one processor being configured to execute the method in the first aspect or any possible implementation of the first aspect.

[0054] In a fourth aspect, a terminal device is provided, the terminal device comprising: the apparatus for video classification provided in the second aspect or the apparatus for video classification provided in the third aspect. Optionally, the terminal includes but is not limited to mobile terminals such as smart phones, tablet computers, computers, PDAs, and may also include terminals such as desktop computers.

[0055] According to a fifth aspect, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the computer program is used to execute the method in the first aspect or any possible implementation of the first aspect.

[0056] According to a sixth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed, the computer program is used to execute the method in the first aspect or any possible implementation of the first aspect.

[0057] In the seventh aspect, a chip or integrated circuit is provided, which includes: a processor, used to call and run a computer program from a memory, so that a device equipped with the chip or integrated circuit executes the method in the first aspect or any possible implementation of the first aspect.

[0058] The video classification method provided in the embodiment of the present application obtains multiple video frames corresponding to the video to be processed; the multiple video frames are input into the video classification model for processing to obtain the classification result corresponding to the video to be processed. In the present application, the terminal generates a large number of pseudo-labeled videos based on multiple unlabeled videos, an initial generator network, an initial classifier network, and an initial discriminator network. The video classification model obtained by training the initial classifier network based on multiple labeled videos and a large number of pseudo-labeled videos will not have the phenomenon of overfitting. Therefore, when the video classification model is used to classify the video, the classification result is more accurate, thereby improving the accuracy of the video classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0060] Figure 1 is a schematic flow chart of a video classification method provided in one embodiment of the present application;

[0061] Figure 2 is a schematic flow chart of a video classification method provided by another embodiment of the present application;

[0062] Figure 3 is a schematic flow chart of a video classification method provided by another embodiment of the present application;

[0063] Figure 4 is a schematic flow chart of a video classification method provided by another embodiment of the present application;

[0064] Figure 5 is a schematic flow chart of a video classification method provided by another embodiment of the present application;

[0065] Figure 6 is a schematic diagram of a video classification device provided by an embodiment of the present application;

[0066] Figure 7 It is a schematic diagram of a video classification device provided by another embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0068] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0069] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0070] In the prior art, most videos do not have classification labels, but video classification models require a large number of videos with classification labels during training. Since it is time-consuming, labor-intensive and costly to manually label videos with classification labels, traditional video classification models usually use a small number of videos with classification labels to train video classification models, which makes the trained video classification models prone to overfitting. When using an overfitted video classification model to classify videos, the video classification results will be inaccurate.

[0071] In view of this, the present application provides a method for video classification, by obtaining multiple video frames corresponding to the video to be processed; the multiple video frames are input into the video classification model for processing, and the classification results corresponding to the video to be processed are obtained. In the present application, the terminal generates a large number of pseudo-labeled videos based on multiple unlabeled videos, an initial generator network, an initial classifier network, and an initial discriminator network. The video classification model obtained by training the initial classifier network based on multiple labeled videos and a large number of pseudo-labeled videos will not have the phenomenon of overfitting. Therefore, when the video classification model is used to classify the video, the classification result is more accurate, thereby improving the accuracy of the video classification. Furthermore, the video classification method provided by the present application can automatically generate a large number of pseudo-labeled videos, without the need to manually classify and label the videos, saving time and manpower, reducing costs, and improving the training speed of the video classification model.

[0072] Figure 1 : is a schematic flow chart of a video classification method provided by an embodiment of the present application. It should be understood that the execution subject of the video classification method provided by the present application is a terminal, which includes but is not limited to mobile terminals such as smart phones, tablet computers, computers, PDAs, etc., and may also include terminals such as desktop computers. Figure 1The video classification method 100 shown may include: S101 and S102, which will be described in detail below.

[0073] S101: Acquire multiple video frames corresponding to the video to be processed.

[0074] The video to be processed refers to a video that needs to be classified. The terminal obtains multiple video frames corresponding to the video to be processed, which can be after obtaining the video to be processed, using a single video frame as the smallest segmentation unit to segment the video to be processed to obtain multiple video frames corresponding to the video to be processed. For example, the video to be processed consists of 30 video frames, and the terminal segments the video to obtain 30 video frames.

[0075] The terminal obtains multiple video frames corresponding to the video to be processed, or the terminal or other terminals pre-segment the video to be processed to obtain multiple video frames corresponding to the video to be processed, and the terminal obtains multiple video frames corresponding to the video to be processed. For example, the terminal obtains multiple video frames corresponding to the video to be processed in the database, or other terminals send multiple video frames corresponding to the video to be processed to the terminal, and the terminal receives multiple video frames corresponding to the video to be processed sent by other terminals. This is only an exemplary description and is not limited to this.

[0076] S102: Input multiple video frames into a video classification model for processing to obtain a classification result corresponding to the video to be processed; wherein the video classification model is obtained by training a sample video set based on an initial generator network, an initial discriminator network, and an initial classifier network; the sample video set includes multiple labeled videos, multiple pseudo-labeled videos, and multiple unlabeled videos; the multiple pseudo-labeled videos are generated based on the multiple unlabeled videos, the initial generator network, the initial classifier network, and the initial discriminator network.

[0077] In this embodiment, a pre-trained video classification model is pre-stored in the terminal. The video classification model is obtained by training a sample video set based on an initial generator network, an initial discriminator network, and an initial classifier network using a machine learning algorithm. The sample video set includes a plurality of labeled videos, a plurality of pseudo-labeled videos, and a plurality of unlabeled videos.

[0078] A labeled video refers to a video with a classification label, and the classification label can understand the classification result corresponding to the video, that is, the category to which the video belongs. A labeled video may include a first video and a first classification label corresponding to the first video. A labeled video can be obtained by manually labeling a video with a classification label, or a video with a classification label can be obtained from the Internet. For example, a video with a classification label can be obtained from a database publicly available on the Internet as a labeled video in this application. This is only an exemplary description and is not limited to this.

[0079] The pseudo-label video is a video with a classification label that can be used to train a video classification model and is generated by the terminal based on the unlabeled video, the initial generator network, the initial classifier network, and the initial discriminator network. The pseudo-label video includes a second video and a second classification label corresponding to the second video. Among them, the unlabeled video refers to a video that does not contain a classification label.

[0080] The initial generator network, the initial discriminator network, and the initial classifier network refer to an untrained generator network model, an untrained discriminator network model, and an untrained classifier network model, respectively, and their corresponding networks can be constructed on the terminal, and their corresponding program codes can be obtained in the network for direct use. Among them, the initial generator network can be used to extract video features from unlabeled videos, and to cooperate with the initial classifier network to generate videos marked with classification labels. The initial discriminator network is used to determine that videos marked with classification labels generated by the initial generator network are false, and to determine that labeled videos are true. The initial classifier network is used to classify unlabeled videos.

[0081] The terminal generates a large number of pseudo-labeled videos based on multiple unlabeled videos, an initial generator network, an initial classifier network, and an initial discriminator network, and trains the initial classifier network based on multiple labeled videos and a large number of pseudo-labeled videos to obtain a trained video classification model.

[0082] It is understandable that the video classification model can be pre-trained by the terminal, or pre-trained by other devices and then the file corresponding to the video classification model can be transplanted to the terminal. In other words, the execution subject of training the video classification model and the execution subject of performing video classification using the video classification model can be the same or different.

[0083] After obtaining multiple video frames corresponding to the video to be processed, the terminal uses the video classification model to process each video frame to obtain the feature vector corresponding to each video frame, generates video features corresponding to the video to be processed based on these feature vectors, and classifies the video features. The classification result obtained is the classification result corresponding to the video to be processed, that is, the category to which the video to be processed belongs.

[0084] The video classification method provided in the embodiment of the present application obtains multiple video frames corresponding to the video to be processed; the multiple video frames are input into the video classification model for processing to obtain the classification result corresponding to the video to be processed. In the present application, the terminal generates a large number of pseudo-labeled videos based on multiple unlabeled videos, an initial generator network, an initial classifier network, and an initial discriminator network. The video classification model obtained by training the initial classifier network based on multiple labeled videos and a large number of pseudo-labeled videos will not have the phenomenon of overfitting. Therefore, when the video classification model is used to classify the video, the classification result is more accurate, thereby improving the accuracy of the video classification.

[0085] Optionally, in some possible implementations of the present application, such as Figure 2 As shown, Figure 2 is a schematic flow chart of a method for video classification in another embodiment of the present application. Figure 1 Based on the method steps shown, the method S200 may include: S201 to S204. Figure 2 The step S201 shown can refer to the above description of S101, and for the sake of brevity, it will not be repeated here. The following will specifically describe steps S202 to S204.

[0086] S202: Using the video classification model to extract feature vectors corresponding to each of the multiple video frames.

[0087] The trained video classification model may include an input layer, multiple convolutional layers, multiple sampling layers, and an output layer (fully connected layer). For example, for each video frame, the video frame is input into the video classification model for processing, and the input layer in the video classification model performs data normalization on the video frame. Specifically, the pixel value corresponding to each pixel in the video frame is obtained, and the pixel value corresponding to each pixel is divided by 255, so that the pixel value corresponding to each pixel falls into [0, 1]. After each pixel in the video frame is processed in this way, an image after data normalization is obtained. The image after data normalization is passed to the first convolutional layer, and the first convolutional layer performs convolution processing on the image, extracts the features corresponding to the image, and outputs a feature map based on the extracted features. Among them, the features may include the color features, contour features, human features, animal features, and object features contained in the image. The first convolutional layer inputs the feature map into the first sampling layer, and the first sampling layer performs feature selection on the feature map, removes redundant features, reconstructs a new feature map, and passes the new feature map to the second convolutional layer. The second convolutional layer performs secondary feature extraction on the new feature map, and outputs the feature map again based on the extracted features. The second convolutional layer passes the re-output feature map to the second sampling layer, and the second sampling layer performs secondary feature selection and reconstructs the feature map. This process is repeated until the last sampling layer in the video classification model completes the image processing and passes it to the output layer, that is, to the fully connected layer. The normalized exponential function (Softmax function) in the fully connected layer classifies the features passed from the previous sampling layer to obtain the features corresponding to the input video frame. Since each feature in the video classification model is expressed in the form of a vector, it can also be understood as the feature vector corresponding to the video frame output by the fully connected layer. Multiple video frames corresponding to the video to be processed are input into the video classification model for the above processing to obtain the feature vector corresponding to each video frame.

[0088] S203: Generate video features corresponding to the video to be processed based on the feature vectors corresponding to each of the multiple video frames.

[0089] Video features are used to represent the core content identified from the video to be processed, that is, to represent the effective features of the video to be processed for classification. Since video features are also expressed in vector form in the video classification model, for ease of understanding, the meaning of video features is used to explain them. For example, the meaning of video features can be an athlete playing basketball, a person giving a speech indoors, an actor dancing indoors, etc. This is only an exemplary description and is not limited to this.

[0090] Specifically, a feature vector sequence is constructed based on the feature vectors corresponding to each of the multiple video frames, that is, the feature vectors corresponding to each video frame are sorted according to the time sequence of each video frame in the video to be processed. These feature vectors are fused according to their order in the feature vector sequence to obtain the video features corresponding to the video to be processed. For example, the feature vector corresponding to the first video frame is convolved with the feature vector corresponding to the second video frame, and the obtained operation result is convolved with the feature vector corresponding to the third video frame, and so on, until the feature vectors corresponding to all video frames are processed, and the final operation result is used as the video feature corresponding to the video to be processed.

[0091] S204: Classify the video features to obtain the classification result.

[0092] During the training process, the video classification model will collect a large number of different video features and the classification results corresponding to each video feature. According to the video feature, the classification result corresponding to it is searched and the classification result found is output. It can be understood that the classification result obtained by classifying the video feature is the classification result corresponding to the video to be processed.

[0093] In the method for video classification provided by the embodiment of the present application, the terminal generates a large number of pseudo-labeled videos based on multiple unlabeled videos, an initial generator network, an initial classifier network, and an initial discriminator network, and the video classification model obtained by training the initial classifier network based on multiple labeled videos and a large number of pseudo-labeled videos will not be overfitted. Therefore, when the video classification model is used to classify the video, the classification result is more accurate, thereby improving the accuracy of the video classification. Furthermore, a large number of pseudo-labeled videos can be automatically generated in the present application, and there is no need to manually classify and label the videos, which saves time and manpower, reduces costs, and improves the training speed of the video classification model.

[0094] Figure 3 FIG. 1 is a schematic flow chart of a video classification method provided by another embodiment of the present application. Figure 3 The method 300 shown in FIG. Figure 1 Before the method 100 shown, the following steps may also be included: S301 to S303. Figure 3 The steps S304 and S305 shown can refer to the above description of S101 and S102, and will not be described here for brevity. The steps S301 to S303 will be described in detail below.

[0095] S301: Training the multiple unlabeled videos based on the initial generator network, the initial classifier network, and the initial discriminator network to obtain a trained generator network.

[0096] Exemplarily, an initial classifier network can be obtained by preliminarily training an untrained classifier network model using multiple labeled videos, so that the initial classifier network can classify the video. The terminal extracts the features corresponding to each unlabeled video through the initial generator network, and generates a new video based on the extracted features. It can be understood that the initial generator network learns the features corresponding to these unlabeled videos and automatically generates new videos based on the learned results. The new video is classified by the initial classifier network to obtain the classification result corresponding to the new video.

[0097] The new video and its corresponding classification results and the labeled video are input into the initial discriminator network. The initial discriminator network will judge the labeled video as true, and judge the new video generated based on the initial generator network and its corresponding classification results as false. The initial discriminator network adjusts its own parameters and feeds the results back to the initial generator network. The initial generator network adjusts its network parameters and continues to generate new videos based on the initial generator network with adjusted network parameters. The videos generated by the initial generator network are then classified by the initial classifier network until the loss functions corresponding to the initial generator network and the initial discriminator network converge. No more training is performed, and the parameters corresponding to the initial generator network at this time are used as the parameters of the trained generator network.

[0098] It can be generally understood that the initial generator network cooperates with the initial classifier network to generate videos that can deceive the initial discriminator network as much as possible and the classification results corresponding to the videos. The initial discriminator network determines that the original labeled videos are true as much as possible, and determines that the videos generated by the initial generator network and the initial classifier network as well as the classification results corresponding to the videos are false. The network parameters of the initial generator network and the initial discriminator are continuously adjusted according to the judgment results of the initial discriminator until the initial generator network is trained. In other words, the videos generated by the trained generator network and the initial classifier network as well as the classification results corresponding to the videos can deceive the initial discriminator network, making the initial discriminator network mistakenly believe that these videos and the classification results corresponding to these videos are similar to the labeled videos.

[0099] The large number of videos generated based on the cooperation between the trained generator network and the initial classifier network and the classification results corresponding to each video are equivalent to obtaining a large number of training sample videos.

[0100] Optionally, in some possible implementations, the initial generator network and the initial classifier network can be used to select videos from multiple unlabeled videos, and the selected videos can be classified by the initial classifier network to obtain corresponding classification results. The selected videos and their corresponding classification results and labeled videos are input into the initial discriminator network for judgment, and the network parameters of the initial generator network are continuously adjusted according to the judgment results until the loss functions corresponding to the initial generator network and the initial discriminator network converge. No more training is performed, and the network parameters corresponding to the initial generator network at this time are used as the network parameters of the trained generator network. The convergence of the loss function means that during the training process, the value corresponding to the loss function tends to a value and no longer changes, or the change range is particularly small.

[0101] S302: Generate multiple pseudo-label videos based on the trained generator network and the multiple unlabeled videos.

[0102] Exemplarily, the trained generator network can extract rich and effective features from multiple unlabeled videos, and generate new videos based on the extracted features. For example, if a certain unlabeled video is a video of an athlete playing badminton, the trained generator network accurately extracts the various action features of the athlete's movement, the movement features of badminton, the movement trajectory of the badminton racket, etc., and generates a new video based on these extracted features. The new video is classified by the initial classifier network to obtain the classification result corresponding to the new video. The new video and the classification result corresponding to the new video are used as pseudo-label videos. A large number of pseudo-label videos can be generated by the same method.

[0103] The untrained generator network may not be very accurate when extracting features, such as when extracting the features of a video of a certain athlete playing badminton, the features of badminton and badminton racket are not very accurate, causing the initial classifier network to classify its corresponding video as playing tennis. And the trained generator network in the present embodiment can extract more detailed features, such as when extracting the features of a video of a certain athlete playing badminton, the features of badminton and badminton racket can be accurately extracted, so that when the initial classifier classifies the video it generates, the classification result is more accurate. Therefore, the pseudo-label video generated is also more real, and is closer to the labeled video.

[0104] Optionally, in some possible implementations, the trained generator network can select videos that can be used to generate pseudo-labeled videos from multiple unlabeled videos, and classify the selected videos through the initial classifier network to obtain corresponding classification results. The selected videos and their corresponding classification results are used as pseudo-labeled videos. A large number of pseudo-labeled videos can be generated by the same method.

[0105] S303: Training the initial classifier network based on the multiple labeled videos and the multiple pseudo-labeled videos to obtain the video classification model.

[0106] The initial classifier network is trained based on multiple labeled videos and a large number of pseudo-labeled videos to obtain a trained video classification model. It is worth noting that the initial classifier network here can be the untrained classifier network model mentioned in S102, or it can be the initial classifier network obtained by preliminarily training the untrained classifier network model using multiple labeled videos mentioned in S301.

[0107] Figure 4 2 is a schematic flow chart of a video classification method provided by another embodiment of the present application. For example, in some possible implementations of the present application, the above S301 may include S3011 to S3015, and the steps S3011 to S3015 will be specifically described below.

[0108] S3011: Based on the initial generator network and the initial classifier network, a target video is obtained from the multiple unlabeled videos; the target video is an unlabeled video whose confidence is greater than a confidence threshold; the confidence is used to preliminarily determine whether the unlabeled video can be used to generate a pseudo-labeled video.

[0109] An unlabeled video with a confidence greater than a confidence threshold is selected as a target video from multiple unlabeled videos through an initial generator network and an initial classifier network. Among them, the confidence threshold is preset and can be adjusted according to the specific implementation situation, and there is no limitation on this. The confidence is used to preliminarily determine whether the unlabeled video can be used to generate a pseudo-labeled video. It can be understood that the target video selected by the initial generator network and the initial classifier network from multiple unlabeled videos is subsequently used to generate a pseudo-labeled video. At this time, since the initial generator network has not been trained, the target video selected at this time is not very accurate. After the initial generator network is trained, the target video selected can be used to generate a pseudo-labeled video. For example, after the target video is selected by the trained generator network, the target video is classified by the initial classifier network to obtain the corresponding classification result, and the target video and its corresponding classification result can be used as a pseudo-labeled video.

[0110] Specifically, the target video can be selected by a preset formula, which is as follows: P = C (D (F (U))), (1)

[0111] In the above formula (1), P represents confidence, C represents the initial classifier network, D represents the feature mapping between unlabeled videos and labeled videos, F represents the initial generator network, and U represents the unlabeled video set. The unlabeled video set refers to a set of multiple unlabeled videos. By comparing the size between the confidence and the confidence threshold, the target video is selected from multiple unlabeled videos.

[0112] S3012: Classify the target video based on the initial classifier network to obtain a classification result corresponding to the target video.

[0113] The selected target video is classified through the initial classifier network to obtain the classification result corresponding to the target video. The target video is first segmented to obtain multiple video frames corresponding to the target video, and the multiple video frames corresponding to the target video are input into the initial classifier network for classification. The specific classification process is the same as Figure 2 In the corresponding embodiment, the process of the video classification model in S202 to S204 processing multiple video frames is similar. Please refer to the description in S202 to S204, which will not be repeated here.

[0114] S3013: The labeled video, the target video, and the classification result corresponding to the target video are input into the initial discriminator network for discriminant processing to obtain a first discriminant result corresponding to the labeled video and a second discriminant result corresponding to the target video.

[0115] The labeled video, the target video, and the classification result corresponding to the target video are input into the initial discriminator network for discrimination processing. During the initial training, the initial discriminator network will judge the labeled video as true, and the target video and the classification result corresponding to the target video as false. That is, the first discrimination result corresponding to the labeled video is true, and the second discrimination result corresponding to the target video is false. It can be understood that when the initial generator network is trained, when the selected target video is judged by the initial discriminator network, the initial discriminator network will judge the target video and the classification result corresponding to the target video as true. In other words, when the trained generator network is obtained, the target video and the classification result corresponding to the target video that deceive the initial discriminator network can be generated based on the trained generator network and the initial classifier network, that is, a pseudo-label video that deceives the initial discriminator network can be generated.

[0116] S3014: Based on the first discrimination result and the second discrimination result, determine a first loss value corresponding to the initial generator network and a second loss value corresponding to the initial discriminator network.

[0117] Exemplarily, a true discrimination result can be recorded as 1, and a false discrimination result can be recorded as 0. The first loss value can be calculated based on the first discrimination result and the loss function corresponding to the initial generator network, and the second loss value can be calculated based on the second discrimination result and the loss function corresponding to the initial discriminator network. In this example, the cross entropy loss function can be used as the loss function corresponding to the initial generator network, and the binary cross entropy function can be used as the loss function corresponding to the initial discriminator network. This is only an exemplary explanation and is not limited to this. For example, the first loss value is calculated based on the first discrimination result and the cross entropy loss function, and the second loss value is calculated based on the second discrimination result and the binary cross entropy function.

[0118] S3015: When the first loss value is less than or equal to the first preset threshold and the second loss value is less than or equal to the second preset threshold, stop training the initial generator network and use the trained initial generator network as the trained generator network.

[0119] The first preset threshold is used to compare with the first loss value, and the second preset threshold is used to compare with the second loss value. According to the comparison result, it can be judged whether the initial generator network meets the training requirements. The first preset threshold and the second preset threshold can be set in advance and can be adjusted at any time during the training of the initial generator network, and there is no limitation on this.

[0120] Exemplarily, when the comparison result is that the first loss value is less than or equal to the first preset threshold, and the second loss value is less than or equal to the second preset threshold, the training of the initial generator network is stopped, and the initial generator network at this time is used as the trained generator network. When any loss value in the comparison result is greater than its corresponding preset threshold, the network parameters in the initial generator network and the initial discriminator network are adjusted, and the execution of S3011 to S3014 is returned until both loss values ​​are less than or equal to their corresponding preset thresholds.

[0121] Figure 5 is a schematic flow chart of a video classification method provided by another embodiment of the present application. For example, in some possible implementations of the present application, Figure 5 As shown, the above S303 may include S3031 to S3034, and steps S3031 to S3034 will be specifically described below.

[0122] S3031: Input multiple video frames corresponding to the first video and multiple video frames corresponding to the second video into the initial classifier network for processing to obtain classification results corresponding to the first video and the second video respectively.

[0123] The labeled video includes a first video and a first classification label corresponding to the first video; the pseudo-label video includes a second video and a second classification label corresponding to the second video.

[0124] During the training process, the sample videos used for training include multiple labeled videos and a large number of pseudo-labeled videos. There is no limit on the order of the videos input into the initial classifier network for processing, that is, there is no limit on the order of the first video and the second video input. It can be that all the video frames contained in the first video corresponding to all the labeled videos are first input into the initial classifier network for processing, and then all the video frames contained in the second video corresponding to all the pseudo-labeled videos are input into the initial classifier network for processing. The video frames contained in the first video and the video frames contained in the second video can also be input crosswise, for example, a video frame contained in the first video is first input and then a video frame contained in the second video is input, etc., and there is no limit on this. Accordingly, when the video frame contained in the first video is input into the initial classifier network, the initial classifier network outputs the classification result corresponding to the first video; when the video frame contained in the second video is input into the initial classifier network, the initial classifier network outputs the classification result corresponding to the second video.

[0125] The process of the initial classifier network classifying the multiple video frames corresponding to the first video and the multiple video frames corresponding to the second video can be referred to the description in S202 to S204, which will not be repeated here.

[0126] S3032: Calculate a third loss value based on the classification results corresponding to the first video and the second video, the first classification label, and the second classification label.

[0127] In this example, an activation function (sigmoid function) can be used as a loss function, and the third loss value is calculated by the activation function. When the input to the initial classifier network is a plurality of video frames corresponding to the first video, the third loss value is calculated based on the classification result corresponding to the first video, the first classification label and the loss function; when the input to the initial classifier network is a plurality of video frames corresponding to the second video, the third loss value is calculated based on the classification result corresponding to the second video, the second classification label and the loss function.

[0128] When the third loss value is calculated, the size between the third loss value and the third preset threshold is determined. When the judgment result is that the third loss value is greater than the third preset threshold, execute S2033; when the judgment result is that the third loss value is less than or equal to the third preset threshold, execute S2034.

[0129] S3033: When the third loss value is greater than the third preset threshold, adjust the parameters of the initial classifier network, and return to the step of inputting the multiple video frames corresponding to the first video and the multiple video frames corresponding to the second video into the initial classifier network for processing to obtain the classification results corresponding to the first video and the second video respectively.

[0130] The third preset threshold is used to compare with the third loss value. According to the comparison result between the third loss value and the third preset threshold, it can be judged whether the initial classifier network meets the training requirements. The third preset threshold can be set in advance and can be adjusted at any time during the training of the initial classifier network. There is no limitation on this.

[0131] For example, during the training process, the terminal compares the third loss value with the third preset threshold value. When the third loss value is greater than the third preset threshold value, it is determined that the current initial classifier network has not yet met the requirements. At this time, it is necessary to adjust the parameters in the initial classifier network, and after the adjustment, return to S3031, and continue to execute S3031 and S3032 until it is determined in S3032 that the third loss value is less than or equal to the third preset threshold value, and then execute S3034.

[0132] S3034: When the third loss value is less than or equal to the third preset threshold, stop training the initial classifier network, and use the trained initial classifier network as the video classification model.

[0133] Exemplarily, during the training process, the terminal compares the third loss value with the third preset threshold value, and when the third loss value is less than or equal to the third preset threshold value, it is determined that the current initial classifier network meets the expected requirements, and the training of the initial classifier network is stopped. The initial classifier network at this time is used as the trained video classification model.

[0134] The video classification model is obtained by training the initial classifier network with multiple labeled videos and a large number of pseudo-labeled videos. The number of training samples is large, the types are rich and comprehensive, and there will be no overfitting, and its loss value is kept in a small range. Therefore, when the video classification model is used to process multiple video frames corresponding to the processed video, the classification results obtained are very accurate.

[0135] In the method for video classification provided in the embodiment of the present application, the terminal generates a large number of pseudo-labeled videos through multiple unlabeled videos, an initial generator network, an initial classifier network, and an initial discriminator network. The video classification model obtained by training the initial classifier network based on multiple labeled videos and a large number of pseudo-labeled videos will not be overfitted. Therefore, when the video classification model is used to classify the video, the classification result is more accurate, thereby improving the accuracy of the video classification. Furthermore, in the present embodiment, a large number of pseudo-labeled videos that can be used for training can be automatically generated, and there is no need to manually classify and label the videos, which saves time and manpower, reduces costs, and improves the training speed of the video classification model.

[0136] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0137] See also Figure 6 , Figure 6 It is a schematic diagram of a video classification device provided in one embodiment of the present application. Figure 6 A schematic block diagram of a video classification apparatus 400 according to an embodiment of the present application is shown, and the apparatus 400 may include a processing unit 410, and optionally, the apparatus may include a storage unit 420. The storage unit 420 is used to execute instructions stored in the storage unit.

[0138] Exemplarily, the apparatus includes a processing unit (module) 410, which is used to:

[0139] Obtain multiple video frames corresponding to the video to be processed;

[0140] Input multiple video frames into a video classification model for processing to obtain a classification result corresponding to the video to be processed; wherein the video classification model is obtained by training a sample video set based on an initial generator network, an initial discriminator network, and an initial classifier network; the sample video set includes multiple labeled videos, multiple pseudo-labeled videos, and multiple unlabeled videos; the multiple pseudo-labeled videos are generated based on the multiple unlabeled videos, the initial generator network, the initial classifier network, and the initial discriminator network.

[0141] Optionally, the processing unit 410 is further configured to:

[0142] The video classification model is used to extract feature vectors corresponding to each of the multiple video frames;

[0143] Generating video features corresponding to the video to be processed based on feature vectors corresponding to each of the multiple video frames;

[0144] The video feature is classified to obtain the classification result.

[0145] Optionally, the processing unit 410 is further configured to:

[0146] Training a plurality of unlabeled videos based on the initial generator network, the initial classifier network, and the initial discriminator network to obtain a trained generator network;

[0147] Generate multiple pseudo-label videos based on the trained generator network and multiple unlabeled videos;

[0148] The initial classifier network is trained based on multiple labeled videos and multiple pseudo-labeled videos to obtain the video classification model.

[0149] Optionally, the processing unit 410 is further configured to:

[0150] Based on the initial generator network and the initial classifier network, a target video is obtained from the multiple unlabeled videos; the target video is an unlabeled video whose confidence is greater than a confidence threshold; the confidence is used to preliminarily determine whether the unlabeled video can be used to generate a pseudo-labeled video;

[0151] Classify the target video based on the initial classifier network to obtain a classification result corresponding to the target video;

[0152] Inputting the labeled video, the target video, and the classification result corresponding to the target video into the initial discriminator network for discriminant processing to obtain a first discriminant result corresponding to the labeled video and a second discriminant result corresponding to the target video;

[0153] Based on the first discrimination result and the second discrimination result, determining a first loss value corresponding to the initial generator network and a second loss value corresponding to the initial discriminator network;

[0154] When the first loss value is less than or equal to the first preset threshold and the second loss value is less than or equal to the second preset threshold, stop training the initial generator network and use the trained initial generator network as the trained generator network.

[0155] In a possible implementation manner of the second aspect, the labeled video includes a first video and a first classification label corresponding to the first video; the pseudo-label video includes a second video and a second classification label corresponding to the second video; and the processing unit is further configured to:

[0156] Inputting a plurality of video frames corresponding to the first video and a plurality of video frames corresponding to the second video into the initial classifier network for processing, and obtaining classification results corresponding to the first video and the second video respectively;

[0157] Calculate a third loss value based on the classification results corresponding to the first video and the second video, the first classification label, and the second classification label;

[0158] When the third loss value is greater than the third preset threshold, the parameters of the initial classifier network are adjusted, and the step of inputting the multiple video frames corresponding to the first video and the multiple video frames corresponding to the second video into the initial classifier network for processing is returned to obtain the classification results corresponding to the first video and the second video respectively.

[0159] Optionally, when the third loss value is less than or equal to the third preset threshold, the processing unit is further configured to: the processing unit 410 is further configured to:

[0160] When the third loss value is less than or equal to the third preset threshold, stop training the initial classifier network and use the trained initial classifier network as the video classification model

[0161] Optionally, the device may further include a transceiver unit 430, which is used to perform specific signal transceiving under the drive of the processing unit 410. The transceiver unit 430 may be a transceiver, an input / output interface, or an interface circuit. The transceiver unit 430, the processing unit 410, and the storage unit 420 are coupled to each other, the storage unit stores instructions, the processing unit 410 is used to execute the instructions stored in the storage unit, and the transceiver unit 430 is used to perform specific signal transceiving under the drive of the processing unit 410.

[0162] Optionally, the transceiver unit 430 may include a receiving unit (module) and a sending unit (module), which are used to execute the steps of receiving information and sending information in each embodiment of the aforementioned method 100, method 200 and method 300.

[0163] See also Figure 7 , Figure 7 FIG. 1 is a schematic diagram of a video classification device provided by another embodiment of the present application. Figure 7 As shown, the video classification device 500 includes: a processor 510, and optionally, a memory 520, the memory 520 is used to store computer-readable instructions, and the processor 510 implements the steps of the above-mentioned video classification method embodiments when executing the computer-readable instructions, such as Figure 1 S101 to S102 are shown. Optionally, the video classification apparatus may further include a transceiver 530, which is used to perform the steps of receiving information and sending information in the above-mentioned methods 100, 200 and 300.

[0164] The video classification device 500 may include, but is not limited to, a processor 510 and a memory 520. Those skilled in the art will appreciate that Figure 7 It is only an example of the apparatus 500 for video classification and does not constitute a limitation of the apparatus 500 for video classification. It may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, the apparatus 500 for video classification may also include input and output terminals, a network access terminal, a bus, etc.

[0165] The processor 510 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0166] The memory 520 may be an internal storage unit of the video classification device 500, such as a hard disk or memory of the video classification device 500. The memory 520 may also be an external storage terminal of the video classification device 500, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the video classification device 500. Further, the memory 520 may also include both an internal storage unit of the video classification device 500 and an external storage terminal. The memory 520 is used to store computer-readable instructions and other programs and data required by the video classification device 500. The memory 520 may also be used to temporarily store data that has been output or is to be output.

[0167] The embodiment of the present application further provides a terminal device, which includes any one of the video classification devices provided in the embodiment of the present application. Alternatively, any one of the video classification devices provided in the embodiment of the present application can be a terminal device.

[0168] The embodiment of the present application also provides a computer readable medium for storing computer program code, wherein the computer program includes instructions for executing the video classification method in the above method 100, method 200, and method 300. The readable medium may be a read-only memory (ROM) or a random access memory (RAM), which is not limited in the embodiment of the present application.

[0169] The present application also provides a computer program product, which includes instructions. When the instructions are executed, a terminal device performs operations of the terminal device corresponding to the above method.

[0170] An embodiment of the present application also provides a chip or integrated circuit, which includes: a processor, which is used to call and run a computer program from a memory, so that a device equipped with the chip or integrated circuit executes the above-mentioned video classification method.

[0171] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0172] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0173] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0174] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for video classification, characterized in that: include: Based on the initial generator network and the initial classifier network, a target video is obtained from multiple unlabeled videos; The target video is an unlabeled video whose confidence is greater than a confidence threshold; The confidence level is used to preliminarily determine whether the unlabeled video can be used to generate a pseudo-labeled video; the target video is classified based on the initial classifier network to obtain a classification result corresponding to the target video; Inputting the labeled video, the target video, and the classification result corresponding to the target video into an initial discriminator network for discriminant processing, to obtain a first discriminant result corresponding to the labeled video and a second discriminant result corresponding to the target video; Based on the first discrimination result and the second discrimination result, determining a first loss value corresponding to the initial generator network and a second loss value corresponding to the initial discriminator network; When the first loss value is less than or equal to a first preset threshold and the second loss value is less than or equal to a second preset threshold, stop training the initial generator network, and use the trained initial generator network as the trained generator network; Generate multiple pseudo-labeled videos according to the trained generator network and the multiple unlabeled videos; train the initial classifier network based on the multiple labeled videos and the multiple pseudo-labeled videos to obtain a video classification model; Obtain multiple video frames corresponding to the video to be processed; Inputting the multiple video frames into the video classification model for processing to obtain a classification result corresponding to the video to be processed; The step of inputting the plurality of video frames into the video classification model for processing to obtain a classification result corresponding to the video to be processed includes: extracting feature vectors corresponding to each of the plurality of video frames using the video classification model; generating video features corresponding to the video to be processed based on the feature vectors corresponding to each of the plurality of video frames; The video features are classified to obtain the classification result.

2. The method according to claim 1, characterized in that The labeled video includes a first video and a first classification label corresponding to the first video; the pseudo-label video includes a second video and a second classification label corresponding to the second video; the initial classifier network is trained based on the multiple labeled videos and the multiple pseudo-label videos to obtain the video classification model, including: Inputting a plurality of video frames corresponding to the first video and a plurality of video frames corresponding to the second video into the initial classifier network for processing, and obtaining classification results corresponding to the first video and the second video respectively; Calculate a third loss value based on the classification results corresponding to the first video and the second video, the first classification label, and the second classification label; When the third loss value is greater than a third preset threshold, the parameters of the initial classifier network are adjusted, and the step of inputting the multiple video frames corresponding to the first video and the multiple video frames corresponding to the second video into the initial classifier network for processing is returned to obtain the classification results corresponding to the first video and the second video respectively.

3. The method according to claim 2, characterized in that After calculating the third loss value based on the classification results corresponding to the first video and the second video, the first classification label, and the second classification label, the method further includes: When the third loss value is less than or equal to the third preset threshold, the training of the initial classifier network is stopped, and the trained initial classifier network is used as the video classification model.

4. A video classification device, characterized in that: The method comprises means for executing the steps of the method according to any one of claims 1 to 3.

5. A video classification device, characterized in that: The method comprises at least one processor and an interface circuit, wherein the at least one processor is configured to execute the method according to any one of claims 1 to 3.

6. A terminal device, characterized in that: The device comprises the video classification device as claimed in claim 4 or 5.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images