A method for training a video classification model and a video classification method

Abstract

The application provides a video classification model training method and a video classification method, which include: obtaining a training video and a corresponding video label thereof; dividing the training video according to a preset number to obtain a division result corresponding to the preset number; wherein the preset number is a training hyperparameter, and an optimal number optimization is determined through a preset optimization algorithm; based on the division result, performing forward inference using a pre-constructed video classification model to obtain a classification result corresponding to the preset number, and determining a target classification result from the classification result; performing loss calculation according to the target classification result and the video label, updating parameters of the pre-constructed video classification model using a calculated loss value, and obtaining a trained video classification model. The application can greatly improve the accuracy of the video classification model.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a training method for a video classification model and a video classification method. Background Technology

[0002] Currently, classifying and labeling videos using video classification models has become an important way to improve the efficiency of downstream video distribution. Video classification models are typically trained on neural networks using deep learning algorithms.

[0003] Typically, training data consists of labeled videos. However, due to the difficulty and cost of labeling, the training data often only has category labels, meaning each video only has a category label (whether it belongs to a specific category), but no labeling of specific time periods—that is, no information on which seconds of the video belong to a particular category. In this case, most of the time in each video trained by the model is irrelevant to the specific category task, with only a small portion being relevant. This results in a video classification model with highly dispersed confidence scores, significantly impacting accuracy and making it insensitive to classifying short video segments. Summary of the Invention

[0004] This invention provides a training method for a video classification model and a video classification method to solve the above-mentioned problems.

[0005] This invention provides a method for training a video classification model, comprising:

[0006] Obtain the training videos and their corresponding video tags;

[0007] The training video is segmented according to a preset number to obtain a segmentation result corresponding to the preset number; wherein, the preset number is a training hyperparameter, and it is determined by optimizing the preset number through a preset optimization algorithm;

[0008] Based on the segmentation results, and using a pre-built video classification model for forward reasoning, classification results corresponding to the preset number are obtained, and the target classification result is determined from the classification results;

[0009] Based on the target classification result and the video label, a loss is calculated, and the calculated loss value is used to update the parameters of the pre-built video classification model to obtain a trained video classification model.

[0010] According to a training method for a video classification model provided by the present invention, the preset number is determined by optimizing the preset number using a preset optimization algorithm, including:

[0011] S1, Initialize the parameters using a predetermined optimization algorithm to obtain the initial quantity;

[0012] S2, Segment the training video according to the initial number to obtain the segmentation result corresponding to the initial number;

[0013] S3, based on the segmentation results, and using a pre-built video classification model for forward reasoning, obtain classification results corresponding to the initial quantity, and determine the target classification result from the classification results;

[0014] S4. Based on the target classification result and the video label, calculate the loss and use the calculated loss value to update the parameters of the pre-built video classification model to obtain the updated video classification model and its corresponding model performance.

[0015] S5. Based on the model performance, the initial quantity is optimized using the predetermined optimization algorithm to obtain the optimized quantity;

[0016] S6, Segment the training video according to the optimized number to obtain the segmentation result corresponding to the optimized number;

[0017] S7. Repeat S3-S6 until the model performance meets the preset accuracy requirements or reaches the preset number of iterations to obtain the optimal number for the training process of the video classification model.

[0018] According to a training method for a video classification model provided by the present invention, the step of segmenting the training video according to a preset number to obtain a segmentation result corresponding to the preset number includes:

[0019] The training video is divided into segments according to the preset number to obtain video segments corresponding to the preset number.

[0020] Accordingly, the step of obtaining a classification result corresponding to the preset number based on the segmentation result and by performing forward inference using a pre-built video classification model includes:

[0021] The video clips are input into the pre-built video classification model for forward inference to obtain classification results corresponding to the preset number.

[0022] According to a training method for a video classification model provided by the present invention, the step of segmenting the training video according to a preset number to obtain a segmentation result corresponding to the preset number includes:

[0023] The training video is input into a pre-built video classification model, and forward inference is performed using each network layer in the pre-built video classification model;

[0024] During the forward inference process, the feature map output by any network layer is selected as the target feature map, and the target feature map is segmented according to the preset number to obtain sub-features corresponding to the preset number.

[0025] Accordingly, the step of obtaining a classification result corresponding to the preset number based on the segmentation result and by performing forward inference using a pre-built video classification model includes:

[0026] Based on the sub-features, obtain the classification results corresponding to the preset number.

[0027] According to the training method of a video classification model provided by the present invention, the pre-built video classification model includes a feature extraction backbone network and a detection network;

[0028] Accordingly, selecting the feature map output by any network layer as the target feature map includes:

[0029] The feature map output by any layer of the feature extraction backbone network or the detection network is selected as the target feature map.

[0030] According to a training method for a video classification model provided by the present invention, the classification result is the probability of the category to which the segmentation result belongs;

[0031] Accordingly, determining the target classification result from the classification results includes:

[0032] The classification result with the highest probability value is taken as the target classification result.

[0033] According to the training method of the video classification model provided by the present invention, the preset optimization algorithm is at least one of ant colony algorithm, genetic algorithm, simulated annealing algorithm, tabu search algorithm or particle swarm optimization algorithm.

[0034] The present invention also provides a video classification method, comprising: acquiring a video to be classified;

[0035] The video to be classified is segmented according to a preset number to obtain a segmentation result corresponding to the preset number; wherein, the preset number is a hyperparameter in a pre-trained video classification model, and the optimal number is determined by a preset optimization algorithm during the training of the video classification model.

[0036] Based on the segmentation results, and using a pre-trained video classification model for classification prediction, a prediction result corresponding to the preset number is obtained. The target prediction result is determined from the prediction results and used as the final prediction result of the video to be classified. The pre-trained video classification model is obtained by training the video classification model as described above.

[0037] The present invention also provides a training apparatus for a video classification model, comprising:

[0038] The video and tag acquisition module is used to acquire training videos and their corresponding video tags;

[0039] The segmentation module is used to segment the training video according to a preset number to obtain a segmentation result corresponding to the preset number; wherein, the preset number is a training hyperparameter, and it is determined by optimizing the preset number through a preset optimization algorithm;

[0040] The classification module is used to obtain classification results corresponding to the preset number based on the segmentation results and by using a pre-built video classification model for forward reasoning, and to determine the target classification result from the classification results;

[0041] The model update module is used to calculate the loss based on the target classification result and the video label, and use the calculated loss value to update the parameters of the pre-built video classification model to obtain a trained video classification model.

[0042] The present invention also provides a video classification device, comprising:

[0043] The module for acquiring videos to be classified is used to acquire videos to be classified.

[0044] The video segmentation module is used to segment the video to be classified according to a preset number to obtain a segmentation result corresponding to the preset number; wherein, the preset number is a hyperparameter in a pre-trained video classification model, and the optimal number is determined by a preset optimization algorithm during the training of the video classification model.

[0045] The prediction module is used to perform classification prediction based on the segmentation result and using a pre-trained video classification model to obtain a prediction result corresponding to the preset number, determine the target prediction result from the prediction result, and use it as the final prediction result of the video to be classified; wherein, the pre-trained video classification model is obtained by a training device for the video classification model as described above.

[0046] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements a training method for any of the video classification models described above or the video classification method described above.

[0047] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a training method for any of the video classification models described above or the video classification method described above.

[0048] This invention provides a training method for a video classification model and a video classification method. The training method involves segmenting the training video according to a preset number m, obtaining segmentation results corresponding to the preset number, and then using a pre-built video classification model to perform forward inference based on the segmentation results to obtain classification results corresponding to the preset number. The target classification result is then determined from these classification results. Therefore, on video data without detailed annotations of occurrence times, the video classification model focuses more on task-related video segments during training, significantly improving the accuracy of the video classification model, increasing its sensitivity to specific behaviors within a short timeframe, raising the model's confidence when a specific behavior occurs, and lowering the confidence when it does not occur, thus greatly improving the algorithm's accuracy. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating the training method of the video classification model provided in an embodiment of the present invention;

[0051] Figure 2 This is a flowchart illustrating the video classification method provided in an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram of the training device structure for the video classification model provided in an embodiment of the present invention;

[0053] Figure 4 This is a schematic diagram of the video classification device provided in an embodiment of the present invention;

[0054] Figure 5 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0056] Figure 1This is a flowchart illustrating the training method of the video classification model provided in this embodiment of the invention; as shown below. Figure 1 As shown, the training method for this video classification model includes:

[0057] S101, Obtain the training video and its corresponding video tag. The training video is a single, continuous video segment with a corresponding video tag. For example, when the video classification model is applied to short video classification scenarios, the video tag might be "games," "food," "beauty," etc. In a driver fatigue detection scenario within a driver monitoring system, the video tag might indicate whether the video shows driver fatigue or not. Specific application scenarios are not limited in this invention.

[0058] S102, the training video is segmented according to a preset number m to obtain segmentation results corresponding to the preset number, i.e., m segmentation results.

[0059] Wherein, the preset quantity m is a training hyperparameter, which has a significant impact on the accuracy of the video classification model. In this invention, automatic machine learning (autoML) is used to automatically optimize this hyperparameter and rapidly iterate to obtain the optimal hyperparameter. More specifically, the training hyperparameter m is determined by optimizing the optimal quantity through a preset optimization algorithm, wherein the preset optimization algorithm is at least one of ant colony optimization, genetic algorithm, simulated annealing algorithm, tabu search algorithm, or particle swarm optimization algorithm.

[0060] Optionally, in this embodiment, the preset optimization algorithm is the particle swarm optimization algorithm.

[0061] In addition, the segmentation of the training video according to the preset number m can be performed before the training video is input into the video classification model, or it can be performed after the feature map is input into the video classification model. For details of the segmentation process, please refer to the description below.

[0062] S103, based on the segmentation results, and using a pre-built video classification model for forward reasoning, obtain classification results corresponding to the preset number, and determine the target classification result from the classification results.

[0063] Specifically, a pre-built video classification model performs forward inference on the input segmentation results to obtain m classification results. Each classification result represents the probability value of the category to which the segmentation result belongs. Taking fatigue detection as an example, the classification result is the probability that the segmented segment contains fatigue behavior. The classification result with the largest probability value among the m classification results is taken as the target classification result, which is the classification result of the entire training video.

[0064] It should be noted that, since the training video is segmented according to a preset number m, when the video label includes fatigue behavior, only a small number of the m segmentation results may show fatigue behavior, meaning a small number of classification results indicate that the video belongs to fatigue, while most segmentation results do not show fatigue behavior, meaning most classification results indicate that the video belongs to non-fatigue. Under this condition, the video belonging to fatigue is output as the classification result for the entire training video. The video classification model trained on this basis increases the confidence when a specific behavior occurs in the video and decreases the confidence when it does not occur, thus concentrating the confidence scores of the classification results for the entire training video and greatly improving the classification sensitivity for shorter videos.

[0065] S104, calculate the loss based on the target classification result and the video label, and use the calculated loss value to update the parameters of the pre-built video classification model to obtain a trained video classification model.

[0066] Specifically, loss functions related to conventional classification algorithms are used to calculate the loss between the target classification result and the video label to obtain a loss value. Based on this loss value, the parameters of the pre-built video classification model are updated using the backpropagation algorithm. Steps S102-S104 are repeated. When the pre-built video classification model reaches a convergence state, the parameter update is stopped, and the trained video classification model is obtained.

[0067] The video classification model training method provided in this invention segmentes the training video according to a preset number m, obtains segmentation results corresponding to the preset number, and performs forward inference based on the segmentation results using a pre-built video classification model to obtain classification results corresponding to the preset number. The target classification result is then determined from these classification results. Therefore, on video data without detailed annotations of occurrence times, the video classification model focuses more on task-related video segments during training, significantly improving the accuracy of the video classification model, increasing its sensitivity to specific behaviors within a short timeframe, raising the model's confidence when a specific behavior occurs, and lowering the confidence when it does not occur, thus greatly improving the algorithm's accuracy.

[0068] In some embodiments of the present invention, the preset quantity is determined by optimizing the preset quantity using a preset optimization algorithm, including:

[0069] S1. Initialize the parameters using a predetermined optimization algorithm to obtain the initial quantity 'a'. Taking the particle swarm optimization algorithm as an example, parameter initialization involves setting the size, initial velocity, and initial position of the particle swarm. Based on the size, initial velocity, and initial position of the particle swarm, as well as the relevant objective function (which includes optimizing model performance), obtain the current global optimum of the entire particle swarm and use this current global optimum as the initial quantity.

[0070] S2, the training video is segmented according to the initial number a to obtain a segmentation results corresponding to the initial number, i.e., a segmentation results.

[0071] S3. Based on the segmentation results, and using a pre-built video classification model for forward reasoning, obtain the classification results corresponding to the initial number, and determine the target classification result from the classification results, that is, determine the classification result with the highest probability value from a segmentation results as the target classification result.

[0072] S4. Based on the target classification result and the video label, calculate the loss and use the calculated loss value to update the parameters of the pre-built video classification model to obtain the updated video classification model and its corresponding model performance.

[0073] Specifically, after using a segmentation results for forward inference and directional propagation, an updated video classification model is obtained, and the model performance, such as the model's classification accuracy and loss value, is also obtained.

[0074] S5. Based on the model performance, the initial quantity is optimized using the predetermined optimization algorithm to obtain the optimized quantity.

[0075] In this step, model performance is used as a guide for finding the optimal quantity. Based on the initial quantity a, optimization is performed to obtain the optimized quantity b.

[0076] S6. Segment the training video according to the optimized number to obtain segmentation results corresponding to the optimized number, that is, obtain b segmentation results.

[0077] S7. Repeat S3-S6 until the model performance meets the preset accuracy requirement or reaches the preset number of iterations to obtain the optimal quantity for the video classification model training process. That is, perform forward inference and directional propagation based on b segmentation results to obtain a new updated video classification model. Then, optimize the optimized quantity b based on the model performance to obtain a new optimized quantity b'. This process is repeated until the preset number of iterations is reached or the accuracy of the model performance reaches the preset accuracy threshold. The final optimized quantity is then used as the optimal quantity m, which is the preset quantity for subsequent model training.

[0078] The video classification model training method provided in this invention continuously optimizes a preset number m based on the training results of the video classification model, thereby obtaining the most suitable number of segments for the training video and improving the accuracy and sensitivity of the video classification model. Furthermore, the optimization algorithm can significantly accelerate the optimization process of hyperparameter m, quickly obtaining the optimal hyperparameters and speeding up the training process.

[0079] In some embodiments of the present invention, the segmentation of the training video can be performed before inputting it into the model, or it can be performed during the forward inference process of the model. The following describes the segmentation before inputting it into the model in detail:

[0080] The step of segmenting the training video according to a preset number to obtain a segmentation result corresponding to the preset number includes:

[0081] The training video is segmented according to the preset number to obtain video segments (i.e., segmentation results) corresponding to the preset number. Accordingly, based on the segmentation results, and using a pre-built video classification model for forward inference to obtain classification results corresponding to the preset number, the process includes:

[0082] The video clips are input into the pre-built video classification model for forward inference to obtain classification results corresponding to the preset number.

[0083] Specifically, the n-frame training video is directly divided into m video segments, and then the m video segments are directly input into the model to obtain m classification results corresponding to the video segments.

[0084] The training method for the video classification model provided in this embodiment of the invention divides the training video into m video segments before inputting it into the model, and determines the target classification result based on the classification results of the m video segments, thereby improving the accuracy and sensitivity of the model.

[0085] In some embodiments of the present invention, segmentation can also be performed during the forward inference process of the model. Specifically, segmenting the training video according to a preset number to obtain a segmentation result corresponding to the preset number includes:

[0086] The training video is input into a pre-built video classification model, and forward inference is performed using the various network layers in the pre-built video classification model.

[0087] During the forward inference process, the feature map output by any network layer is selected as the target feature map, and the target feature map is segmented according to the preset number to obtain sub-features corresponding to the preset number.

[0088] Accordingly, the step of obtaining a classification result corresponding to the preset number based on the segmentation result and by performing forward inference using a pre-built video classification model includes:

[0089] Based on the sub-features, obtain the classification results corresponding to the preset number.

[0090] Specifically, as known from existing technology, conventional video classification models consist of multiple network layers. The output of one network layer serves as the input to the next, and the classification result is ultimately obtained through the calculations of each network layer, thus completing forward inference. In this process, the output (i.e., feature map) of a certain network layer is selected as the target feature map. This N-dimensional (here, N is the time dimension) feature map is segmented according to a preset number m to obtain m sub-features (i.e., segmentation results). These m sub-features are then input into subsequent network layers to continue forward inference, thereby obtaining the classification result corresponding to the sub-features.

[0091] More specifically, the pre-built video classification model includes a feature extraction backbone network (i.e., a Backbone network) and a detection network (including a Neck network, a Bottleneck network, and a Head network).

[0092] The Backbone network, used for feature extraction, extracts information from the classification results and is used by the detection network; it can be ResNet, VGG, etc. The Neck network, positioned between the Backbone and Head networks, better utilizes the features extracted by the Backbone. The Bottleneck adjusts the input and output dimensions of the two networks. The Head uses the features output by the Neck network for prediction.

[0093] Accordingly, selecting the feature map output by any network layer as the target feature map includes:

[0094] The feature map output by any layer of the feature extraction backbone network or the detection network is selected as the target feature map. Specifically, the feature map output by a certain layer in the feature extraction backbone network can be selected as the target feature map, or the feature map output by a certain layer in the detection network after the feature extraction backbone network can be selected as the target feature map.

[0095] Optionally, in this embodiment, the feature map output of any network layer in the detection network after the feature extraction backbone network is selected for segmentation. This feature map contains more useful information. Compared with directly segmenting the training video into m video segments or segmenting the output of the earlier network layers in the model, segmenting the output of the later network layers can reduce the subsequent computation.

[0096] The training method for the video classification model provided in this embodiment of the invention obtains m sub-features by segmenting the feature map during the forward inference process of the model, and obtains m classification results based on the sub-features, thereby determining the target classification result, thereby improving the accuracy and sensitivity of the model and reducing the amount of computation.

[0097] Figure 2 This is a flowchart illustrating the video classification method provided in an embodiment of the present invention; as shown below. Figure 2 As shown, this video classification method includes:

[0098] S201, Obtain the video to be classified. For example, obtain short videos in a short video classification scenario, or cabin monitoring videos in a driver fatigue detection scenario, etc.

[0099] S202, the video to be classified is segmented according to a preset number to obtain a segmentation result corresponding to the preset number.

[0100] The preset quantity is a hyperparameter in a pre-trained video classification model, and the optimal quantity is determined by a preset optimization algorithm during the training process of the video classification model.

[0101] In this step, the video to be classified obtained above is segmented. This segmentation can be done by directly dividing the video into a predetermined number of video segments, or by segmenting the feature map during the forward inference process of the video to be classified to obtain a predetermined number of sub-features. See the description below for details.

[0102] It should be noted that the preset number in the video classification method is a fixed hyperparameter, which is determined during the training of the video classification model. The preset number during the training process is continuously updated by the optimization algorithm until a fixed value is obtained after the training is completed. This fixed value is the preset number in the video classification method.

[0103] S203, based on the segmentation results, and using a pre-trained video classification model to perform classification prediction, obtain prediction results corresponding to the preset number, determine the target prediction result from the prediction results, and use it as the final prediction result of the video to be classified.

[0104] The pre-trained video classification model is obtained through the training method described above.

[0105] In this step, each segmentation result corresponds to a prediction result. The prediction result with the highest probability value is output as the target result, thereby determining the category to which the video to be classified belongs.

[0106] The video classification method provided in this invention segmentes the video to be classified according to a preset number m, then makes predictions to obtain m prediction results, and finally determines the final category of the video to be classified from the m prediction results. This improves the confidence of the trained video classification model when a specific behavior occurs and reduces the confidence when the behavior does not occur, thus greatly improving the accuracy of the algorithm.

[0107] In some embodiments of the present invention, the step of segmenting the video to be classified according to a preset number to obtain a segmentation result corresponding to the preset number includes:

[0108] The video to be classified is divided into segments according to the preset number to obtain video segments corresponding to the preset number.

[0109] Accordingly, the step of performing classification prediction based on the segmentation result and using a pre-trained video classification model to obtain a prediction result corresponding to the preset number includes:

[0110] The video clip is input into the pre-trained video classification model for classification prediction, and a prediction result corresponding to the preset number is obtained.

[0111] It should be noted that the process of segmenting the video to be classified into m video segments is similar to the training method of a video classification model, except that m is a fixed value and will not be continuously optimized and updated as the model is trained. The detailed segmentation process will not be elaborated here.

[0112] In some embodiments of the present invention, the step of segmenting the video to be classified according to a preset number to obtain a segmentation result corresponding to the preset number includes:

[0113] The video to be classified is input into a pre-trained video classification model, and forward inference is performed using each network layer in the pre-trained video classification model.

[0114] During the forward inference process, the feature map output by any network layer is selected as the target feature map, and the target feature map is segmented according to the preset number to obtain sub-features corresponding to the preset number.

[0115] Accordingly, the step of performing classification prediction based on the segmentation result and using a pre-trained video classification model to obtain a prediction result corresponding to the preset number includes:

[0116] Based on the sub-features, obtain the prediction results corresponding to the preset quantity.

[0117] It should be noted that the process of segmenting the feature map corresponding to the video to be classified into m sub-features, as mentioned above, is similar to the training method of video classification models, except that m is a fixed value and will not be continuously optimized and updated as the model is trained. The detailed segmentation process will not be elaborated here.

[0118] The training apparatus for the video classification model provided by the present invention will be described below. The training apparatus for the video classification model described below can be referred to in correspondence with the training method for the video classification model described above.

[0119] Figure 3 This is a schematic diagram of the training device structure for the video classification model provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the training device for the video classification model includes a video and label acquisition module 301, a segmentation module 302, a classification module 303, and a model update module 304.

[0120] The video and tag acquisition module 301 is used to acquire training videos and their corresponding video tags.

[0121] The training video is a single, continuous video segment, each with a corresponding video tag. For example, when the video classification model is applied to short video classification scenarios, the video tag might be "games," "food," "beauty," etc. In a driver fatigue detection scenario within a driver monitoring system, the video tag might indicate whether the video shows driver fatigue or not. This invention does not limit the specific application scenarios.

[0122] The segmentation module 302 is used to segment the training video according to a preset number m, and obtain segmentation results corresponding to the preset number, i.e., m segmentation results.

[0123] Wherein, the preset quantity m is a training hyperparameter, which has a significant impact on the accuracy of the video classification model. In this invention, automatic machine learning (autoML) is used to automatically optimize this hyperparameter and rapidly iterate to obtain the optimal hyperparameter. More specifically, the training hyperparameter m is determined by optimizing the optimal quantity through a preset optimization algorithm, wherein the preset optimization algorithm is at least one of ant colony optimization, genetic algorithm, simulated annealing algorithm, tabu search algorithm, or particle swarm optimization algorithm.

[0124] Optionally, in this embodiment, the preset optimization algorithm is the particle swarm optimization algorithm.

[0125] In addition, the segmentation of the training video according to the preset number m can be performed before the training video is input into the video classification model, or it can be performed after the feature map is input into the video classification model. For details of the segmentation process, please refer to the description above.

[0126] The classification module 303 is used to obtain classification results corresponding to the preset number based on the segmentation results and by using a pre-built video classification model for forward reasoning, and to determine the target classification result from the classification results.

[0127] Specifically, a pre-built video classification model performs forward inference on the input segmentation results to obtain m classification results. Each classification result represents the probability value of the category to which the segmentation result belongs. Taking fatigue detection as an example, the classification result is the probability that the segmented segment contains fatigue behavior. The classification result with the largest probability value among the m classification results is taken as the target classification result, which is the classification result of the entire training video.

[0128] It should be noted that, since the training video is segmented according to a preset number of segments *m*, when the video label includes fatigue behavior, only a small number of the *m* segmentation results may show fatigue behavior, meaning a small number of classification results indicate the video belongs to fatigue. Conversely, most segmentation results may not show fatigue behavior, meaning most classification results indicate the video belongs to non-fatigue. Under this condition, the video belonging to fatigue behavior is output as the classification result for the entire training video. The video classification model trained on this basis can significantly improve the classification sensitivity of shorter videos, increasing the model's confidence when specific behaviors occur in the video and decreasing the confidence when they do not.

[0129] The model update module 304 is used to perform loss calculation based on the target classification result and the video label, and use the calculated loss value to update the parameters of the pre-built video classification model to obtain a trained video classification model.

[0130] Specifically, the loss function associated with conventional classification algorithms is used to calculate the loss between the target classification result and the video label to obtain the loss value. Based on the loss value, the parameters of the pre-built video classification model are updated through the backpropagation algorithm. The segmentation module 302, the classification module 303, and the model update module 304 are repeatedly executed. When the pre-built video classification model reaches the convergence state, the parameter update is stopped, and the trained video classification model is obtained.

[0131] In addition, after obtaining the trained video classification model, the video to be classified is directly input into the trained video classification model to obtain the classification result of the video to be classified.

[0132] The training apparatus for the video classification model provided in this embodiment of the invention segments the training video according to a preset number m, obtains segmentation results corresponding to the preset number, and performs forward inference based on the segmentation results using a pre-built video classification model to obtain classification results corresponding to the preset number, and determines the target classification result from the classification results. Therefore, on video data without detailed annotations of occurrence times, the video classification model focuses more on task-related video segments during training, significantly improving the accuracy of the video classification model, increasing its sensitivity to specific behaviors in short time intervals, raising the model's confidence when specific behaviors occur, and lowering the confidence when they do not occur, greatly improving the algorithm's accuracy.

[0133] Figure 4 This is a schematic diagram of the video classification device provided in an embodiment of the present invention, such as... Figure 4 As shown, the video classification device includes a video acquisition module 401, a video segmentation module 402, and a prediction module 403.

[0134] The unclassified video acquisition module 401 is used to acquire videos to be classified.

[0135] The video segmentation module 402 is used to segment the video to be classified according to a preset number, and obtain a segmentation result corresponding to the preset number.

[0136] The preset quantity is a hyperparameter in a pre-trained video classification model, and the optimal quantity is determined by a preset optimization algorithm during the training process of the video classification model.

[0137] The prediction module 403 is used to perform classification prediction based on the segmentation result and using a pre-trained video classification model to obtain a prediction result corresponding to the preset number, determine the target prediction result from the prediction result, and use it as the final prediction result of the video to be classified.

[0138] The pre-trained video classification model is obtained through a training device for the video classification model as described above.

[0139] The video classification device provided in this embodiment of the invention segments the video to be classified according to a preset number m, then makes predictions to obtain m prediction results, and finally determines the final category of the video to be classified from the m prediction results. This improves the confidence of the trained video classification model when a specific behavior occurs and reduces the confidence when the behavior does not occur, thus greatly improving the accuracy of the algorithm.

[0140] Figure 5 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as... Figure 5As shown, the electronic device may include a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a training method for a video classification model. The training method for the video classification model includes: acquiring training videos and their corresponding video tags; segmenting the training videos according to a preset number to obtain segmentation results corresponding to the preset number; wherein the preset number is a training hyperparameter, and it is determined by optimizing the preset number using a preset optimization algorithm; based on the segmentation results, performing forward inference using a pre-built video classification model to obtain classification results corresponding to the preset number, and determining a target classification result from the classification results; calculating a loss based on the target classification result and the video tags, and updating the parameters of the pre-built video classification model using the calculated loss value to obtain a trained video classification model.

[0141] Alternatively, it can be used to perform a video classification method, which includes: acquiring a video to be classified; segmenting the video to be classified according to a preset number to obtain a segmentation result corresponding to the preset number; wherein the preset number is a hyperparameter in a pre-trained video classification model, and its optimal number is determined by a preset optimization algorithm during the training of the video classification model; based on the segmentation result, and using the pre-trained video classification model to perform classification prediction, obtaining a prediction result corresponding to the preset number; determining a target prediction result from the prediction result, and using it as the final prediction result for the video to be classified; wherein the pre-trained video classification model is obtained through the video classification model training method described above.

[0142] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0143] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a method for training a video classification model. The method for training the video classification model includes: acquiring training videos and their corresponding video tags; segmenting the training videos according to a preset number to obtain segmentation results corresponding to the preset number; wherein the preset number is a training hyperparameter, and it is determined by optimizing the preset number using a preset optimization algorithm; based on the segmentation results, performing forward inference using a pre-built video classification model to obtain classification results corresponding to the preset number, and determining a target classification result from the classification results; calculating a loss based on the target classification result and the video tags, and updating the parameters of the pre-built video classification model using the calculated loss value to obtain a trained video classification model.

[0144] Alternatively, it can be used to perform a video classification method, which includes: acquiring a video to be classified; segmenting the video to be classified according to a preset number to obtain a segmentation result corresponding to the preset number; wherein the preset number is a hyperparameter in a pre-trained video classification model, and its optimal number is determined by a preset optimization algorithm during the training of the video classification model; based on the segmentation result, and using the pre-trained video classification model to perform classification prediction, obtaining a prediction result corresponding to the preset number; determining a target prediction result from the prediction result, and using it as the final prediction result for the video to be classified; wherein the pre-trained video classification model is obtained through the video classification model training method described above.

[0145] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0146] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a video classification model, characterized in that, include: Obtain the training videos and their corresponding video tags; The training video is input into a pre-built video classification model, and forward inference is performed using each network layer in the pre-built video classification model; During the forward inference process, the feature map output by any network layer is selected as the target feature map, and the target feature map is segmented according to a preset number to obtain sub-features corresponding to the preset number; wherein, the preset number is a training hyperparameter, and it is determined by optimizing the optimal number through a preset optimization algorithm; Based on the sub-features, forward reasoning is performed using the pre-built video classification model to obtain classification results corresponding to the preset number, and the target classification result is determined from the classification results; Based on the target classification result and the video label, a loss is calculated, and the calculated loss value is used to update the parameters of the pre-built video classification model to obtain a trained video classification model.

2. The training method for the video classification model according to claim 1, characterized in that, The preset quantity is determined by optimizing the preset quantity using a preset optimization algorithm, including: S1, Initialize the parameters using a predetermined optimization algorithm to obtain the initial quantity; S2, Segment the training video according to the initial number to obtain the segmentation result corresponding to the initial number; S3, based on the segmentation results, and using a pre-built video classification model for forward reasoning, obtain classification results corresponding to the initial quantity, and determine the target classification result from the classification results; S4. Based on the target classification result and the video label, calculate the loss and use the calculated loss value to update the parameters of the pre-built video classification model to obtain the updated video classification model and its corresponding model performance. S5. Based on the model performance, the initial quantity is optimized using the predetermined optimization algorithm to obtain the optimized quantity; S6, Segment the training video according to the optimized number to obtain the segmentation result corresponding to the optimized number; S7. Repeat S3-S6 until the model performance meets the preset accuracy requirements or reaches the preset number of iterations to obtain the optimal number for the training process of the video classification model.

3. The training method for the video classification model according to claim 1, characterized in that, The method further includes: The training video is divided into segments according to the preset number to obtain video segments corresponding to the preset number. Accordingly, the step of obtaining a classification result corresponding to the preset number based on the segmentation result and by performing forward inference using a pre-built video classification model includes: The video clips are input into the pre-built video classification model for forward inference to obtain classification results corresponding to the preset number.

4. The training method for the video classification model according to claim 1, characterized in that, The pre-built video classification model includes a feature extraction backbone network and a detection network; Accordingly, selecting the feature map output by any network layer as the target feature map includes: The feature map output by any layer of the feature extraction backbone network or the detection network is selected as the target feature map.

5. The training method for the video classification model according to claim 1, characterized in that, The classification result is the probability of the category to which the segmentation result belongs; Accordingly, determining the target classification result from the classification results includes: The classification result with the highest probability value is taken as the target classification result.

6. The training method for the video classification model according to claim 1, characterized in that, The preset optimization algorithm is at least one of the following: ant colony optimization, genetic algorithm, simulated annealing algorithm, tabu search algorithm, or particle swarm optimization.

7. A video classification method, characterized in that, include: Get the videos to be categorized; The video to be classified is segmented according to a preset number to obtain a segmentation result corresponding to the preset number; wherein, the preset number is a hyperparameter in a pre-trained video classification model, and the optimal number is determined by a preset optimization algorithm during the training of the video classification model. Based on the segmentation results, and using a pre-trained video classification model for classification prediction, a prediction result corresponding to the preset number is obtained. The target prediction result is determined from the prediction results and used as the final prediction result of the video to be classified. The pre-trained video classification model is obtained by the training method of the video classification model as described in any one of claims 1-6.

8. A training device for a video classification model, characterized in that, include: The video and tag acquisition module is used to acquire training videos and their corresponding video tags; The segmentation module is used for: The training video is input into a pre-built video classification model, and forward inference is performed using each network layer in the pre-built video classification model; During the forward inference process, the feature map output by any network layer is selected as the target feature map, and the target feature map is segmented according to a preset number to obtain sub-features corresponding to the preset number; wherein, the preset number is a training hyperparameter, and it is determined by optimizing the optimal number through a preset optimization algorithm; The classification module is used to perform forward reasoning based on the sub-features using the pre-built video classification model to obtain classification results corresponding to the preset number, and to determine the target classification result from the classification results; The model update module is used to calculate the loss based on the target classification result and the video label, and use the calculated loss value to update the parameters of the pre-built video classification model to obtain a trained video classification model.

9. A video classification device, characterized in that, include: The module for acquiring videos to be classified is used to acquire videos to be classified. The video segmentation module is used to segment the video to be classified according to a preset number to obtain a segmentation result corresponding to the preset number; wherein, the preset number is a hyperparameter in a pre-trained video classification model, and the optimal number is determined by a preset optimization algorithm during the training of the video classification model. The prediction module is used to perform classification prediction based on the segmentation result and using a pre-trained video classification model to obtain a prediction result corresponding to the preset number, determine the target prediction result from the prediction result, and use it as the final prediction result of the video to be classified; wherein the pre-trained video classification model is obtained by the training device of the video classification model as described in claim 8.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the training method of the video classification model as described in any one of claims 1 to 6 or the video classification method as described in claim 7.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the training method of the video classification model as described in any one of claims 1 to 6 or the video classification method as described in claim 7.

Images