Video tag prediction model processing method, video tag prediction method and device

Abstract

The present application relates to a processing method of a video label prediction model, a video label prediction method, an apparatus, a computer device, a storage medium and a computer program product, which can be applied to video label prediction scenarios, video classification scenarios, video recommendation scenarios, etc. The processing method of the video label prediction model includes: dividing a sample video into a plurality of video segments of equal length, performing label prediction on each video segment respectively through a video label prediction model, obtaining segment label prediction results corresponding to each video segment respectively, fusing the probabilities of the same annotated labels in the segment label prediction results corresponding to each video segment, obtaining the video label prediction results corresponding to the sample video, and training the video label prediction model according to the prediction loss constructed based on at least one annotated label of the sample video and the probability that the sample video belongs to each annotated label. The video label prediction model trained by this method can improve the accuracy of video label prediction.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for processing a video tag prediction model, a video tag prediction method, an apparatus, a computer device, a storage medium, and a computer program product. Background Art

[0002] With the development of computer technology and Internet technology, video has become one of the important channels for people to obtain information. For example, live video can provide people with real-time information, e-commerce video can provide people with product information, self-media video can provide people with information on many aspects of life, and so on.

[0003] In some scenarios, it is necessary to identify the video tags to which a video belongs, so as to classify the video based on the video tags, or recommend videos to users based on the video tags. For example, for the Tom and Jerry animation video, its tags are identified as "animation", "tom and jerry", "Tom", etc. At present, the commonly used video tag recognition method is to extract the video frame features of the target video, fuse the video frame features to obtain video features of fixed dimensions, and predict the video tags based on the video features. However, the video tags identified by this video tag prediction method are not accurate enough. Summary of the invention

[0004] Based on this, it is necessary to provide a processing method of a video tag prediction model, a video tag prediction method, an apparatus, a computer device, a storage medium and a computer program product that can improve the accuracy of video tag recognition in response to the above technical problems.

[0005] The present application provides a method for processing a video tag prediction model. The method comprises:

[0006] Acquire a sample video and at least one annotated label of the sample video, wherein the at least one annotated label belongs to a video label set;

[0007] Dividing the sample video into a plurality of video segments of equal length;

[0008] Using a video tag prediction model, a tag prediction is performed on each of the video clips to obtain a tag prediction result corresponding to each of the video clips; the tag prediction result includes a probability that the video clip belongs to each of the annotated tags in the video tag set;

[0009] The probabilities of the same labeled labels in the segment label prediction results corresponding to the video segments are merged to obtain the video label prediction results corresponding to the sample video; the video label prediction results include the probabilities of the sample video belonging to each labeled label in the video label set;

[0010] The model parameters of the video label prediction model are updated according to the prediction loss constructed based on at least one labeled label of the sample video and the probability that the sample video belongs to each labeled label, so as to obtain a trained video label prediction model.

[0011] The present application also provides a processing device for a video tag prediction model. The device comprises:

[0012] An acquisition module, used to acquire a sample video and at least one annotated label of the sample video, wherein the at least one annotated label belongs to a video label set;

[0013] A division module, used for dividing the sample video into a plurality of video segments of equal length;

[0014] A prediction module, used to perform label prediction on each of the video segments respectively through a video label prediction model to obtain segment label prediction results corresponding to each of the video segments; the segment label prediction results include the probability that the video segment belongs to each annotated label in the video label set;

[0015] A fusion module is used to fuse the probabilities of the same labeled labels in the segment label prediction results corresponding to the video segments to obtain the video label prediction results corresponding to the sample video; the video label prediction results include the probabilities that the sample video belongs to each labeled label in the video label set;

[0016] The updating module is used to update the model parameters of the video label prediction model according to the prediction loss constructed based on at least one labeled label of the sample video and the probability that the sample video belongs to each labeled label, so as to obtain a trained video label prediction model.

[0017] In one embodiment, the division module is further used to: determine the video length of the sample video; determine the segment length corresponding to the video length of the sample video based on the positive correlation between the video length and the segment length; and divide the sample video into multiple video segments of equal length according to the determined segment length.

[0018] In one embodiment, the division module is further used to: obtain a set of segment lengths; randomly select a segment length from the set of segment lengths; and divide the sample video into a plurality of video segments of equal length according to the selected segment lengths.

[0019] In one embodiment, the division module is further used to: obtain a specified segment length; and divide the sample video into a plurality of video segments of equal length according to the specified segment length.

[0020] In one embodiment, the fusion module is also used to: determine the maximum value of the probabilities corresponding to the same labeled label from the segment label prediction results corresponding to each of the video segments; use the maximum value of the probabilities corresponding to each labeled label as the probability that the sample video belongs to each labeled label; and obtain the video label prediction result corresponding to the sample video based on the probability that the sample video belongs to each labeled label.

[0021] In one embodiment, the prediction module is also used to: for each of the video clips, perform feature extraction on the video frames of the video clip through the feature extraction network in the video label prediction model to obtain feature vectors corresponding to each of the video frames; fuse the feature vectors corresponding to each of the video frames through the feature fusion network in the video label prediction model to obtain the feature vector corresponding to the video clip; perform label prediction on the feature vector corresponding to the video clip through the label recognition network in the video label prediction model to obtain a segment label prediction result corresponding to the video clip.

[0022] In one embodiment, the prediction module is also used to: determine the maximum value among the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each of the video frames through the feature fusion network in the video tag prediction model, as the vector value of each feature dimension corresponding to the video clip; and generate the feature vector corresponding to the video clip according to the vector value of each feature dimension corresponding to the video clip.

[0023] In one embodiment, the prediction module is also used to: determine the average value of the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each of the video frames through the feature fusion network in the video tag prediction model, as the vector value of each feature dimension corresponding to the video clip; and generate the feature vector corresponding to the video clip according to the vector value of each feature dimension corresponding to the video clip.

[0024] The present application provides a video tag prediction method. The method comprises:

[0025] Get the target video;

[0026] After dividing the target video into a plurality of video segments of equal length, performing label prediction on each of the video segments using a trained video label prediction model to obtain segment label prediction results corresponding to each of the video segments; the segment label prediction results include the probability that the video segment belongs to each video label;

[0027] The probabilities of the same video labels in the segment label prediction results corresponding to the video segments are merged to obtain the video label prediction results corresponding to the target video; the video label prediction results include the probabilities that the target video belongs to each video label;

[0028] According to the probability that the target video belongs to each video tag, a video tag is added to the target video.

[0029] The present application also provides a video tag prediction device. The device comprises:

[0030] An acquisition module, used to acquire a target video;

[0031] A prediction module is used to divide the target video into a plurality of video segments of equal length, and then perform label prediction on each of the video segments using a trained video label prediction model to obtain segment label prediction results corresponding to each of the video segments; the segment label prediction results include probabilities that the video segments belong to each video label;

[0032] A fusion module is used to fuse the probabilities of the same video labels in the segment label prediction results corresponding to the video segments to obtain the video label prediction results corresponding to the target video; the video label prediction results include the probabilities that the target video belongs to each video label;

[0033] The identification module is used to add a video tag to the target video according to the probability that the target video belongs to each video tag.

[0034] In one embodiment, the prediction module is also used to: for each of the video clips, perform feature extraction on the video frames of the video clip through the feature extraction network in the video label prediction model to obtain feature vectors corresponding to each of the video frames; fuse the feature vectors corresponding to each of the video frames through the feature fusion network in the video label prediction model to obtain the feature vector corresponding to the video clip; perform label prediction on the feature vector corresponding to the video clip through the label recognition network in the video label prediction model to obtain a segment label prediction result corresponding to the video clip.

[0035] In one embodiment, the prediction module is also used to: determine the maximum value among the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each of the video frames through the feature fusion network in the video tag prediction model, as the vector value of each feature dimension corresponding to the video clip; and generate the feature vector corresponding to the video clip according to the vector value of each feature dimension corresponding to the video clip.

[0036] In one embodiment, the prediction module is also used to: determine the average value of the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each of the video frames through the feature fusion network in the video tag prediction model, as the vector value of each feature dimension corresponding to the video clip; and generate the feature vector corresponding to the video clip according to the vector value of each feature dimension corresponding to the video clip.

[0037] In one embodiment, the fusion module is also used to: determine the maximum value of the probabilities corresponding to the same labeled label from the segment label prediction results corresponding to each of the video segments; use the maximum value of the probabilities corresponding to each labeled label as the probability that the sample video belongs to each labeled label; and obtain the video label prediction result corresponding to the sample video based on the probability that the sample video belongs to each labeled label.

[0038] In one embodiment, the video tag prediction device also includes a push module, which is used to: determine the interest tags corresponding to the target account; when the interest tags include at least one video tag added for the target video, push the target video to the terminal corresponding to the target account.

[0039] In one embodiment, the video tag prediction device also includes a push module, which is used to: add video tags to the video clip according to the probability that the video clip belongs to each video tag; determine the interest tags corresponding to the target account; when the interest tags include at least one video tag added to the video clip, push the video clip to the terminal corresponding to the target account.

[0040] The present application also provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the processing method of the video tag prediction model and / or the steps of the video tag prediction method when executing the computer program.

[0041] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the processing method of the video tag prediction model and / or the steps of the video tag prediction method.

[0042] The present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the processing method of the video tag prediction model and / or the steps of the video tag prediction method are implemented.

[0043] The processing method, video label prediction method, device, computer equipment, storage medium and computer program product of the above-mentioned video label prediction model first divide the sample video into multiple video segments of equal length, and then use the video label prediction model to predict the label of each video segment respectively, and obtain the segment label prediction results corresponding to each video segment. Since the sample video is divided and then the features are extracted based on the video segments, the video length based on the label prediction is shortened, and the label prediction can be performed while reducing the loss of video content information, thereby improving the accuracy of the label prediction of each video segment during the training process. In addition, different sample videos have different lengths, so the lengths of video segments of different sample videos are also different, so that the video label prediction model can adapt to the changes in different video lengths; then, the probability of the same annotation label in the segment label prediction results corresponding to each video segment is fused to obtain the video label prediction result corresponding to the sample video, so that the video label prediction result can be close to the annotation label corresponding to the sample video as a whole; then, the video label prediction model is trained based on the prediction loss constructed by at least one annotation label of the sample video and the probability that the sample video belongs to each annotation label, which can improve the training performance of the model, thereby improving the accuracy of the video label prediction of the trained video label prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A diagram showing an application environment of a video tag prediction method in an embodiment;

[0045] Figure 2 A schematic diagram of a process flow of a method for processing a video tag prediction model in one embodiment;

[0046] Figure 3 It is a schematic diagram of a process of fusing segment label prediction results of various video segments in one embodiment;

[0047] Figure 4 A schematic diagram of a process for obtaining a video tag prediction result corresponding to a sample video in one embodiment;

[0048] Figure 5 A schematic diagram of a process for obtaining a segment label prediction result corresponding to a video segment in one embodiment;

[0049] Figure 6 A schematic flow chart of a method for processing a video tag prediction model in another embodiment;

[0050] Figure 7 A schematic diagram of a flow chart of a video tag prediction method in one embodiment;

[0051] Figure 8 A schematic diagram of a process for obtaining a video tag prediction result corresponding to a target video in one embodiment;

[0052] Fig. 9 is a structural block diagram of a processing device for a video tag prediction model in one embodiment;

[0053] Fig.10 is a structural block diagram of a video tag prediction device in one embodiment;

[0054] Fig.11 is an internal structure diagram of a computer device in one embodiment;

[0055] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] The processing method of the video tag prediction model and the video tag prediction method provided in the embodiments of the present application involve artificial intelligence (AI) technology. Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.

[0058] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0059] The processing method of the video tag prediction model provided in the embodiment of the present application mainly relates to the machine learning technology (Machine Learning, ML) of artificial intelligence. Machine learning is a multi-field interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all fields of artificial intelligence. Machine learning and deep learning generally include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formula-based learning.

[0060] For example, in an embodiment of the present application, a video label prediction model is trained based on the prediction loss constructed based on at least one labeled label of a sample video and the probability that the sample video belongs to each labeled label, and finally a video label prediction model for identifying video labels is obtained.

[0061] The video tag prediction method provided in the embodiment of the present application mainly relates to the computer vision technology (Computer Vision, CV) of artificial intelligence. Computer vision is a science that studies how to make machines "see". To put it more concretely, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing to make computer processing more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology generally includes image processing, image segmentation, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, synchronous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0062] The video label prediction method provided in the embodiment of the present application mainly relates to the video semantic understanding technology (VSU) in the field of computer vision technology. For example, in the embodiment of the present application, after the target video is divided into a plurality of video segments of equal length, the segment label prediction results corresponding to each video segment are obtained through the trained video label prediction model, and the probabilities corresponding to the same video labels in the segment label prediction results corresponding to each video segment are fused to obtain the video label prediction results corresponding to the target video.

[0063] The processing method of the video tag prediction model provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. In one embodiment, taking the processing method of the video tag prediction model provided by the server 104 in the embodiment of the present application as an example, the terminal 102 obtains a sample video and at least one annotated tag of the sample video, at least one annotated tag belongs to a video tag set, and sends the sample video and at least one annotated tag of the sample video to the server 104, and the server 104 divides the sample video into a plurality of video clips of equal length; the video tag prediction model is used to predict the tags of each video clip, and the segment tag prediction results corresponding to each video clip are obtained, and the segment tag prediction results include the probability that the video clip belongs to each annotated tag in the video tag set; the probabilities corresponding to the same annotated tags in the segment tag prediction results corresponding to each video clip are merged to obtain the video tag prediction results corresponding to the sample video, and the video tag prediction results include the probability that the sample video belongs to each annotated tag in the video tag set; the video tag prediction model is trained according to the prediction loss constructed based on at least one annotated tag of the sample video and the probability that the sample video belongs to each annotated tag.

[0064] The video tag prediction method provided in the embodiment of the present application can also be applied to Figure 1 In the application environment shown. In one embodiment, taking the server 104 executing the video tag prediction method provided in the embodiment of the present application as an example, the terminal 102 obtains the target video and sends the target video to the server 104. After the server 104 divides the target video into a plurality of video segments of equal length, the label prediction is performed on each video segment through the trained video tag prediction model to obtain the segment tag prediction results corresponding to each video segment; the segment tag prediction results include the probability that the video segment belongs to each video tag; the probabilities corresponding to the same video tags in the segment tag prediction results corresponding to each video segment are merged to obtain the video tag prediction results corresponding to the target video; the video tag prediction results include the probability that the target video belongs to each video tag; according to the probability that the target video belongs to each video tag, a video tag is added to the target video.

[0065] Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be, but is not limited to, various desktop computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented as an independent server or a server cluster or cloud server composed of multiple servers.

[0066] The video tag prediction method provided in the embodiments of the present application can be applied to video tag prediction scenarios, video classification scenarios, video recommendation scenarios, and the like.

[0067] For example, for a large number of videos on a video website or in a video application, the method provided in the embodiment of the present application can be used to parse the video content offline or online to obtain the video tags corresponding to the videos. Based on the video tags corresponding to the videos, the large number of videos can be classified.

[0068] For another example, based on the video tags corresponding to the video, the video that the user is interested in can be recommended. In addition, since in the embodiment of the present application, after the target video is divided into multiple video segments of equal length, the segment tag prediction results corresponding to each video segment are obtained through the trained video tag prediction model, the video segments that the user is interested in can be recommended based on the segment tag prediction results, rather than the lengthy complete video.

[0069] In one embodiment, Figure 2 As shown, a method for processing a video tag prediction model is provided, and the method is applied to a computer device ( Figure 1 The terminal 102 or server 104 in the example is used for explanation, and the following steps are included:

[0070] Step S202: obtaining a sample video and at least one annotated label of the sample video, wherein the at least one annotated label belongs to a video label set.

[0071] Among them, the sample video is the video data used to train the video label prediction model in the embodiment of the present application. The video label prediction model is a model that has the ability to predict video labels after training. Specifically, it can be a model trained with sample videos and corresponding annotated labels as training data for predicting video labels for videos, such as predicting at least one video label to which the video belongs. Video labels are used to describe the category to which the video belongs. The category can be a coarse-grained category, such as "animation", or a fine-grained category, such as "Tom and Jerry", "Tom", etc. The annotated label is a video label annotated on the sample video.

[0072] In one embodiment, when a video label prediction model is trained using sample videos, a single label may be annotated on the sample video to train the video label prediction model to have a single label prediction capability, such as the Tom and Jerry animation video, whose label may be "animation"; or multiple label labels may be annotated on the sample video to train the video label prediction model to have a multi-label prediction capability, such as the Tom and Jerry animation video, whose label may also be "animation", "Tom and Jerry", and "Tom". The label labels corresponding to each sample video are taken as a union to obtain a video label set.

[0073] In one embodiment, the computer device obtains sample videos, each of which has a single annotated label, and trains a video label prediction model based on the sample videos and the corresponding single annotated labels to obtain a video label prediction model for single label prediction of the video. In other embodiments, the computer device obtains sample videos, each of which has at least two annotated labels, and trains a video label prediction model based on the sample videos and the corresponding at least two annotated labels to obtain a video label prediction model for multi-label prediction of the video.

[0074] The following is an introduction to the inventor’s thinking:

[0075] In the video label prediction scheme of traditional technology, the following steps are usually included: ① extract the video frame features of the target video through the model, ② fuse the video frame features to obtain the video features of fixed dimensions, and ③ predict the video label based on the video features. The inventor found that, on the one hand, when the target video is long, the video features obtained through steps ① and ② will lose a lot of video content information, thereby affecting the accuracy of the video label prediction in step ③; on the other hand, when training the model, the annotator annotates the sample video based on subjective feelings. The video label annotated by the annotator actually reflects the video label corresponding to the sample video as a whole. After the video features obtained by the model lose a lot of video content information, it is difficult to predict the prediction results that match the video labels annotated by the annotator, resulting in poor model training performance.

[0076] In view of the above problems, the inventors first divide the sample video into multiple video segments of equal length, and then use the video label prediction model to predict the labels of each video segment respectively, and fuse the probabilities of the same annotation labels in the segment label prediction results corresponding to each video segment to obtain the video label prediction results corresponding to the sample video. Firstly, by shortening the video length used to generate video features, the video content information loss of the video features is reduced, and the accuracy of label prediction during training is improved. Secondly, the predicted video label prediction results can be close to the annotation labels corresponding to the sample video as a whole, that is, they are in line with the annotation ideas of the annotator, thereby improving the training performance of the model, and then improving the accuracy of video label prediction by the trained video label prediction model.

[0077] Step S204: divide the sample video into a plurality of video segments of equal length.

[0078] In one embodiment, the computer device obtains the dynamically generated segment length, and divides the sample video into a plurality of video segments of equal length according to the dynamically generated segment length, and the length of each video segment is the dynamically generated segment length. The segment length is the length of the video segment, and the segment length can be represented by the duration of the video segment, the number of frames of the video segment, etc. Since the segment length is dynamically generated, this is equivalent to data augmentation of the training data, and the video label prediction model is trained using video segments of different lengths, so that the trained video label prediction model can process video segments of different lengths.

[0079] In one embodiment, the computer device obtains a specified segment length, and divides the sample video into a plurality of video segments of equal length according to the specified segment length, and the length of each video segment is the specified segment length. The video tag prediction model is trained using the video segments of the specified length, so that the trained video tag prediction model can have superior processing performance for video segments of the length. Optionally, the computer device obtains multiple segment lengths, and trains the video tag prediction model using each segment length as the specified segment length in turn, so that the trained video tag prediction model can have superior processing performance for video segments of different lengths.

[0080] For example, for a sample video V = {V 1 ,V 2 ,…,V M}, where M indicates that the sample video V has M video frames. The computer device obtains the segment length t, and divides the sample video V into multiple video segments of equal length according to the segment length t, each video segment has t video frames, so the sample video V has M / t=N video segments. If the sample video V cannot be divided by t, the computer device can add a blank frame to the last video segment, so that the last video segment also has t video frames.

[0081] Step 206 , perform label prediction on each video segment respectively through the video label prediction model to obtain segment label prediction results corresponding to each video segment respectively; the segment label prediction results include the probability that the video segment belongs to each annotated label in the video label set.

[0082] In one embodiment, the number of video label prediction models is single, and the computer device inputs each video clip into the video label prediction model, and performs label prediction on each video clip in turn through the video label prediction model to obtain the segment label prediction results corresponding to each video clip. In other embodiments, the number of video label prediction models is more than one, and the computer device inputs each video clip into more than one video label prediction model, and performs label prediction on the input video clip through each video label prediction model to obtain the segment label prediction results corresponding to each video clip. Optionally, more than one video label prediction model may adopt the same or different model structures.

[0083] In one embodiment, the segment label prediction result is a multi-dimensional vector, the dimension of the segment label prediction result is consistent with the number of annotated labels in the video label set, and the vector value of each dimension of the segment label prediction result represents the probability that the video segment belongs to the annotated label corresponding to the dimension.

[0084] For example, assuming that the video label set includes 5 annotated labels, the computer device predicts labels for video clip 1, video clip 2, and video clip 3 respectively through the video label prediction model, and obtains the fragment label prediction result 1 "[0.10.2 0.1 0.5 0.1]", the fragment label prediction result 2 "[0.3 0.2 0.1 0.3 0.1]" and the fragment label prediction result 3 "[0.2 0.2 0.4 0.1 0.1]", and each fragment label prediction result is a 5-dimensional vector. Taking the fragment label prediction result 1 "[0.1 0.2 0.1 0.5 0.1]" as an example, "0.2" can represent the probability that video clip 1 is the second annotated label in the video label set.

[0085] Step S208, the probabilities of the same labeled labels in the segment label prediction results corresponding to each video segment are merged to obtain the video label prediction results corresponding to the sample video; the video label prediction results include the probabilities that the sample video belongs to each labeled label in the video label set.

[0086] In one embodiment, the video tag prediction result is a multi-dimensional vector, the dimension of the video tag prediction result is consistent with the number of annotated tags in the video tag set, and the vector value of each dimension of the video tag prediction result represents the probability that the sample video belongs to the annotated tag corresponding to the dimension.

[0087] In one embodiment, referring to Figure 3 , Figure 3The figure is a flow chart of fusing the segment label prediction results corresponding to each video segment in one embodiment. It can be seen that the computer device obtains segment label prediction result 1, segment label prediction result 2, and segment label prediction result 3. Each segment label prediction result is a G-dimensional vector, and the vector value of each dimension represents the probability that the video segment belongs to the label corresponding to the dimension. The probabilities corresponding to the same label in each segment label prediction result are fused to obtain the probability that the sample video belongs to each label, and the video label prediction result corresponding to the sample video is obtained based on the probability that the sample video belongs to each label.

[0088] In one embodiment, the computer device takes the average of the probabilities corresponding to the same labeled labels from the segment label prediction results corresponding to each video segment, and uses the average probability corresponding to each labeled label as the probability that the sample video belongs to each labeled label.

[0089] Continuing with the above example, the computer device takes the average of the probabilities corresponding to the same labeled labels in the segment label prediction result 1 "[0.1 0.2 0.1 0.50.1]", the segment label prediction result 2 "[0.3 0.2 0.1 0.3 0.1]" and the segment label prediction result 3 "[0.2 0.20.4 0.10.1]", and obtains the video label prediction result corresponding to the sample video as "[0.2 0.2 0.2 0.3 0.1]".

[0090] Step S210 , updating the model parameters of the video label prediction model according to the prediction loss constructed based on at least one labeled label of the sample video and the probability that the sample video belongs to each labeled label, to obtain a trained video label prediction model.

[0091] In one embodiment, after the computer device updates the model parameters of the video label prediction model, it determines whether the training stop condition is met. If so, a trained video label prediction model is obtained. If not, the computer device returns to step S202 to obtain a sample video and at least two annotated labels of the sample video to continue training.

[0092] In one embodiment, the training stop condition may be that the number of training times reaches a preset number, or the loss value calculated by the predicted loss is less than a preset value, etc.

[0093] In one embodiment, a computer device converts at least one annotated label of a sample video into a vector representation (hereinafter referred to as a label vector), where the dimension of the label vector is consistent with the number of annotated labels in a video label set, and the vector value of each dimension of the label vector represents the probability that the sample video belongs to the annotated label corresponding to the dimension.

[0094] Continuing with the above example, assuming that the video label set includes 5 label labels, and the sample video corresponds to 2 of them, then the label vector of the sample video can be expressed as "[0 1 0 1 0]", where "1" indicates that the sample video belongs to the label label corresponding to the dimension, and "0" indicates that the sample video does not belong to the label label corresponding to the dimension.

[0095] In one embodiment, the computer device constructs a prediction loss based on the difference between the video label prediction result corresponding to the sample video and the label vector, and updates the model parameters of the video label prediction model according to the prediction loss.

[0096] Continuing with the above example, the computer device constructs a prediction loss based on the difference between the video label prediction result “[0.2 0.2 0.2 0.30.1]” and the label vector “[0 1 0 1 0]”.

[0097] It can be understood that a universal loss function satisfies the requirements of the embodiments of the present application for prediction loss, so the computer device can use a universal loss function to construct the prediction loss. Universal loss functions include cross entropy loss function, cosine similarity loss function, etc. In one embodiment, the computer device obtains the gradient corresponding to the current round of training based on the gradient descent algorithm in the direction of minimizing the prediction loss, and updates the model parameters of the video label prediction model according to the gradient. The gradient descent algorithm can be a stochastic gradient descent algorithm, or an algorithm optimized based on the stochastic gradient descent algorithm, such as a stochastic gradient descent algorithm with a momentum term. In one embodiment, refer to Figure 4 , Figure 4 The figure is a flow chart of obtaining the video label prediction result corresponding to the sample video in one embodiment. It can be seen that the computer device divides the sample video into multiple video clips of equal length, inputs each video clip into the video label prediction model, performs label prediction on each video clip through the video label prediction model, obtains the segment label prediction result corresponding to each video clip, fuses the probabilities corresponding to the same annotated label in the segment label prediction results corresponding to each video clip, obtains the video label prediction result corresponding to the sample video, constructs the prediction loss based on the difference between the video label prediction result corresponding to the sample video and the label vector corresponding to the annotated label, and trains the video label prediction model according to the prediction loss.

[0098] In the processing method of the above-mentioned video label prediction model, the sample video is first divided into multiple video segments of equal length, and then the label prediction is performed on each video segment through the video label prediction model to obtain the segment label prediction results corresponding to each video segment. Since the sample video is divided and then the features are extracted based on the video segments, the video length based on which the label prediction is based is shortened, and the label prediction can be performed while reducing the loss of video content information, thereby improving the accuracy of the label prediction for each video segment during the training process. Moreover, different sample videos have different lengths, and the lengths of video segments of different sample videos are also different, so the video label prediction model can adapt to the changes in different video lengths. Then In the segment label prediction results corresponding to each video segment, the probabilities of the same labeled labels are fused to obtain the video label prediction results corresponding to the sample video, so that the video label prediction results can be close to the labeled labels corresponding to the sample video as a whole; then the video label prediction model is trained based on the prediction loss constructed based on at least one labeled label of the sample video and the probability that the sample video belongs to each labeled label, which can improve the training performance of the model, thereby improving the accuracy of the video label prediction model after training; and, it is only necessary to label the video labels corresponding to the sample video as a whole, and there is no need to label the video labels of each video segment divided by the sample video, which saves the labeling cost.

[0099] In one embodiment, dividing a sample video into a plurality of video segments of equal length includes: determining a video length of the sample video; determining a segment length corresponding to the video length of the sample video based on a positive correlation between the video length and the segment length; and dividing the sample video into a plurality of video segments of equal length according to the determined segment length.

[0100] In one embodiment, the computer device determines the segment length corresponding to the video length of the sample video according to the video length of the sample video, and divides the sample video into a plurality of video segments of equal length according to the determined segment length.

[0101] In one embodiment, a positive correlation between the video length and the segment length is preset, which can be expressed by the following formula:

[0102] M=kt

[0103] Wherein, M represents the video length; t represents the segment length; k represents the positive correlation coefficient between the video length and the segment length, and k is a positive value.

[0104] In one embodiment, the number of video label prediction models is more than one, and the positive correlation coefficient may be the number of video label prediction models. For example, if there are three video label prediction models, then for a sample video with a video length of 300 frames, the segment length is 100 frames, and for a sample video with a video length of 3000 frames, the segment length is 1000 frames. In this way, regardless of the length of the sample video, the video label prediction model can efficiently process it, thereby ensuring the processing efficiency of the video label prediction model for videos of different lengths.

[0105] In this embodiment, the segment length is adaptively generated according to the video length, so that both long videos and short videos can be used as training data for the video label prediction model, thereby expanding the scope of training data; and, the video label prediction model is trained using video segments of different lengths, which not only enables the trained video label prediction model to have compatible processing capabilities for video segments of different lengths, but also enables the trained video label prediction model to achieve a balance between efficiency and performance for video segments of different lengths.

[0106] In one embodiment, dividing the sample video into a plurality of video segments of equal length includes: obtaining a segment length set; randomly selecting a segment length from the segment length set; and dividing the sample video into a plurality of video segments of equal length according to the selected segment length.

[0107] The segment length set may be a set of multiple segment lengths, for example, {t 1 ,t 2 ,…,t N}, where N represents the number of fragment lengths in the fragment length set.

[0108] In one embodiment, the computer device randomly selects a segment length from a segment length set, and divides the sample video into a plurality of video segments of equal length according to the selected segment length.

[0109] In one embodiment, the computer device selects a segment length from a segment length set according to the video length of the sample video. For example, the computer device calculates the segment length according to the positive correlation between the video length and the segment length, and selects a segment length closest to the calculated segment length from the segment length set.

[0110] In this embodiment, a segment length is randomly selected from a segment length set, and a video label prediction model is trained using video segments of different lengths, so that the trained video label prediction model can process video segments of different lengths.

[0111] In one embodiment, the probabilities of the same labeled labels in the segment label prediction results corresponding to each video segment are fused to obtain the video label prediction results corresponding to the sample video, including: determining the maximum value of the probabilities of the same labeled labels from the segment label prediction results corresponding to each video segment; taking the maximum value of the probabilities corresponding to each labeled label as the probability that the sample video belongs to each labeled label; and obtaining the video label prediction results corresponding to the sample video based on the probability that the sample video belongs to each labeled label.

[0112] In one embodiment, a computer device takes the maximum value of the probability of the same labeled label from the segment label prediction results corresponding to each video segment, and uses the maximum probability corresponding to each labeled label as the probability that the sample video belongs to each labeled label. Based on the probability that the sample video belongs to each labeled label, a video label prediction result corresponding to the sample video is obtained.

[0113] In one embodiment, the video label prediction result corresponding to the sample video can be expressed by the following formula:

[0114]

[0115] in, Represents the video label prediction result p corresponding to the sample video video The value of the kth dimension, p video There are G dimensions, k∈{1,2,…,G}; the sample video is divided into N video clips, Indicates the value of the segment label prediction result corresponding to the Nth video segment in the sample video in the kth dimension.

[0116] Continuing with the above example, the computer device takes the maximum value of the probability of the same labeled label for the segment label prediction result 1 "[0.1 0.2 0.1 0.50.1]", the segment label prediction result 2 "[0.3 0.2 0.1 0.3 0.1]" and the segment label prediction result 3 "[0.2 0.20.4 0.10.1]", and obtains the video label prediction result corresponding to the sample video as "[0.3 0.2 0.4 0.5 0.1]". The computer device constructs a prediction loss based on the difference between the video label prediction result "[0.30.2 0.4 0.5 0.1]" and the label vector "[0 1 0 1 0]".

[0117] In this embodiment, the probability of the same annotation label in the segment label prediction results corresponding to each video segment is maximized to obtain the video label prediction results corresponding to the sample video, so that the predicted video label prediction results can be close to the annotation label corresponding to the sample video as a whole, that is, in line with the annotation ideas of the annotator, thereby improving the training performance of the model.

[0118] In one embodiment, a label prediction is performed on each video clip through a video label prediction model to obtain a fragment label prediction result corresponding to each video clip, including: for each video clip, feature extraction is performed on the video frame of the video clip through a feature extraction network in the video label prediction model to obtain a feature vector corresponding to each video frame; feature fusion network in the video label prediction model is used to fuse the feature vectors corresponding to each video frame to obtain a feature vector corresponding to the video clip; label recognition network in the video label prediction model is used to predict the feature vector corresponding to the video clip to obtain a fragment label prediction result corresponding to the video clip.

[0119] Here, the feature vector refers to the result of expressing other forms of data in mathematical form. For example, if a video frame is represented in the mathematical form of "[0 0 0 1 0 0 0 0 0 0 0...]", then "[0 0 0 1 0 0 0 0 0 00...]" is the feature vector of the video frame. It can be understood that there is no limitation on what kind of vector the video frame is converted into, as long as the video frame can be mathematically represented. For example, the video frame can be converted into a high-dimensional sparse vector or a low-dimensional dense vector.

[0120] In one embodiment, referring to Figure 5 , Figure 5 The flowchart of obtaining the prediction result of the segment label corresponding to the video segment in one embodiment is shown. It can be seen that, taking video segment 1 as an example, the computer device inputs the video segment 1 into the video label prediction model, and extracts the features of the video frames of the video segment 1 through the feature extraction network in the video label prediction model, and obtains the feature vectors corresponding to each video frame, and the feature vector corresponding to each video frame is D-dimensional, and the value of D is related to the network structure adopted by the feature extraction network. For example, the network structure adopted by the feature extraction network is ResNet (Residual Network), and the value of D is 2048-dimensional; through the feature fusion network in the video label prediction model, the feature vectors corresponding to each video frame are fused to obtain the feature vector corresponding to the video segment, and the feature dimension of the feature vector corresponding to the video segment is consistent with that of the feature vector corresponding to each video frame, which is also D-dimensional; through the label recognition network in the video label prediction model, the feature vector corresponding to the video segment is label predicted to obtain the segment label prediction result corresponding to the video segment, and the segment label prediction result is G-dimensional, and the value of G is consistent with the number of annotated labels in the video label set.

[0121] In one embodiment, the feature extraction network can adopt a general network structure with feature extraction function, such as ResNet, VGG Net (Visual Geometry Group Network), Efficient Net (EfficientNetwork) or ViT (Vision Transformer) and other deep neural networks. The feature fusion network can adopt a general network structure with feature fusion function, such as a max-pooling network or a mean-pooling network. The label recognition network can be composed of a fully-connected layer and an activation function, such as sigmoid.

[0122] In one embodiment, a computer device uses a feature fusion network in a video tag prediction model to fuse vector values ​​corresponding to the same feature dimension in feature vectors corresponding to each video frame, obtain vector values ​​corresponding to each feature dimension of the video clip, and generate a feature vector corresponding to the video clip based on the vector values ​​corresponding to each feature dimension of the video clip.

[0123] In one embodiment, the feature vectors corresponding to each video frame are fused through a feature fusion network in a video tag prediction model to obtain a feature vector corresponding to a video clip, including: determining the maximum value among the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each video frame through a feature fusion network in the video tag prediction model, as the vector value corresponding to each feature dimension of the video clip; generating a feature vector corresponding to the video clip according to the vector value corresponding to each feature dimension of the video clip.

[0124] In one embodiment, the feature fusion network may be a max-pooling network. The computer device determines the maximum value of the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each video frame through the feature fusion network, and then generates a feature vector corresponding to the video clip. The feature vector corresponding to the video clip can be expressed by the following formula:

[0125] z frag =MaxPooling{z frame-1 ,z frame-2 ,…z frame-t}

[0126] and,

[0127] Among them, z frag represents the feature vector corresponding to the video clip, which has t video frames, z frame-trepresents the feature vector corresponding to the t-th video frame in the video clip; MaxPooling represents the maximum pooling process; represents the value of the feature vector corresponding to the video clip in the jth dimension. The feature vector corresponding to the video clip has a total of D dimensions, j∈{1,2,…,D}; Represents the value of the feature vector corresponding to the tth video frame in the video clip in the jth dimension.

[0128] In one embodiment, the feature vectors corresponding to each video frame are fused through a feature fusion network in a video tag prediction model to obtain a feature vector corresponding to a video clip, including: determining, through a feature fusion network in a video tag prediction model, in the feature vectors corresponding to each video frame, the average value of the vector values ​​corresponding to the same feature dimension as the vector value corresponding to each feature dimension of the video clip; and generating a feature vector corresponding to the video clip according to the vector value corresponding to each feature dimension of the video clip.

[0129] In one embodiment, the feature fusion network may be a mean-pooling network, and the computer device determines the average value of the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each video frame through the feature fusion network, and then generates a feature vector corresponding to the video clip. The feature vector corresponding to the video clip can be expressed by the following formula:

[0130] z frag = MeanPooling{z frame-1 ,z frame-2 ,…z frame-t}

[0131] and,

[0132] Among them, z frag represents the feature vector corresponding to the video clip, which has t video frames, z frame-t Represents the feature vector corresponding to the tth video frame in the video clip; MeanPooling represents average pooling processing; represents the value of the feature vector corresponding to the video clip in the jth dimension. The feature vector corresponding to the video clip has a total of D dimensions, j∈{1,2,…,D}; Represents the value of the feature vector corresponding to the tth video frame in the video clip in the jth dimension.

[0133] In this embodiment, feature extraction is performed on the video frames of each video clip to obtain feature vectors corresponding to each video frame, and the feature vectors corresponding to each video frame are fused to obtain a feature vector corresponding to the video clip. Label prediction is performed on the feature vector corresponding to the video clip to obtain a segment label prediction result corresponding to the video clip. Since the length of the video used to generate video features is shortened, the loss of video content information of the video features is reduced, which helps to improve the accuracy of label prediction during training, thereby improving the training performance of the model.

[0134] In one embodiment, Figure 6 As shown, a method for processing a video tag prediction model is provided, and the method is applied to a computer device ( Figure 1 The terminal 102 or server 104 in the example is used for explanation, and the following steps are included:

[0135] Step S602: Obtain a sample video and at least two annotated tags of the sample video, where the at least two annotated tags belong to a video tag set.

[0136] In one embodiment, a computer device obtains sample videos, each sample video has at least two annotated labels, and trains a video label prediction model based on the sample videos and the corresponding at least two annotated labels to obtain a video label prediction model for performing multi-label prediction on the video.

[0137] For the sample video V = {V 1 ,V 2 ,…,V M} is used as an example, where M represents that the sample video V has M video frames. The video label set includes G annotated labels.

[0138] Step S604, determining the video length of the sample video, determining the segment length corresponding to the video length of the sample video according to the positive correlation between the video length and the segment length, and dividing the sample video into a plurality of video segments of equal length according to the determined segment length.

[0139] In one embodiment, the computer device determines the segment length t according to the video length, and divides the sample video V into multiple video segments of equal length according to the segment length t, each video segment has t video frames, so the sample video V has M / t=N video segments. If the sample video V cannot be divided by t, the computer device can add a blank frame to the last video segment, so that the last video segment also has t video frames.

[0140] Step S606, for each video clip, feature extraction is performed on the video frames of the video clip through the feature extraction network in the video label prediction model to obtain feature vectors corresponding to each video frame; the feature fusion network in the video label prediction model is used to fuse the feature vectors corresponding to each video frame to obtain a feature vector corresponding to the video clip; the label recognition network in the video label prediction model is used to perform label prediction on the feature vector corresponding to the video clip to obtain a segment label prediction result corresponding to the video clip; the segment label prediction result includes the probability that the video clip belongs to each labeled label in the video label set.

[0141] In one embodiment, the computer device performs multi-label prediction on N video clips respectively through the video label prediction model to obtain the segment label prediction results {p 1 ,p 2 ,…p N}, each fragment label prediction result is a G-dimensional vector.

[0142] Next, we get the segment label prediction result p corresponding to the video segment N. N The steps are introduced:

[0143] First, the computer device inputs the video segment N into the video label prediction model, and extracts features from the video frames of the video segment N through the feature extraction network in the video label prediction model to obtain the feature vectors {z frame-1 ,z frame-2 ,…z frame-t}, the feature vector corresponding to each video frame is D-dimensional, and the value of D is related to the network structure adopted by the feature extraction network.

[0144] Next, the computer device fuses the feature vectors corresponding to each video frame through the feature fusion network in the video tag prediction model to obtain the feature vector z corresponding to the video segment N. frag , the feature vector corresponding to the video segment N is also D-dimensional.

[0145] In one embodiment, the feature fusion network may be a max-pooling network. The computer device determines the maximum value of the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each video frame through the feature fusion network, and then generates a feature vector corresponding to the video segment N. The feature vector corresponding to the video segment N can be expressed by the following formula:

[0146] z frag =MaxPooling{z frame-1, z frame-2, …z frame-t}

[0147] and,

[0148] In one embodiment, the feature fusion network may be a mean-pooling network, and the computer device determines the average value of the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each video frame through the feature fusion network, and then generates a feature vector corresponding to the video segment N. The feature vector corresponding to the video segment N can be expressed by the following formula:

[0149] z frag = MeanPooling{z frame-1 ,z frame-2 ,…z frame-t}

[0150] and,

[0151] Among them, z frag represents the feature vector corresponding to the video segment N, which has t video frames, z frame-t Represents the feature vector corresponding to the t-th video frame in the video segment N; MaxPooling represents the maximum pooling process; MeanPooling represents the average pooling process; represents the value of the feature vector corresponding to the video segment N in the jth dimension. The feature vector corresponding to the video segment N has a total of D dimensions, j∈{1,2,…,D}; Represents the value of the feature vector corresponding to the t-th video frame in the video segment N in the j-th dimension.

[0152] Next, the computer device uses the label recognition network in the video label prediction model to identify the feature vector z corresponding to the video segment N. frag Perform label prediction to obtain the segment label prediction result p corresponding to the video segment N N , p N is the G dimension.

[0153] Step S608, from the segment label prediction results corresponding to each video segment, determine the maximum value of the probabilities corresponding to the same labeled label; use the maximum value of the probabilities corresponding to each labeled label as the probability that the sample video belongs to each labeled label; based on the probability that the sample video belongs to each labeled label, obtain the video label prediction result corresponding to the sample video; the video label prediction result includes the probability that the sample video belongs to each labeled label in the video label set.

[0154] In one embodiment, the video label prediction result corresponding to the sample video can be expressed by the following formula:

[0155]

[0156] in, Represents the video label prediction result p corresponding to the sample video video The value of the kth dimension, p video There are G dimensions, k∈{1,2,…,G}; the sample video is divided into N video clips, Indicates the value of the segment label prediction result corresponding to the Nth video segment in the sample video in the kth dimension.

[0157] Step S610, updating the model parameters of the video label prediction model according to the prediction loss constructed based on at least two annotated labels of the sample video and the probability that the sample video belongs to each annotated label, to obtain a trained video label prediction model.

[0158] In one embodiment, after the computer device updates the model parameters of the video label prediction model, it determines whether the training stop condition is met. If so, a trained video label prediction model is obtained. If not, the computer device returns to step S202 to obtain a sample video and at least two annotated labels of the sample video to continue training.

[0159] In one embodiment, a computer device converts at least two annotated labels of a sample video into a label vector, where the label vector is G-dimensional and consists of 0 / 1, where 0 indicates that the sample video does not belong to the annotated label corresponding to the dimension, and 1 indicates that the sample video belongs to the annotated label corresponding to the dimension.

[0160] Taking the Binary Cross Entropy Loss function as an example, the prediction loss can be constructed by the following formula:

[0161] Loss(x,y)=y×log(x)+(1-y)log(1-x)

[0162] Among them, x can be the video label prediction result corresponding to the sample video; y can be the label vector corresponding to the sample video.

[0163] In this embodiment, the sample video is first divided into a plurality of video segments of equal length, and then a label prediction is performed on each video segment through a video label prediction model to obtain a segment label prediction result corresponding to each video segment. Since the sample video is divided and then features are extracted based on the video segments, the video length based on which the label prediction is based is shortened, and label prediction can be performed while reducing the loss of video content information, thereby improving the accuracy of label prediction for each video segment during the training process. Moreover, different sample videos have different lengths, so the lengths of video segments of different sample videos are also different, so that the video label prediction model can adapt to changes in different video lengths. Then, the probabilities of the same annotated labels in the segment label prediction results corresponding to each video segment are fused to obtain a video label prediction result corresponding to the sample video, so that the video label prediction result can be close to the annotated label corresponding to the sample video as a whole. Then, the video label prediction model is trained based on the prediction loss constructed based on at least one annotated label of the sample video and the probability that the sample video belongs to each annotated label, which can improve the training performance of the model, thereby improving the accuracy of video label prediction by the trained video label prediction model.

[0164] In one embodiment, Figure 7 As shown, a video tag prediction method is provided, and the method is applied to a computer device ( Figure 1 The terminal 102 or server 104 in the example is used for explanation, and the following steps are included:

[0165] Step S702, obtaining the target video.

[0166] Among them, the target video is the video whose video label is to be predicted in the embodiment of the present application.

[0167] Step S704, after dividing the target video into multiple video segments of equal length, perform label prediction on each video segment through the trained video label prediction model to obtain segment label prediction results corresponding to each video segment; the segment label prediction results include the probability that the video segment belongs to each video label.

[0168] In one embodiment, a computer device predicts labels for each video segment using a trained video label prediction model to obtain segment label prediction results corresponding to each video segment. The dimension of the segment label prediction results predicted by the video label prediction model is consistent with the number of annotated labels in the video label set used when training the video label prediction model.

[0169] Regarding the specific implementation of step S704, reference may be made to the specific implementation of step S204 and step S206 in the above embodiment, which will not be repeated here. The specific implementation of the application side should match the specific implementation of the training side. For example, when the video label prediction model is trained, the length of the segment corresponding to the video length of the sample video is determined according to the video length of the sample video. Then, when the video label prediction model predicts the video label, the length of the segment corresponding to the video length of the target video should also be determined according to the video length of the target video.

[0170] Step S706, the probabilities corresponding to the same video labels in the segment label prediction results corresponding to the video segments are merged to obtain the video label prediction results corresponding to the target video; the video label prediction results include the probabilities that the target video belongs to each video label.

[0171] Regarding the specific implementation of step S706, please refer to the specific implementation of step S208 in the above embodiment, which will not be repeated here. The specific implementation of the application side should match the specific implementation of the training side. For example, when the video tag prediction model is trained, the probability of the corresponding same annotation label is maximized. Then, when the video tag prediction model predicts the video tag, the probability of the corresponding same video tag should also be maximized.

[0172] Step S708: adding a video tag to the target video according to the probability that the target video belongs to each video tag.

[0173] In one embodiment, the computer device adds the video tag with the highest probability to the target video according to the probability that the target video belongs to each video tag, or adds the first Q video tags with the highest probability to the target video, and the value of Q can be determined according to actual applications.

[0174] In one embodiment, referring to Figure 8 , Figure 8 The figure is a flow chart of obtaining the video label prediction result corresponding to the target video in one embodiment. It can be seen that the computer device divides the target video into multiple video clips of equal length, inputs each video clip into the trained video label prediction model, performs label prediction on each video clip through the video label prediction model, obtains the segment label prediction result corresponding to each video clip, fuses the probability of the same video label in the segment label prediction result corresponding to each video clip, and obtains the video label prediction result corresponding to the target video.

[0175] In the above-mentioned video label prediction method, after dividing the target video into multiple video segments of equal length, the label prediction is performed on each video segment through the trained video label prediction model to obtain the segment label prediction results corresponding to each video segment. Since the video length on which the label prediction is based is shortened, label prediction can be performed while reducing the loss of video content information, thereby improving the accuracy of label prediction for each video segment; then, the probabilities of the same video labels in the segment label prediction results corresponding to each video segment are fused to obtain the video label prediction results corresponding to the target video, so that the video label prediction results can reflect the video label corresponding to the target video as a whole, thereby improving the accuracy of video label prediction.

[0176] In one embodiment, a label prediction is performed on each video clip through a trained video label prediction model to obtain a fragment label prediction result corresponding to each video clip, including: for each video clip, a feature extraction network in the video label prediction model is used to extract features from the video frames of the video clip to obtain feature vectors corresponding to each video frame; a feature fusion network in the video label prediction model is used to fuse the feature vectors corresponding to each video frame to obtain a feature vector corresponding to the video clip; a label recognition network in the video label prediction model is used to predict labels for the feature vectors corresponding to the video clip to obtain a fragment label prediction result corresponding to the video clip.

[0177] In one embodiment, the feature vectors corresponding to each video frame are fused through a feature fusion network in a video tag prediction model to obtain a feature vector corresponding to a video clip, including: determining the maximum value among the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each video frame through a feature fusion network in the video tag prediction model, as the vector value corresponding to each feature dimension of the video clip; generating a feature vector corresponding to the video clip according to the vector value corresponding to each feature dimension of the video clip.

[0178] In one embodiment, feature vectors corresponding to each video frame are fused through a feature fusion network in a video tag prediction model to obtain feature vectors corresponding to a video clip, including: determining, through a feature fusion network in a video tag prediction model, in the feature vectors corresponding to each video frame, the average value of the vector values ​​corresponding to the same feature dimension as the vector value corresponding to each feature dimension of the video clip; and generating a feature vector corresponding to the video clip according to the vector value corresponding to each feature dimension of the video clip.

[0179] Regarding the specific implementation of this embodiment, the specific implementation of obtaining the segment label prediction result corresponding to the video segment of the sample video in the above embodiment can be referred to, and will not be repeated here. The specific implementation of the application side should match the specific implementation of the training side. For example, when the video label prediction model is trained, the maximum value of the vector values ​​corresponding to the same feature dimension is used as the vector value of each feature dimension of the video segment of the sample video. Then, when the video label prediction model predicts the video label, the maximum value of the vector values ​​corresponding to the same feature dimension should also be used as the vector value of each feature dimension of the video segment of the target video.

[0180] In this embodiment, feature extraction is performed on the video frames of each video clip to obtain feature vectors corresponding to each video frame, and the feature vectors corresponding to each video frame are fused to obtain a feature vector corresponding to the video clip. Label prediction is performed on the feature vector corresponding to the video clip to obtain a segment label prediction result corresponding to the video clip. Since the video length used to generate video features is shortened, the loss of video content information of the video features is reduced, which helps to improve the accuracy of video label prediction.

[0181] In one embodiment, the probabilities of the same video label in the segment label prediction results corresponding to each video segment are fused to obtain the video label prediction results corresponding to the target video, including: determining the maximum value of the probabilities corresponding to the same labeled label from the segment label prediction results corresponding to each video segment; taking the maximum value of the probability corresponding to each labeled label as the probability that the sample video belongs to each labeled label; and obtaining the video label prediction results corresponding to the sample video based on the probability that the sample video belongs to each labeled label.

[0182] Regarding the specific implementation of this embodiment, reference may be made to the specific implementation of obtaining the video label prediction result corresponding to the sample video in the above embodiment, which will not be repeated here.

[0183] In this embodiment, the probability of the same video label in the segment label prediction results corresponding to each video segment is maximized to obtain the video label prediction result corresponding to the target video, so that the video label prediction result can reflect the video label corresponding to the target video as a whole, thereby improving the accuracy of video label prediction.

[0184] In one embodiment, the method further includes: determining an interest tag corresponding to the target account; when the interest tag includes at least one video tag added for the target video, pushing the target video to a terminal corresponding to the target account.

[0185] Among them, the interest tag is the video tag of interest corresponding to the target account.

[0186] In this embodiment, when the interest tags corresponding to the target account include at least one video tag added for the target video, the target video is pushed to the terminal corresponding to the target account to achieve accurate video recommendation.

[0187] In one embodiment, the method also includes: adding video tags to the video clips according to the probability that the video clips belong to each video tag; determining the interest tags corresponding to the target account; and when the interest tags include at least one video tag added to the video clips, pushing the video clips to the terminal corresponding to the target account.

[0188] In one embodiment, the computer device adds the video tag with the highest probability to the video segment according to the probability that the video segment belongs to each video tag, or adds the first Q video tags with the highest probability to the video segment, and the value of Q can be determined according to actual applications.

[0189] In this embodiment, when the interest tags corresponding to the target account include at least one video tag added for the video clip, the video clip is pushed to the terminal corresponding to the target account to recommend the video clip of interest to the user instead of the lengthy complete video.

[0190] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0191] Based on the same inventive concept, the embodiment of the present application also provides a processing device for implementing the video tag prediction model involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiment of the processing device for one or more video tag prediction models provided below can refer to the limitations of the processing method for the video tag prediction model above, and will not be repeated here.

[0192] In one embodiment, Fig. 9 As shown, a processing device for a video tag prediction model is provided, including: an acquisition module 902, a division module 904, a prediction module 906, a fusion module 908 and an update module 910, wherein:

[0193] An acquisition module 902 is used to acquire a sample video and at least one annotated label of the sample video, wherein the at least one annotated label belongs to a video label set;

[0194] A division module 904, used to divide the sample video into a plurality of video segments of equal length;

[0195] Prediction module 906, used to perform label prediction on each video segment respectively through the video label prediction model to obtain segment label prediction results corresponding to each video segment respectively; the segment label prediction results include the probability that the video segment belongs to each annotated label in the video label set;

[0196] The fusion module 908 is used to fuse the probabilities of the same labeled labels in the segment label prediction results corresponding to each video segment to obtain the video label prediction results corresponding to the sample video; the video label prediction results include the probabilities of the sample video belonging to each labeled label in the video label set;

[0197] The updating module 910 is used to update the model parameters of the video label prediction model according to the prediction loss constructed based on at least one labeled label of the sample video and the probability that the sample video belongs to each labeled label, so as to obtain a trained video label prediction model.

[0198] In one embodiment, the division module 904 is further used to: determine the video length of the sample video; determine the segment length corresponding to the video length of the sample video based on the positive correlation between the video length and the segment length; and divide the sample video into multiple video segments of equal length according to the determined segment length.

[0199] In one embodiment, the division module 904 is further used to: obtain a segment length set; randomly select a segment length from the segment length set; and divide the sample video into a plurality of video segments of equal length according to the selected segment length.

[0200] In one embodiment, the division module 904 is further used to: obtain a specified segment length; and divide the sample video into a plurality of video segments of equal length according to the specified segment length.

[0201] In one embodiment, the fusion module 908 is also used to: determine the maximum value of the probabilities corresponding to the same labeled label from the segment label prediction results corresponding to each video segment; use the maximum value of the probability corresponding to each labeled label as the probability that the sample video belongs to each labeled label; and obtain the video label prediction result corresponding to the sample video based on the probability that the sample video belongs to each labeled label.

[0202] In one embodiment, the prediction module 906 is also used to: for each video clip, perform feature extraction on the video frames of the video clip through the feature extraction network in the video label prediction model to obtain feature vectors corresponding to each video frame; fuse the feature vectors corresponding to each video frame through the feature fusion network in the video label prediction model to obtain the feature vector corresponding to the video clip; perform label prediction on the feature vector corresponding to the video clip through the label recognition network in the video label prediction model to obtain the segment label prediction result corresponding to the video clip.

[0203] In one embodiment, the prediction module 906 is also used to: determine the maximum value of the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each video frame through the feature fusion network in the video tag prediction model, as the vector value corresponding to each feature dimension of the video clip; and generate the feature vector corresponding to the video clip according to the vector value corresponding to each feature dimension of the video clip.

[0204] In one embodiment, the prediction module 906 is also used to: determine the average value of the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each video frame through the feature fusion network in the video tag prediction model, as the vector value corresponding to each feature dimension of the video clip; and generate a feature vector corresponding to the video clip according to the vector value corresponding to each feature dimension of the video clip.

[0205] The processing device of the above-mentioned video label prediction model first divides the sample video into multiple video segments of equal length, and then performs label prediction on each video segment through the video label prediction model to obtain the segment label prediction results corresponding to each video segment. Since the sample video is divided and then features are extracted based on the video segments, the video length based on which the label prediction is based is shortened, and label prediction can be performed while reducing the loss of video content information, thereby improving the accuracy of label prediction for each video segment during the training process. Moreover, different sample videos have different lengths, so the lengths of video segments of different sample videos are also different, so that the video label prediction model can adapt to changes in different video lengths; then, the probabilities of the same annotated labels in the segment label prediction results corresponding to each video segment are merged to obtain the video label prediction results corresponding to the sample video, so that the video label prediction results can be close to the annotated labels corresponding to the sample video as a whole; then, the video label prediction model is trained based on the prediction loss constructed based on at least one annotated label of the sample video and the probability that the sample video belongs to each annotated label, which can improve the training performance of the model, thereby improving the accuracy of video label prediction by the trained video label prediction model.

[0206] Based on the same inventive concept, the embodiment of the present application also provides a device for implementing the above-mentioned video tag prediction. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more video tag prediction device embodiments provided below can refer to the limitations of the video tag prediction method above, and will not be repeated here.

[0207] In one embodiment, Fig.10 As shown, a video tag prediction device is provided, including: an acquisition module 1002, a prediction module 1004, a fusion module 1006 and a recognition module 1008, wherein:

[0208] An acquisition module 1002 is used to acquire a target video;

[0209] Prediction module 1004 is used to divide the target video into multiple video segments of equal length, and then predict the labels of each video segment using the trained video label prediction model to obtain the segment label prediction results corresponding to each video segment; the segment label prediction results include the probability that the video segment belongs to each video label;

[0210] The fusion module 1006 is used to fuse the probabilities of the same video labels in the segment label prediction results corresponding to each video segment to obtain the video label prediction result corresponding to the target video; the video label prediction result includes the probability that the target video belongs to each video label;

[0211] The identification module 1008 is used to add a video tag to the target video according to the probability that the target video belongs to each video tag.

[0212] In one embodiment, the prediction module 1004 is also used to: for each video clip, extract features from the video frames of the video clip through the feature extraction network in the video label prediction model to obtain feature vectors corresponding to each video frame; fuse the feature vectors corresponding to each video frame through the feature fusion network in the video label prediction model to obtain a feature vector corresponding to the video clip; and perform label prediction on the feature vector corresponding to the video clip through the label recognition network in the video label prediction model to obtain a segment label prediction result corresponding to the video clip.

[0213] In one embodiment, the prediction module 1004 is also used to: determine the maximum value of the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each video frame respectively through the feature fusion network in the video tag prediction model, as the vector value of each feature dimension corresponding to the video clip; generate the feature vector corresponding to the video clip according to the vector value of each feature dimension corresponding to the video clip.

[0214] In one embodiment, the prediction module 1004 is also used to: determine the average value of the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each video frame through the feature fusion network in the video tag prediction model, as the vector value of each feature dimension corresponding to the video clip; generate the feature vector corresponding to the video clip according to the vector value of each feature dimension corresponding to the video clip.

[0215] In one embodiment, the fusion module 1006 is further used to: determine the maximum value of the probabilities corresponding to the same labeled label from the segment label prediction results corresponding to each video segment; use the maximum value of the probabilities corresponding to each labeled label as the probability that the sample video belongs to each labeled label; and obtain the video label prediction result corresponding to the sample video based on the probability that the sample video belongs to each labeled label.

[0216] In one embodiment, the video tag prediction device also includes a push module, which is used to: determine the interest tags corresponding to the target account; when the interest tags include at least one video tag added to the target video, push the target video to the terminal corresponding to the target account.

[0217] In one embodiment, the video tag prediction device also includes a push module, which is used to: add video tags to the video clips according to the probability that the video clips belong to each video tag; determine the interest tags corresponding to the target account; when the interest tags include at least one video tag added to the video clip, push the video clip to the terminal corresponding to the target account.

[0218] The above-mentioned video label prediction device divides the target video into multiple video segments of equal length, and then predicts the label of each video segment through the trained video label prediction model to obtain the segment label prediction results corresponding to each video segment. Since the video length on which the label prediction is based is shortened, label prediction can be performed while reducing the loss of video content information, thereby improving the accuracy of label prediction for each video segment; then, the probabilities of the same video labels in the segment label prediction results corresponding to each video segment are fused to obtain the video label prediction results corresponding to the target video, so that the video label prediction results can reflect the video label corresponding to the target video as a whole, thereby improving the accuracy of video label prediction.

[0219] The processing device of the video tag prediction model or each module in the video tag prediction device can be implemented in whole or in part by software, hardware, or a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0220] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.11 As shown. The computer device includes a processor, a memory and a network interface connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the processing data of the video tag prediction model and / or the video tag prediction data. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer device also includes an input and output interface, which is a connection circuit for exchanging information between the processor and the external device. They are connected to the processor via a system bus, referred to as an I / O interface. When the computer program is executed by the processor, a processing method for a video tag prediction model and / or a video tag prediction method is implemented.

[0221] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.12 As shown. The computer device includes a processor, a memory, a communication interface, an input device and a display unit connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer device also includes an input and output interface, which is a connection circuit for exchanging information between the processor and the external device. They are connected to the processor via a system bus, referred to as an I / O interface. When the computer program is executed by the processor, a processing method for a video tag prediction model and / or a video tag prediction method is implemented.

[0222] Those skilled in the art will understand that Fig.11 , Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0223] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the processing method of the video tag prediction model and / or the video tag prediction method in any of the above embodiments is implemented.

[0224] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processing method of the video tag prediction model and / or the video tag prediction method in any of the above embodiments is implemented.

[0225] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the processing method of the video tag prediction model and / or the video tag prediction method in any of the above embodiments.

[0226] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0227] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0228] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0229] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for processing a video tag prediction model, characterized in that: The method comprises: Acquire a sample video and at least one annotated label of the sample video, wherein the at least one annotated label belongs to a video label set; Dividing the sample video into a plurality of video segments of equal length; Using a video tag prediction model, a tag prediction is performed on each of the video clips to obtain a tag prediction result corresponding to each of the video clips; the tag prediction result includes a probability that the video clip belongs to each of the annotated tags in the video tag set; From the segment label prediction results corresponding to each of the video segments, determine the maximum value of the probabilities corresponding to the same labeled label, use the maximum value of the probabilities corresponding to each labeled label as the probability that the sample video belongs to each labeled label, and obtain the video label prediction result corresponding to the sample video based on the probability that the sample video belongs to each labeled label; the video label prediction result includes the probability that the sample video belongs to each labeled label in the video label set; The model parameters of the video label prediction model are updated according to the prediction loss constructed based on at least one labeled label of the sample video and the probability that the sample video belongs to each labeled label, so as to obtain a trained video label prediction model, and the trained video label prediction model is used to perform label prediction on the video.

2. The method according to claim 1, characterized in that The step of dividing the sample video into a plurality of video segments of equal length comprises: Determining the video length of the sample video; Determining the segment length corresponding to the video length of the sample video according to the positive correlation between the video length and the segment length; According to the determined segment lengths, the sample video is divided into a plurality of video segments of equal length.

3. The method according to claim 1, characterized in that The step of dividing the sample video into a plurality of video segments of equal length comprises: Get the fragment length collection; Randomly selecting a fragment length from the set of fragment lengths; According to the selected segment lengths, the sample video is divided into a plurality of video segments of equal length.

4. The method according to claim 1, characterized in that The step of dividing the sample video into a plurality of video segments of equal length comprises: Get the specified fragment length; According to the specified segment length, the sample video is divided into a plurality of video segments of equal length.

5. The method according to claim 1, characterized in that The step of performing label prediction on each of the video segments using a video label prediction model to obtain segment label prediction results corresponding to each of the video segments includes: For each of the video clips, extracting features from the video frames of the video clips through a feature extraction network in the video tag prediction model to obtain feature vectors corresponding to each of the video frames; The feature vectors corresponding to the video frames are fused through a feature fusion network in the video tag prediction model to obtain a feature vector corresponding to the video clip; The label recognition network in the video label prediction model is used to perform label prediction on the feature vector corresponding to the video segment to obtain a segment label prediction result corresponding to the video segment.

6. The method according to claim 5, characterized in that The step of fusing the feature vectors corresponding to the video frames through the feature fusion network in the video tag prediction model to obtain the feature vector corresponding to the video clip includes: By using the feature fusion network in the video tag prediction model, in the feature vectors corresponding to each of the video frames, the maximum value of the vector values ​​corresponding to the same feature dimension is determined as the vector value of each feature dimension corresponding to the video clip; A feature vector corresponding to the video clip is generated according to the vector value of each feature dimension corresponding to the video clip.

7. The method according to claim 5, characterized in that The step of fusing the feature vectors corresponding to the video frames through the feature fusion network in the video tag prediction model to obtain the feature vector corresponding to the video clip includes: By using the feature fusion network in the video tag prediction model, in the feature vectors corresponding to each of the video frames, an average value of the vector values ​​corresponding to the same feature dimension is determined as the vector value corresponding to each of the feature dimensions of the video clip; A feature vector corresponding to the video clip is generated according to the vector value of each feature dimension corresponding to the video clip.

8. A video tag prediction method, characterized in that: The method comprises: Get the target video; After dividing the target video into a plurality of video segments of equal length, performing label prediction on each of the video segments using a trained video label prediction model to obtain segment label prediction results corresponding to each of the video segments; the segment label prediction results include the probability that the video segment belongs to each video label; From the segment label prediction results corresponding to each of the video segments, determine the maximum value of the probabilities corresponding to the same labeled label, use the maximum value of the probabilities corresponding to each labeled label as the probability that the target video belongs to each labeled label, and obtain the video label prediction result corresponding to the target video based on the probability that the target video belongs to each labeled label; the video label prediction result includes the probability that the target video belongs to each video label; According to the probability that the target video belongs to each video tag, a video tag is added to the target video.

9. The method according to claim 8, characterized in that The step of performing label prediction on each of the video clips using the trained video label prediction model to obtain the segment label prediction results corresponding to each of the video clips includes: For each of the video clips, extracting features from the video frames of the video clips through a feature extraction network in the video tag prediction model to obtain feature vectors corresponding to each of the video frames; The feature vectors corresponding to the video frames are fused through a feature fusion network in the video tag prediction model to obtain a feature vector corresponding to the video clip; The label recognition network in the video label prediction model is used to perform label prediction on the feature vector corresponding to the video segment to obtain a segment label prediction result corresponding to the video segment.

10. The method according to claim 9, characterized in that The step of fusing the feature vectors corresponding to the video frames through the feature fusion network in the video tag prediction model to obtain the feature vector corresponding to the video clip includes: By using the feature fusion network in the video tag prediction model, in the feature vectors corresponding to the respective video frames, the maximum value among the vector values ​​corresponding to the same feature dimension is determined as the vector value corresponding to each feature dimension of the video clip; A feature vector corresponding to the video clip is generated according to the vector values ​​of each feature dimension corresponding to the video clip.

11. The method according to claim 9, characterized in that The step of fusing the feature vectors corresponding to the video frames through the feature fusion network in the video tag prediction model to obtain the feature vector corresponding to the video clip includes: By using the feature fusion network in the video tag prediction model, in the feature vectors corresponding to each of the video frames, an average value of the vector values ​​corresponding to the same feature dimension is determined as the vector value corresponding to each of the feature dimensions of the video clip; A feature vector corresponding to the video clip is generated according to the vector values ​​of each feature dimension corresponding to the video clip.

12. The method according to claim 8, characterized in that The method further comprises: Determine the interest tags that correspond to the target account; When the interest tag includes at least one video tag added for the target video, the target video is pushed to a terminal corresponding to the target account.

13. The method according to claim 8, characterized in that The method further comprises: Adding video tags to the video clips according to the probabilities that the video clips belong to the respective video tags; Determine the interest tags that correspond to the target account; When the interest tags include at least one video tag added for the video clip, the video clip is pushed to a terminal corresponding to the target account.

14. A processing device for a video tag prediction model, characterized in that: The device comprises: An acquisition module, used to acquire a sample video and at least one annotated label of the sample video, wherein the at least one annotated label belongs to a video label set; A division module, used for dividing the sample video into a plurality of video segments of equal length; A prediction module, used to perform label prediction on each of the video segments respectively through a video label prediction model to obtain segment label prediction results corresponding to each of the video segments; the segment label prediction results include the probability that the video segment belongs to each annotated label in the video label set; A fusion module, used to determine the maximum value of the probabilities corresponding to the same labeled label from the segment label prediction results corresponding to each of the video segments, and use the maximum value of the probabilities corresponding to each labeled label as the probability that the sample video belongs to each labeled label, and obtain the video label prediction result corresponding to the sample video based on the probability that the sample video belongs to each labeled label; the video label prediction result includes the probability that the sample video belongs to each labeled label in the video label set; An updating module is used to update the model parameters of the video label prediction model according to the prediction loss constructed based on at least one labeled label of the sample video and the probability that the sample video belongs to each labeled label, so as to obtain a trained video label prediction model, and the trained video label prediction model is used to perform label prediction on the video.

15. The device according to claim 14, characterized in that The division module is also used to determine the video length of the sample video; determine the segment length corresponding to the video length of the sample video based on the positive correlation between the video length and the segment length; and divide the sample video into multiple video segments of equal length according to the determined segment length.

16. The device according to claim 14, characterized in that The division module is further used to obtain a set of segment lengths; randomly select a segment length from the set of segment lengths; and divide the sample video into a plurality of video segments of equal length according to the selected segment lengths.

17. The device according to claim 14, characterized in that The division module is further used to obtain a specified segment length; and divide the sample video into a plurality of video segments of equal length according to the specified segment length.

18. The device according to claim 14, characterized in that The prediction module is also used to, for each of the video clips, perform feature extraction on the video frames of the video clip through the feature extraction network in the video label prediction model to obtain feature vectors corresponding to each of the video frames; fuse the feature vectors corresponding to each of the video frames through the feature fusion network in the video label prediction model to obtain the feature vector corresponding to the video clip; and perform label prediction on the feature vector corresponding to the video clip through the label recognition network in the video label prediction model to obtain a segment label prediction result corresponding to the video clip.

19. The device according to claim 18, characterized in that The prediction module is also used to determine the maximum value among the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each of the video frames through the feature fusion network in the video tag prediction model, as the vector value of each feature dimension corresponding to the video clip; and generate the feature vector corresponding to the video clip according to the vector value of each feature dimension corresponding to the video clip.

20. The device according to claim 18, characterized in that The prediction module is also used to determine the average value of the vector values ​​corresponding to the same feature dimension in the feature vectors corresponding to each of the video frames through the feature fusion network in the video tag prediction model, as the vector value of each feature dimension corresponding to the video clip; and generate the feature vector corresponding to the video clip according to the vector value of each feature dimension corresponding to the video clip.

21. A video tag prediction device, characterized in that: The device comprises: An acquisition module, used to acquire a target video; A prediction module is used to divide the target video into a plurality of video segments of equal length, and then perform label prediction on each of the video segments using a trained video label prediction model to obtain segment label prediction results corresponding to each of the video segments; the segment label prediction results include probabilities that the video segments belong to each video label; A fusion module, used to determine the maximum value of the probabilities corresponding to the same labeled label from the segment label prediction results corresponding to each of the video segments, and use the maximum value of the probabilities corresponding to each labeled label as the probability that the target video belongs to each labeled label, and obtain the video label prediction result corresponding to the target video according to the probability that the target video belongs to each labeled label; the video label prediction result includes the probability that the target video belongs to each video label; The identification module is used to add a video tag to the target video according to the probability that the target video belongs to each video tag.

22. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 13 are implemented.

23. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

24. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

Images