A big data intelligent classification method for video content

Abstract

The present invention relates to the field of video processing technology, and in particular to a big data intelligent classification method for video content, which first obtains a target video and a description label of the target video, including an object label for describing an object appearing in the target video, and then judges the accuracy of the object label according to the content of the target video. If the object label is accurate, the target video is classified according to the description label through a first preset classification model, and if the object label is inaccurate, the target video is classified through a second preset classification model. Compared with the prior art, the present invention realizes the determination of the accuracy of the description label by comparing the object label in the description label with the video content, and adopts different classification models for classification, which improves the classification efficiency and accuracy, and avoids the adverse effects caused by inaccurate description labels.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a big data intelligent classification method for video content. Background Art

[0002] With the advancement of the Internet, watching videos has become the main form of entertainment for people. However, due to the massive amount of video data on the Internet and the low efficiency of manual review, in order to provide better content services and maintain a healthy network, intelligent video classification technology has emerged. The research on intelligent classification of massive videos by using big data has become the current main direction.

[0003] In existing methods, in order to improve the accuracy of video classification, a common way is to rely on the description tags defined by the video uploader when uploading for auxiliary classification. However, not all description tags provided by the uploader are accurate and reliable. For example, some uploaders may deliberately add wrong description tags to bad videos, or randomly choose tags that are irrelevant to the video content. If classification is too dependent on the uploader's description tags, it will seriously affect the accuracy of subsequent classification.

[0004] Therefore, a new classification method is needed to avoid the negative impact of inaccurate description labels on classification results. Summary of the invention

[0005] In view of this, it is necessary to provide a big data intelligent classification method for video content to solve the problem in the prior art that inaccurate description labels will have a negative impact on the classification results.

[0006] The present invention provides a big data intelligent classification method for video content, comprising:

[0007] Acquire a target video and a description tag of the target video, wherein the description tag includes an object tag for describing an object appearing in the target video;

[0008] Determining the accuracy of the object label according to the content of the target video;

[0009] If the object label is accurate, classify the target video using a first preset classification model according to the description label to obtain a classification result;

[0010] If the object label is inaccurate, the target video is classified using a second preset classification model to obtain a classification result.

[0011] Furthermore, judging the accuracy of the object label according to the content of the target video includes:

[0012] Extracting a plurality of first video frames from the target video;

[0013] Identify the contents of the plurality of first video frames, and count the frequencies at which objects corresponding to the object labels appear in the plurality of first video frames;

[0014] The accuracy of the object label is obtained according to the relationship between the frequency of the object corresponding to the object label in the multiple first video frames and the preset frequency threshold.

[0015] Furthermore, the description tag also includes a category tag for describing the video category of the target video, and each category tag corresponds to a preset image extraction model; and extracting a plurality of first video frames from the target video includes:

[0016] The target video is processed based on a preset image extraction model corresponding to the category label to obtain a plurality of the first video frames.

[0017] Furthermore, the target video is classified by a first preset classification model according to the description tag to obtain a classification result, including:

[0018] The first video frame in which the object corresponding to the object label appears is used as a key frame;

[0019] Selecting multiple adjacent video frames of a key frame from the target video, and using the key frame and the corresponding multiple adjacent video frames as second video frames;

[0020] The classification result is obtained based on the first preset classification model according to the multiple second video frames in combination with the description tags.

[0021] Further, selecting a plurality of adjacent video frames of a key frame from the target video includes:

[0022] Get the number of relevant areas of the key frame, where the relevant area of ​​the key frame is the area where the object corresponding to the object label in the key frame appears;

[0023] According to the number of relevant areas of the key frame, the number of selected adjacent video frames corresponding to the key frame is obtained;

[0024] Based on the selected number of adjacent video frames corresponding to the key frame, multiple adjacent video frames of the key frame are selected from the target video.

[0025] Furthermore, obtaining the number of selected adjacent video frames corresponding to the key frame according to the number of related areas of the key frame includes:

[0026] The number of adjacent video frames selected is obtained by the following formula: in, Select the number of adjacent video frames, is the floor function, and are different adjustment parameters, is the number of relevant regions of the keyframe, is the total number of frames of the target video.

[0027] Furthermore, obtaining the classification result based on the first preset classification model in combination with the description tags according to the plurality of second video frames includes:

[0028] Obtaining a relevant region of each second video frame, wherein the relevant region of an adjacent video frame in the second video frame is a relevant region of a key frame corresponding to the second video frame;

[0029] According to the similarity between the content in the relevant area in each second video frame and the object corresponding to the object label, the attention weight of each relevant area in each video frame is obtained;

[0030] Based on the attention weight, obtaining a feature fusion vector of the target video according to a plurality of second video frames;

[0031] The feature fusion vector is input into the first preset classification model to obtain the classification result.

[0032] Furthermore, obtaining the feature fusion vector of the target video based on the attention weight according to the plurality of second video frames includes:

[0033] According to each second video frame, obtaining a feature vector of each second video frame;

[0034] Based on the attention weight of the relevant area in each second video frame, weighted optimization is performed on the feature vector of each second video frame to obtain a weighted feature vector of each second video frame;

[0035] The weighted feature vectors of each second video frame are fused to obtain a feature fusion vector of the target video.

[0036] The beneficial effects of the present invention are:

[0037] The present invention provides a big data intelligent classification method for video content, which first obtains a target video and a description tag of the target video, wherein the description tag includes an object tag for describing an object appearing in the target video, and then judges the accuracy of the object tag according to the content of the target video. If the object tag is accurate, the target video is classified according to the description tag through a first preset classification model to obtain a classification result. If the object tag is inaccurate, the target video is classified through a second preset classification model to obtain a classification result. Compared with the prior art, the present invention realizes the determination of the accuracy of the description tag by comparing the object tag in the description tag with the video content, and adopts different classification models to improve the efficiency of video classification according to different determination results. If the description tag is accurate, the video is classified through the first preset classification model in combination with the description tag to improve the classification efficiency and accuracy. If the description tag is inaccurate, the classification is performed only through the video itself, avoiding the adverse effects caused by inaccurate description tags. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A method flow chart of an embodiment of a big data intelligent classification method for video content provided by the present invention;

[0039] Figure 2 for Figure 1 A method flow chart of an embodiment of step S102;

[0040] Figure 3 for Figure 1 Flow chart of a method of an embodiment of step S103. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Combination Figure 1 As shown, a specific embodiment of the present invention discloses a big data intelligent classification method for video content, including:

[0043] S101, obtaining a target video and a description tag of the target video, where the description tag includes an object tag for describing an object appearing in the target video;

[0044] S102, judging the accuracy of the object label according to the content of the target video;

[0045] S103, if the object label is accurate, classify the target video using a first preset classification model according to the description label to obtain a classification result;

[0046] S104: If the object label is inaccurate, classify the target video using a second preset classification model to obtain a classification result.

[0047] Compared with the prior art, the present invention compares the object label in the description label with the video content to determine the accuracy of the description label. Different classification models are used to improve the efficiency of video classification according to the different judgment results. If the description label is accurate, the video is classified by the first preset classification model in combination with the description label to improve the classification efficiency and accuracy. If the description label is inaccurate, the video is classified only by the video itself to avoid the adverse effects caused by inaccurate description labels.

[0048] In a preferred embodiment, in the above step S101, the description tag is a tag related to the video input by the user when the user uploads the target video.

[0049] Further, combined with Figure 2 As shown, in a preferred embodiment, the above step S102, judging the accuracy of the object label according to the content of the target video, specifically includes:

[0050] S201, extracting a plurality of first video frames from the target video;

[0051] S202, identifying contents of a plurality of first video frames, and counting the frequencies at which objects corresponding to the object labels appear in the plurality of first video frames;

[0052] S203 , obtaining the accuracy of the object label according to the relationship between the frequency of the object corresponding to the object label in the plurality of first video frames and a preset frequency threshold.

[0053] For the target video, it is obviously impractical to identify the content of each frame and compare the object label. Therefore, in this embodiment, several first video frames are extracted and the first video frames are used to represent the entire target video to determine the accuracy of the description label.

[0054] In step S201, the first video frame may be selected by any existing method, which is conducive to extracting multiple first video frames at equal intervals according to a specific ratio. Different methods may also be used for extraction based on certain rules and for different situations.

[0055] Specifically, in a preferred embodiment, the description tag further includes a category tag for describing the video category of the target video, and each category tag corresponds to a preset image extraction model; the above step S201, extracting a plurality of first video frames from the target video, includes:

[0056] The target video is processed based on a preset image extraction model corresponding to the category label to obtain a plurality of the first video frames.

[0057] The category label in the above process is a method for describing the video category. The system can provide several default options for the category label, which is selected by the video uploader when uploading the video. It should be noted that the category indicated by the category label is the video category of the preliminary classification, such as feature films, advertisements, life videos, etc. This category is the objective category of the video content itself and has nothing to do with the specific needs of the classification system. The category indicated by the category label is different from the category in the classification result finally obtained by the first preset classification model or the second preset classification model. The category in the final classification result can be flexibly set to any category that is unrelated to the content of the video itself according to specific needs. This category is usually more subjective and serves the specific needs of the system, such as the age group suitable for the target video, the category of the target video that a certain type of user is interested in, whether the target video contains characters and keywords blocked by a certain user, etc. Through the category of the classification result, the system can provide users with a better push experience.

[0058] The preset image extraction model in the above process is a different method for extracting video frames under different circumstances. It is a program or module designed manually in a big data classification system. Its specific methods may include a color histogram method, a motion detection method, a video quality assessment method, an optical flow analysis method, a facial recognition algorithm, etc. Through different types of labels, different preset image extraction models can be selected, so that the extracted first video frame can be more in line with the actual situation and improve the accuracy of object label judgment.

[0059] For example, for different types of videos, different preset image extraction models can be used to identify and extract the first video frame. The following are some example application scenarios:

[0060] Feature films and TV series: You can use color histograms and motion detection algorithms to identify frames of key scenes, important characters, or plot twists.

[0061] Advertisements and promotional videos: Image quality assessment algorithms can be used to extract the clearest and most attractive frames to showcase the features of a product or service.

[0062] News reporting and documentaries: Optical flow analysis and object tracking algorithms can be used to capture frames where important events, people, or objects appear.

[0063] Video logs and vlogs: Facial recognition algorithms can be used to extract frames with the most interesting emotional expressions or moments.

[0064] Furthermore, in the above step S202, how to identify the content of the first video frame and how to count the frequency of occurrence of the object corresponding to the object label in multiple first video frames are both existing technologies that can be understood by technical personnel in this field, so they are not explained in detail in this article.

[0065] In the above step S203, the preset frequency threshold can be set manually or calculated based on models such as machine learning. When the frequency of the object corresponding to the object label in multiple first video frames is greater than the preset frequency threshold, it indicates that the object label is accurate, then to a certain extent it can be regarded as accurate in description, and vice versa.

[0066] If it is determined that the description label is accurate, step S103 can be performed. If the object label is accurate, the target video is classified according to the description label through a first preset classification model to obtain a classification result.

[0067] The first preset classification model may be any existing classification model that is combined with description tags to classify the target video. When the description tags are accurate, combining the description tags for classification can make the classification more efficient and accurate.

[0068] The following are several methods that can be combined with description tags for classification (i.e., examples of several first preset classification models):

[0069] Text-video matching: Match or calculate the similarity of the video description tags. Natural language processing technology can be used to convert the semantic description into a vector representation, and then the video is associated with the corresponding tag by calculating the similarity between the vectors or matching them.

[0070] Reinforcement Learning: Use reinforcement learning methods to obtain strategies for video classification from description labels. Build a reinforcement learning agent to classify videos based on description labels at each time step, and continuously optimize the classification strategy through interaction with the environment, so that the agent can classify videos more accurately.

[0071] Attention-based classification: This method uses an attention mechanism to automatically focus on regions or frames in the video that are related to the description label. The attention weight can be calculated based on object detection results, keyframe extraction, or region proposals, and then the features of the focused region or frame are used for the classification task.

[0072] Classification combined with semantic segmentation: This method uses a semantic segmentation model to extract segmentation results of different objects in the video, and then classifies them according to the information describing the labels. Object labels can be mapped to the categories of semantic segmentation results, and the occurrence of different objects in the video can be counted to provide auxiliary information for video classification.

[0073] Combination Figure 3 As shown, in a preferred embodiment, in the above step S103, if the object label is accurate, the target video is classified according to the description label by using a first preset classification model to obtain a classification result, which specifically includes:

[0074] S301, taking the first video frame in which an object corresponding to an object tag appears as a key frame;

[0075] S302, selecting a plurality of adjacent video frames of a key frame from the target video, and using the key frame and the corresponding plurality of adjacent video frames as second video frames;

[0076] S303: Obtain the classification result based on the first preset classification model according to the plurality of second video frames in combination with the description tags.

[0077] A key frame is a frame that clearly contains an object corresponding to an object label. Using it as a priori condition can improve the speed of video classification. In this embodiment, multiple adjacent video frames are reselected based on the key frame, that is, frames within the adjacent range of the key frame. These frames can also contain sufficient object information. By inputting these video frames and the key frame as the second video frame into the first preset classification model, the classification accuracy and speed can be greatly improved.

[0078] Specifically, in a preferred embodiment, in the above step S302, selecting a plurality of adjacent video frames of a key frame from the target video specifically includes:

[0079] Get the number of relevant areas of the key frame, where the relevant area of ​​the key frame is the area where the object corresponding to the object label in the key frame appears;

[0080] According to the number of relevant areas of the key frame, the number of selected adjacent video frames corresponding to the key frame is obtained;

[0081] Based on the selected number of adjacent video frames corresponding to the key frame, multiple adjacent video frames of the key frame are selected from the target video.

[0082] The above process controls the number of selected adjacent video frames according to the number of relevant areas in the key frame, so as to control the number of the second video frames finally obtained to be not too large while containing sufficient information, thereby causing meaningless calculation.

[0083] Furthermore, in a preferred embodiment, the number of adjacent video frames selected is obtained by the following formula: in, Select the number of adjacent video frames, is the floor function, and are different adjustment parameters, is the number of relevant regions of the keyframe, is the total number of frames of the target video.

[0084] This formula ensures that the number of selected adjacent video frames does not exceed the total number of video frames. , and the number of frames selected is related to the number of regions By adjusting the coefficient The value of can control the relative number of selected frames. The larger the value, the more frames and related regions are selected. The stronger the correlation is; on the contrary, if The smaller the value, the fewer frames are selected. The total number of video frames affects the number of selected frames. The size of the video will be adjusted, but it will not exceed the total number of frames of the target video.

[0085] Further, in a preferred embodiment, the embodiment adopts an improved classification method based on the attention mechanism to implement the above step S103. Specifically, in this embodiment, step S303, according to the plurality of second video frames, combined with the description tags, based on the first preset classification model to obtain the classification result, specifically includes:

[0086] Obtaining a relevant region of each second video frame, wherein the relevant region of an adjacent video frame in the second video frame is a relevant region of a key frame corresponding to the second video frame;

[0087] According to the similarity between the content in the relevant area in each second video frame and the object corresponding to the object label, the attention weight of each relevant area in each video frame is obtained;

[0088] Based on the attention weight, obtaining a feature fusion vector of the target video according to a plurality of second video frames;

[0089] The feature fusion vector is input into the first preset classification model to obtain the classification result.

[0090] Specifically, in a preferred embodiment, the above steps are based on the attention weights, and obtain the feature fusion vector of the target video according to multiple second video frames, specifically including:

[0091] According to each second video frame, obtaining a feature vector of each second video frame;

[0092] Based on the attention weight of the relevant area in each second video frame, weighted optimization is performed on the feature vector of each second video frame to obtain a weighted feature vector of each second video frame;

[0093] The weighted feature vectors of each second video frame are fused to obtain a feature fusion vector of the target video.

[0094] The present invention also provides a more detailed embodiment to clearly illustrate the above process S303:

[0095] The classification method based on the attention mechanism mainly uses the attention mechanism to automatically focus on the areas or frames related to the object labels in the video to improve the accuracy of video classification. The main steps of this method are:

[0096] Feature extraction: First, extract features from the target video. A common method is to use a convolutional neural network (CNN) to process the video frames and obtain the visual features of each frame. This can be achieved by extracting frame-level features on a pre-trained CNN network (such as ResNet, Inception, etc.). In this embodiment, a feature vector is extracted for each second video frame, and these vectors are used as video features of the target video.

[0097] Attention calculation: Attention weights are calculated using relevant regions. This can be achieved in different ways, such as using a spatial-based attention model to calculate the attention weights for each region, or using an attention-based recurrent neural network (AttRNN) to model the frames of the time series and derive the attention weights for each frame.

[0098] For example, the spatial-based attention weight process includes:

[0099] Similarity calculation: For each second video frame, a feature similarity calculation method (such as cosine similarity, Euclidean distance, etc.) can be used to measure the similarity or correlation score between each relevant region and the object corresponding to the object label.

[0100] Normalization: When converting similarity into attention weights, it is usually necessary to normalize the weights so that they are in the range of [0, 1]. A common method is to apply the softmax function to obtain the attention weights, ensuring that the sum of the weights is 1.

[0101] Another example is the attention-based recurrent neural network (AttRNN):

[0102] Attention network: In AttRNN, a common approach is to use an attention network to calculate frame-level attention weights. The network can use a fully connected layer, a convolutional layer, etc., taking the features of each frame as input and outputting the corresponding attention weights.

[0103] Activation function: After obtaining the attention weights, an activation function, such as the softmax function, is usually used to convert the attention weights into probability values ​​to ensure that the sum of the attention weights is equal to 1.

[0104] After obtaining the attention weights, feature fusion can be performed: the feature vector is fused with the attention weights of the region. A common method is to multiply the feature vector of each video frame by the attention weight of the corresponding region, and then aggregate them into a feature vector. The features of different frames can be averaged, max pooled, etc.

[0105] Finally, the fused features are input into a classification model (i.e., the first preset classification model) for classification. This classification model can be a traditional machine learning algorithm, such as support vector machine (SVM), random forest, etc., or a model based on deep learning, such as a fully connected neural network or a recurrent neural network (RNN).

[0106] It is understandable that the relevant details in the above process are prior art that can be understood by those skilled in the art and will not be elaborated in detail herein.

[0107] Further, if it is determined that the description label is inaccurate, step S104 may be performed. If the object label is inaccurate, the target video is classified by a second preset classification model to obtain a classification result.

[0108] Unlike the first preset classification model, the second preset classification model does not need to take the object label as the input of the model when classifying the target video. It only classifies according to the target video itself. Although its efficiency may not be as good as the first preset classification model, it is not affected by incorrect object labels.

[0109] The second preset classification model can be any existing classification model, such as a classification model based on content features, such as classification based on color histogram, texture features, shape features, etc. Another example is a classification model based on machine learning, common algorithms include support vector machine (SVM), decision tree, random forest, etc., or classification based on deep learning, common ones include convolutional neural network (CNN), recurrent neural network (RNN), etc.

[0110] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0111] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A big data intelligent classification method for video content, characterized in that: include: Acquire a target video and a description tag of the target video, wherein the description tag includes an object tag for describing an object appearing in the target video; Determining the accuracy of the object label according to the content of the target video; If the object label is accurate, classify the target video using a first preset classification model according to the description label to obtain a classification result; If the object label is inaccurate, classify the target video using a second preset classification model to obtain a classification result; The step of classifying the target video according to the description tag by using a first preset classification model to obtain a classification result includes: The first video frame in which the object corresponding to the object label appears is used as a key frame; Selecting multiple adjacent video frames of a key frame from the target video, and using the key frame and the corresponding multiple adjacent video frames as second video frames; Obtaining the classification result based on the first preset classification model according to the plurality of second video frames and the description tags; Selecting a plurality of adjacent video frames of a key frame from the target video includes: Get the number of relevant areas of the key frame, where the relevant area of ​​the key frame is the area where the object corresponding to the object label in the key frame appears; According to the number of relevant areas of the key frame, the number of selected adjacent video frames corresponding to the key frame is obtained; Selecting a plurality of adjacent video frames of the key frame from the target video based on the selected number of adjacent video frames corresponding to the key frame; The step of obtaining the number of selected adjacent video frames corresponding to the key frame according to the number of related regions of the key frame includes: The number of adjacent video frames selected is obtained by the following formula: Among them, S is the number of adjacent video frames selected, is a floor rounding function, a and b are different adjustment parameters, n is the number of relevant regions of the key frame, and N is the total number of frames of the target video; The obtaining the classification result based on the first preset classification model according to the plurality of second video frames in combination with the description tags includes: Obtaining a relevant region of each second video frame, wherein the relevant region of an adjacent video frame in the second video frame is a relevant region of a key frame corresponding to the second video frame; According to the similarity between the content in the relevant area in each second video frame and the object corresponding to the object label, the attention weight of each relevant area in each video frame is obtained; Based on the attention weight, obtaining a feature fusion vector of the target video according to a plurality of second video frames; The feature fusion vector is input into the first preset classification model to obtain the classification result.

2. The method for intelligent classification of big data for video content according to claim 1 is characterized in that: The step of determining the accuracy of the object label according to the content of the target video includes: Extracting a plurality of first video frames from the target video; Identify the contents of the plurality of first video frames, and count the frequencies at which objects corresponding to the object labels appear in the plurality of first video frames; The accuracy of the object label is obtained according to the relationship between the frequency of the object corresponding to the object label in the multiple first video frames and the preset frequency threshold.

3. The method for intelligent classification of big data for video content according to claim 2 is characterized in that: The description tag also includes a category tag for describing the video category of the target video, and each category tag corresponds to a preset image extraction model; extracting a plurality of first video frames from the target video includes: The target video is processed based on a preset image extraction model corresponding to the category label to obtain a plurality of the first video frames.

4. The method for intelligent classification of big data for video content according to claim 1, characterized in that: The step of obtaining the feature fusion vector of the target video based on the attention weight and the plurality of second video frames includes: According to each second video frame, obtaining a feature vector of each second video frame; Based on the attention weight of the relevant area in each second video frame, weighted optimization is performed on the feature vector of each second video frame to obtain a weighted feature vector of each second video frame; The weighted feature vectors of each second video frame are fused to obtain a feature fusion vector of the target video.

Images