Video classification processing method and device, computer equipment and storage medium
By extracting multiple candidate attribute features of video samples and determining representative attribute features, automatic video classification is realized, solving the problem of low video screening efficiency and accuracy in the prior art, and improving the efficiency and accuracy of video classification.
Patent Information
- Application Number
- CN202311255978.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-26
- Publication Date
- 2025-03-28
AI Technical Summary
In the prior art, when selecting high-quality videos from multiple videos, the screening efficiency and accuracy are low, and the manual screening efficiency is low and there is subjectivity.
By extracting multiple candidate attribute features of video samples, including text type features, picture type features and audio type features, the classification results of video samples are predicted, and representative attribute features are determined based on the importance of each candidate attribute feature, which is used to automatically classify videos.
It improves the efficiency and accuracy of video screening and classification, avoids the subjectivity of manual classification, and enhances the objectivity and accuracy of video classification.
Smart Images
Figure CN119719416A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of video processing, and in particular to a video classification processing method, device, computer equipment and storage medium. Background Art
[0002] With the development of computer technology and Internet technology, various multimedia applications have emerged, and users can publish multimedia videos in their favorite multimedia applications for sharing. For multimedia videos published in applications, the quality is uneven, with some having high quality and some having low quality.
[0003] Currently, the method of selecting high-quality videos or boutique videos from multiple videos is mainly through rough screening or manual screening through category labels. However, the accuracy of category label screening is relatively low, and many videos that deviate from the topic will be mixed in. Although manual screening can improve a certain accuracy, it is less efficient and will introduce a certain degree of subjectivity. Summary of the invention
[0004] In view of this, the embodiments of the present disclosure provide a method, apparatus, computer device and storage medium for processing video classification to solve the problem of reduced efficiency and accuracy in screening high-quality videos.
[0005] In a first aspect, an embodiment of the present disclosure provides a method for processing video classification, comprising: obtaining a video set, the video set including multiple video samples; for any video sample in the video set, extracting multiple candidate attribute features of the video sample; wherein the multiple candidate attribute features include text type features, picture type features, and multiple type sub-features under audio type features; predicting a classification result of the video sample based on the multiple candidate attribute features, and determining a representative attribute feature from the multiple candidate attribute features according to the importance of each candidate attribute feature in the classification result prediction process; wherein the representative attribute feature is used to determine the classification result of the target video to be processed.
[0006] The video classification processing method provided by the embodiment of the present disclosure extracts candidate attribute features of the video sample, screens out representative attribute features from the candidate attribute features, and then automatically classifies the target video to be processed according to the representative attribute features, thereby improving the video screening efficiency and further improving the video classification efficiency. At the same time, it avoids the subjectivity of manual classification and improves the accuracy of video classification.
[0007] In a second aspect, an embodiment of the present disclosure provides a processing device for video classification, comprising: an acquisition module, used to acquire a video set, the video set including multiple video samples; a feature extraction module, used to extract multiple candidate attribute features of the video sample for any video sample in the video set; wherein the multiple candidate attribute features include text type features, picture type features and multiple type sub-features under audio type features; a video classification module, used to predict the classification result of the video sample based on the multiple candidate attribute features, and determine the representative attribute feature from the multiple candidate attribute features according to the importance of each candidate attribute feature in the classification result prediction process, wherein the representative attribute feature is used to determine the classification result of the target video to be processed.
[0008] In a third aspect, an embodiment of the present disclosure provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the video classification processing method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.
[0009] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the video classification processing method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0011] Figure 1 is a flowchart of a method for processing video classification according to some embodiments of the present disclosure;
[0012] Figure 2 is a flowchart of another video classification processing method according to some embodiments of the present disclosure;
[0013] Figure 3 is a structural block diagram of a video classification processing device according to an embodiment of the present disclosure;
[0014] Figure 4 It is a schematic diagram of the hardware structure of the computer device of the embodiment of the present disclosure. DETAILED DESCRIPTION
[0015] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.
[0016] In the related art, the method of selecting high-quality videos or boutique videos from multiple videos is mainly to perform rough screening or manual screening through category labels. However, the accuracy of category label screening is relatively low, and many videos that deviate from the topic will be mixed in. Although manual screening can improve a certain accuracy, the efficiency of manual screening is low and will introduce a certain degree of subjectivity.
[0017] Based on this, this technical solution extracts attribute features of video samples in multiple dimensions to mine out more important representative attribute features, and automatically classifies the target video through the representative attribute features, thereby achieving objectivity and accuracy of video classification.
[0018] According to an embodiment of the present disclosure, an embodiment of a processing method for video classification is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0019] In this embodiment, a video classification processing method is provided, which can be used in computer devices, such as mobile phones, computers, etc. Figure 1 is a flowchart of a method for processing video classification according to an embodiment of the present disclosure, such as Figure 1 As shown, the process includes the following steps:
[0020] Step S101: obtaining a video set, where the video set includes a plurality of video samples.
[0021] The video set is a set of video samples with multiple types of labels, and each type of label includes at least one or more video samples. The type label of the video sample is used to characterize the type of content contained in the video sample, such as news, finance, animals, variety shows, etc.
[0022] Specifically, the video collection may be a collection of various types of video samples randomly retrieved from a local video database, or may be pulled from an authorized video platform. Of course, it may also be obtained through other means, which are not specifically limited here.
[0023] Step S102: for any video sample in the video set, extract multiple candidate attribute features of the video sample.
[0024] The multiple candidate attribute features include text type features, picture type features, and multiple type sub-features under audio type features.
[0025] The candidate attribute feature is used to characterize the modal feature of the category to which the video sample belongs. Specifically, the candidate attribute feature can be a type sub-feature corresponding to a text type feature, a type sub-feature corresponding to a picture type feature, or a type sub-feature corresponding to an audio type feature.
[0026] Among them, text type features are text features extracted from the text information corresponding to the video sample, including the core theme of the text, the degree of text structuring, the amount of text information, and the richness of vocabulary; picture type features are picture features extracted from the playback video track of the video sample, including picture stability, video resolution, etc.; audio type features are audio features extracted from the audio track of the video sample, including audio quality, speaking object switching frequency, and tone fluctuation degree.
[0027] Step S103: predicting the classification result of the video sample based on the multiple candidate attribute features, and determining a representative attribute feature from the multiple candidate attribute features according to the importance of each candidate attribute feature in the classification result prediction process.
[0028] Among them, the representative attribute features are used to determine the classification result of the target video to be processed.
[0029] The importance level indicates the influence of each candidate attribute feature on the classification result prediction accuracy. That is, if a candidate attribute feature directly affects the classification accuracy of the video sample, it means that the candidate attribute feature is more important.
[0030] Specifically, different candidate attribute features have different corresponding importance levels. Here, we can compare multiple type sub-features under text type features, picture type features, and audio type features to mine multiple representative attribute features with high importance rankings.
[0031] Then, when a target video to be processed needs to be classified, the corresponding representative attribute features can be directly extracted from the target video to be processed, and the category to which the target video to be processed belongs can be determined according to the representative attribute features. According to the classification result of the target video to be processed, it is added to the corresponding video content pool.
[0032] The video classification processing method provided in this embodiment extracts candidate attribute features of video samples, screens out representative attribute features from the candidate attribute features, and then automatically classifies the target video to be processed according to the representative attribute features, thereby improving the video screening efficiency and then improving the video classification efficiency. At the same time, it avoids the subjectivity of manual classification and improves the accuracy of video classification.
[0033] In this embodiment, a video classification processing method is provided, which can be used in computer devices, such as mobile phones, computers, etc. Figure 2 is a flowchart of a method for processing video classification according to an embodiment of the present disclosure, such as Figure 2 As shown, the process includes the following steps:
[0034] Step S201: Obtain a video set, which includes a plurality of video samples. For detailed description, please refer to the relevant description of the above embodiment, which will not be repeated here.
[0035] Step S202: for any video sample in the video set, extract multiple candidate attribute features of the video sample.
[0036] The multiple candidate attribute features include text type features, picture type features, and multiple type sub-features under audio type features.
[0037] Specifically, the above step S202 may include:
[0038] Step S2021, identifying audio data of the video sample, determining text type features represented by the audio data, and extracting multiple text type sub-features from the text type features.
[0039] For any video sample in the video collection, it is composed of a video track and an audio track. When extracting the text type features of a video sample, first, the video track and audio track of the video sample need to be decomposed to identify the audio data corresponding to the audio track. Then, the audio data is converted into video content represented by text to obtain text type features related to the video content. Then, the text type features are analyzed to determine the various text type sub-features contained in the text type features, including: whether the text has a core theme, the degree of text structuring (logical coherence), the degree of formality of the text, the amount of text information, the richness of vocabulary, the quality of the text, etc.
[0040] Step S2022: Detect the picture display state of the video sample, determine the picture type feature of the video sample, and extract multiple picture type sub-features from the picture type feature.
[0041] The screen display state indicates the display state of the playback screen of the video sample. For any video sample in the video set, it has a corresponding video playback screen. When playing the video sample, the playback screen of the video sample is captured, and the screen type features related to the playback screen are determined in combination with the screen display state of the playback screen during the playback process. The screen type features are analyzed to determine the various screen type sub-features contained in the screen type features, including: screen stability, whether the switching is frequent, whether the lens is shaking, and video resolution (horizontal or vertical screen).
[0042] Step S2023, identifying audio data in the video sample, determining audio type features characterizing the quality of the audio data, and extracting multiple audio type sub-features from the audio type features.
[0043] For any video sample in the video set, the video track and audio track of the video sample are decomposed to identify the audio data corresponding to the audio track. By analyzing the audio data quality, the audio type feature related to the audio data quality is determined. The audio type feature is analyzed to determine the multiple audio type sub-features contained in the audio type feature, including speaker switching frequency and audio quality.
[0044] The video classification processing method provided in this embodiment identifies the features of the video samples in various dimensions to determine multiple candidate attribute features of each video sample, thereby enriching the candidate attribute feature extraction of the video samples and facilitating the combination of the candidate attribute features to determine the influence of each candidate attribute feature and its combined features on the classification of the video samples.
[0045] As an optional implementation, before extracting a plurality of candidate attribute features of the video sample, the method may further include:
[0046] Step a1, extracting video attribute features of the video sample, where the video attribute features include one or more of playback duration, language, and accent;
[0047] Step a2: filtering the video samples in the video set according to the video attribute features to obtain filtered video samples.
[0048] The video attribute feature is used to characterize the playback attributes of the current video, namely, playback duration, language, and accent, etc. The playback duration is the total duration from the start to the end of the playback of the video sample; the language is the type of audio language in the video sample, such as Chinese, English, Russian, etc.; the accent indicates whether there is a regional accent in the audio data of the video sample.
[0049] By parsing the video track and audio track corresponding to the video sample, the start and end times of the video track and audio track are detected, and the earlier start time is determined as the target start time, and the later end time is determined as the target end time. The playback duration of the video sample is the duration between the target start time and the target end time. By parsing the audio track corresponding to the video sample, the audio data corresponding to the audio track can be determined. By detecting the audio data, the language information and accent information carried by the audio data can be determined.
[0050] The video samples in the video set are filtered in combination with the video attribute features to eliminate the video samples that do not meet the video attribute features, and obtain multiple filtered video samples. For example, if the playback time is set to N, the playback time of each video sample in the video set can be detected to determine whether it reaches the playback time N, and the video samples that do not meet the playback time are filtered out. For example, if the language is set to Chinese and does not contain an accent, then by detecting the audio data in each video sample, the video samples that do not meet the Chinese language and have an accent are filtered out.
[0051] The video classification processing method provided in this embodiment filters multiple video samples in a video set by video attribute features, thereby achieving a preliminary screening of video samples, thereby eliminating meaningless video samples and ensuring the classification accuracy of subsequent video samples.
[0052] Step S203: predicting the classification result of the video sample based on the multiple candidate attribute features, and determining a representative attribute feature from the multiple candidate attribute features according to the importance of each candidate attribute feature in the classification result prediction process.
[0053] Among them, the representative attribute features are used to determine the classification result of the target video to be processed.
[0054] Specifically, the above step S203 may include:
[0055] Step S2031, obtaining the actual classification result corresponding to the video sample.
[0056] Each video sample in the video set has a corresponding type label, and the type label is used to represent the actual classification result of the video sample.
[0057] Step S2032, comparing the actual classification result with the predicted classification result of the video sample, and determining the classification precision and recall of the video sample based on the comparison result.
[0058] The actual classification result is compared with the classification result of the predicted video sample to determine whether the two are consistent. If the two are consistent, it means that the classification result of the predicted video sample is accurate. Otherwise, it means that the classification result of the predicted video sample is wrong.
[0059] The prediction and classification results of each video sample in the video set are counted to determine the number of video samples with accurate predictions and the number of video samples with incorrect predictions. Combining the total number of video samples contained in the video set and the number of video samples with accurate predictions, the classification accuracy of the video samples can be calculated.
[0060] Step S2033, determining the importance of each candidate attribute feature in the classification result prediction process based on the classification precision and recall.
[0061] The higher the classification precision and recall, the more important the candidate attribute features are. The classification results are predicted by taking the candidate attribute features of the video sample as the main attribute features of the video sample to obtain the corresponding predicted classification results. The classification precision and recall corresponding to each candidate attribute feature can be determined by combining the predicted classification results. The classification precision and recall is used to characterize the importance of the candidate attribute features in the classification prediction process of the video sample.
[0062] Step S2034: sort the importance of each candidate attribute feature, and determine a representative attribute feature from the multiple candidate attribute features based on the ranking result of the importance.
[0063] The candidate attribute features corresponding to the video samples are sorted according to their importance, and multiple representative attribute features with higher importance are mined from multiple candidate attribute features. For example, after sorting the candidate attribute features according to their importance, the representative attribute features with higher importance are determined to be text information volume, vocabulary difference, text topicality, text structure, text formality, and speaker switching frequency.
[0064] The video classification processing method provided in this embodiment determines the importance of each candidate attribute feature for the classification of the video sample in combination with the classification precision and recall rate corresponding to the video sample, and mines representative attribute features from the candidate attribute features based on the importance, thereby ensuring the feature mining effect.
[0065] In some optional embodiments, the representative attribute feature is determined by a pre-trained video processing model. The video processing model is trained by:
[0066] Step b1, obtaining a video sample set, the video sample set including a plurality of video samples to be processed, the video samples to be processed having corresponding video tags, the video tags being used to characterize the classification of the video samples to be processed;
[0067] Step b2, using the video processing model to perform video classification on the candidate attribute features corresponding to any video sample to be processed, so as to output a predicted classification label of the sample to be processed;
[0068] Step b3, comparing the predicted classification label and the video label, and determining the video precision and recall of the video sample to be processed and the classification attribute characteristics corresponding to the video precision and recall based on the comparison result;
[0069] Step b4, calibrating the video processing model through video precision and classification attribute features.
[0070] The video sample set is a set of pre-collected video samples to be processed, and includes a plurality of different types of video samples to be processed. Specifically, the video sample set can be obtained by online collection or by accessing a video database. The method for collecting the video sample set is not specifically limited here.
[0071] The video sample has its own corresponding video label. For any video sample to be processed in the video sample set, the candidate attribute features of the video sample to be processed are extracted by parsing its video track and audio track. Then, the candidate attribute features of the video sample to be processed are input into the video processing model to predict the video classification and output the corresponding predicted classification label.
[0072] Compare the predicted classification label and the video label to generate a comparison result. The video precision and recall of the video sample to be processed and the classification attribute characteristics corresponding to the video precision and recall are determined through the comparison result. The video processing model is iteratively trained in combination with the classification attribute characteristics of the video precision and recall to achieve correction of the video processing model, so that the representative attribute characteristics of the output of the video processing model are more accurate.
[0073] In the above implementation, the video processing model is trained in combination with the classification attribute features of the video precision-recall rate, thereby achieving accurate extraction of representative attribute features for video samples, and facilitating accurate classification of the video samples to be processed in combination with the representative attribute features.
[0074] In some optional embodiments, the video processing model includes a first sub-model and a second sub-model. The first sub-model is used to generate text content of the video sample to be processed, so as to classify the video sample to be processed based on the text content; the second sub-model is used to generate audio content of the video sample to be processed, so as to classify the video sample to be processed based on the audio content.
[0075] The video processing model integrates the first sub-model and the second sub-model, wherein the first sub-model can be a binary classification model. Taking the long text processing models longformer and long-T5 as examples, the real data is first labeled, that is, whether the text content has a core theme, whether it is structured, and whether it is formalized. Combined with the labels, a binary classification model based on longformer is trained, and the long-T5 model is trained to determine whether the text content of the video sample to be processed has a core theme, whether it is structured, and whether it is formalized.
[0076] Among them, the second sub-model can be an audio classification model, which is mainly used to parse the video sample to be processed, generate the audio content of the video sample to be processed, and classify the video sample to be processed based on the audio content.
[0077] In the above implementation, the first sub-model and the second sub-model are respectively used to determine the text classification result and the audio classification result of the video sample to be processed, thereby improving the classification recall effect of the video sample to be processed.
[0078] In some optional embodiments, the above method may further include:
[0079] Step c1, detecting whether the target tag corresponding to the video sample is in the tag vocabulary corresponding to the target category, where the target category is used to characterize the video type;
[0080] Step c2: when the target tag corresponding to the video sample is in the tag vocabulary corresponding to the target category, the classification result of the video sample is determined based on the target tag.
[0081] Each target category has multiple levels of labels, such as level 1 labels, level 2 labels, level 3 labels, and level 4 labels. The label vocabulary is a vocabulary consisting of multiple levels of labels for the target category. For any video sample, the target label contained in the metadata information of the video sample is obtained to determine whether the target label is in the label vocabulary. When the target label corresponding to the video sample is in the label vocabulary corresponding to the target category, it means that the video sample belongs to the target category. At this time, the classification result of the video sample can be determined based on the target label and placed in the corresponding video content pool.
[0082] The video classification processing method provided in this embodiment combines the target label corresponding to the detected video sample and the label vocabulary corresponding to the target category to determine the classification result of the video sample, further ensures the classification accuracy of the video sample, and enriches the video content pool of various types.
[0083] In some optional embodiments, the above method may further include:
[0084] Step d1, obtaining the knowledge label corresponding to each video sample;
[0085] Step d2, screening multiple video samples in the video set based on the knowledge tags to determine the knowledge attributes corresponding to each video sample;
[0086] Step d3, classify the video samples according to the knowledge attributes and determine the classification results of the video samples.
[0087] The knowledge label is used to characterize whether a video sample is knowledgeable. The knowledge label is manually assigned by watching the video sample. The knowledge label can be used to determine whether the video sample has useful content and high-value authors.
[0088] Each video sample in the video collection is classified according to the knowledge label to determine whether the video sample has the knowledge attribute, and the content type of the video sample is characterized by the knowledge attribute. Thus, by classifying each video sample according to the knowledge attribute, the content type of each video sample can be determined, and the classification result for the content type can be obtained and placed in the corresponding video content pool.
[0089] The video classification processing method provided in this embodiment combines knowledge tags to determine the content type of the video sample, thereby achieving classification results for the video sample and further enriching the video content pool of various types.
[0090] In the present embodiment, a processing device for video classification is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0091] This embodiment provides a video classification processing device, such as Figure 3 As shown, including:
[0092] The acquisition module 301 is used to acquire a video set, where the video set includes multiple video samples.
[0093] The feature extraction module 302 is used to extract multiple candidate attribute features of any video sample in the video set, wherein the multiple candidate attribute features include text type features, picture type features, and multiple type sub-features under audio type features.
[0094] The video classification module 303 is used to predict the classification results of the video samples based on multiple candidate attribute features, and determine representative attribute features from the multiple candidate attribute features according to the importance of each candidate attribute feature in the classification result prediction process, wherein the representative attribute feature is used to determine the classification result of the target video to be processed.
[0095] In some optional embodiments, the feature extraction module 302 may include:
[0096] The text feature determination unit is used to identify the audio data of the video sample, determine the text type features represented by the audio data, and extract multiple text type sub-features from the text type features.
[0097] The picture feature determination unit is used to detect the picture display state of the video sample, determine the picture type feature of the video sample, and extract multiple picture type sub-features from the picture type feature.
[0098] The audio feature determination unit is used to identify audio data in a video sample, determine an audio type feature characterizing the quality of the audio data, and extract a plurality of audio type sub-features from the audio type feature.
[0099] In some optional embodiments, the above device may further include:
[0100] The attribute feature extraction module is used to extract video attribute features of the video sample, where the video attribute features include one or more of playback duration, language, and accent.
[0101] The filtering module is used to filter the video samples in the video set according to the video attribute characteristics to obtain filtered video samples.
[0102] In some optional embodiments, the video classification module 303 may include:
[0103] The actual classification acquisition unit is used to obtain the actual classification result corresponding to the video sample.
[0104] The comparison unit is used to compare the actual classification result with the predicted classification result of the video sample, and determine the classification precision and recall rate of the video sample based on the comparison result.
[0105] The importance determination unit is used to determine the importance of each candidate attribute feature in the classification result prediction process based on the classification precision and recall.
[0106] The ranking unit is used to rank the importance of each candidate attribute feature, and determine a representative attribute feature from multiple candidate attribute features based on the ranking result of the importance.
[0107] In some optional embodiments, the representative attribute feature is determined by a pre-trained video processing model. Accordingly, the above device may also include:
[0108] Model training module, used to train video processing models.
[0109] Specifically, the above-mentioned model training module is used to: obtain a video sample set, which includes multiple video samples to be processed, and the video samples to be processed have corresponding video labels, and the video labels are used to characterize the classification of the video samples to be processed; use the video processing model to perform video classification on the candidate attribute features corresponding to any video sample to be processed, so as to output the predicted classification label of the sample to be processed; compare the predicted classification label with the video label, and determine the video precision and recall of the video sample to be processed and the classification attribute features corresponding to the video precision and recall based on the comparison results; calibrate the video processing model through the video precision and recall and the classification attribute features.
[0110] In some optional embodiments, the video processing model includes a first sub-model and a second sub-model. The first sub-model is used to generate text content of the video sample to be processed, so as to classify the video sample to be processed based on the text content; the second sub-model is used to generate audio content of the video sample to be processed, so as to classify the video sample to be processed based on the audio content.
[0111] In some optional embodiments, the above device may further include:
[0112] Category detection module, used to detect whether the target label corresponding to the video sample is in the label vocabulary corresponding to the target category, and the target category is used to characterize the video type;
[0113] The label classification module is used to determine the classification result of the video sample based on the target label when the target label corresponding to the video sample is in the label vocabulary corresponding to the target category.
[0114] In some optional embodiments, the above device may further include:
[0115] A knowledge label acquisition module is used to obtain the knowledge label corresponding to each video sample;
[0116] The screening module is used to screen multiple video samples in the video collection based on the knowledge tags to determine the knowledge attributes corresponding to each video sample.
[0117] The knowledge classification module is used to classify video samples according to knowledge attributes and determine the classification results of the video samples.
[0118] The video classification processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.
[0119] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.
[0120] The video classification processing device provided by the embodiment of the present disclosure extracts candidate attribute features of the video sample, screens out representative attribute features from the candidate attribute features, and then automatically classifies the target video to be processed according to the representative attribute features, thereby improving the video screening efficiency and further improving the video classification efficiency. At the same time, it avoids the subjectivity of manual classification and improves the accuracy of video classification.
[0121] The present disclosure also provides a computer device having the above Figure 3 The video classification processing device is shown.
[0122] See also Figure 4 , Figure 4 is a schematic diagram of a computer device provided by an optional embodiment of the present disclosure, such as Figure 4 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.
[0123] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.
[0124] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.
[0125] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0126] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.
[0127] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.
[0128] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.
[0129] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A video classification processing method, characterized in that: include: Acquire a video set, wherein the video set includes a plurality of video samples; For any video sample in the video set, extract multiple candidate attribute features of the video sample; wherein the multiple candidate attribute features include a text type feature, a picture type feature, and multiple type sub-features under an audio type feature; Predicting a classification result of the video sample based on the multiple candidate attribute features, and determining a representative attribute feature from the multiple candidate attribute features according to the importance of each of the candidate attribute features in the classification result prediction process; The representative attribute feature is used to determine the classification result of the target video to be processed.
2. The method according to claim 1, characterized in that The extracting of multiple candidate attribute features of the video sample includes: Identifying audio data of the video sample, determining the text type feature represented by the audio data, and extracting a plurality of text type sub-features from the text type feature; and / or, Detecting a picture display state of the video sample, determining a picture type feature of the video sample, and extracting a plurality of picture type sub-features from the picture type feature; and / or, The audio data in the video sample is identified, an audio type feature characterizing the quality of the audio data is determined, and a plurality of audio type sub-features are extracted from the audio type feature.
3. The method according to claim 1 or 2, characterized in that: Before extracting a plurality of candidate attribute features of the video sample, the method further includes: Extracting video attribute features of the video sample, wherein the video attribute features include one or more of playback duration, language, and accent; The video samples in the video set are filtered according to the video attribute features to obtain filtered video samples.
4. The method according to claim 1, characterized in that: Determining a representative attribute feature from the plurality of candidate attribute features according to the importance of each of the candidate attribute features in the classification result prediction process includes: Obtaining actual classification results corresponding to the video samples; Comparing the actual classification result with the predicted classification result of the video sample, and determining the classification precision and recall rate of the video sample based on the comparison result; Determining the importance of each of the candidate attribute features in the classification result prediction process based on the classification precision and recall; The importance of each of the candidate attribute features is ranked, and the representative attribute feature is determined from the plurality of candidate attribute features based on the ranking result of the importance.
5. The method according to claim 1 or 4, characterized in that: The representative attribute feature is determined by a pre-trained video processing model; the training method of the video processing model includes: Acquire a video sample set, wherein the video sample set includes a plurality of video samples to be processed, and the video samples to be processed have corresponding video tags, and the video tags are used to characterize the classification of the video samples to be processed; Using the video processing model to perform video classification on the candidate attribute features corresponding to any of the video samples to be processed, so as to output a predicted classification label of the sample to be processed; Comparing the predicted classification label with the video label, and determining the video precision and recall of the video sample to be processed and the classification attribute feature corresponding to the video precision and recall based on the comparison result; The video processing model is calibrated according to the video precision-recall rate and the classification attribute characteristics.
6. The method according to claim 5, characterized in that The video processing model includes a first sub-model and a second sub-model; The first sub-model is used to generate text content of the video sample to be processed, so as to classify the video sample to be processed based on the text content; The second sub-model is used to generate audio content of the video sample to be processed, so as to classify the video sample to be processed based on the audio content.
7. The method according to claim 1, characterized in that Also includes: Detecting whether a target tag corresponding to the video sample is in a tag vocabulary corresponding to a target category, where the target category is used to characterize a video type; When the target tag corresponding to the video sample is in the tag vocabulary corresponding to the target category, the classification result of the video sample is determined based on the target tag.
8. The method according to claim 1, characterized in that Also includes: Obtaining knowledge labels corresponding to each of the video samples; Screening a plurality of video samples in the video set based on the knowledge tags to determine knowledge attributes corresponding to each of the video samples; The video samples are classified according to the knowledge attributes to determine the classification results of the video samples.
9. A video classification processing device, characterized in that: include: An acquisition module, used to acquire a video set, wherein the video set includes a plurality of video samples; A feature extraction module, configured to extract a plurality of candidate attribute features of any video sample in the video set; wherein the plurality of candidate attribute features include a text type feature, a picture type feature, and a plurality of type sub-features under an audio type feature; A video classification module is used to predict the classification result of the video sample based on the multiple candidate attribute features, and determine a representative attribute feature from the multiple candidate attribute features according to the importance of each candidate attribute feature in the classification result prediction process, wherein the representative attribute feature is used to determine the classification result of the target video to be processed.
10. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the video classification processing method according to any one of claims 1 to 8 by executing the computer instructions.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the video classification processing method according to any one of claims 1 to 8.