Live broadcast content category identification method and device, equipment and storage medium

Abstract

The invention relates to the technical field of live broadcast, and discloses a live broadcast content category identification method and device, equipment and a storage medium. The category identification method comprises the following steps: acquiring live broadcast content screenshots; classifying and identifying the live broadcast content screenshots based on a plurality of preset classification models to obtain a plurality of category results; mapping the plurality of category results into a preset category to obtain a plurality of to-be-determined categories; and performing weighted voting on the plurality of undetermined categories according to a preset category to determine the category of the live broadcast content screenshot. According to the invention, real-time monitoring and automatic identification of the live broadcast content are realized, the accuracy and robustness of illegal content identification are improved, the comprehensiveness and diversity of category identification are guaranteed, the pressure of manual auditing is reduced, the efficiency of identification and auditing and the capability of recalling the illegal content are improved, a good network environment and live broadcast experience are favorably maintained, and the user experience is improved. And the management efficiency of the live broadcast content and the security and legality of the live broadcast content are improved.

Description

Technical Field

[0001] The present invention relates to the field of live broadcast technology, and in particular to a method, device, equipment and storage medium for identifying the category of live broadcast content. Background Art

[0002] With the development of the Internet and mobile communications, the live broadcast industry has risen rapidly, and the amount of live broadcast content has increased massively. However, the massive growth of live broadcast content has also brought challenges to live broadcast content management, especially the growth of illegal content, which has brought adverse effects to society and also brought more review workload to live broadcast platforms. Live broadcast platforms need to efficiently identify and handle these illegal content to maintain a good network ecology and user experience, improve content management efficiency and the security and legality of live broadcast content.

[0003] Currently, live content review is mainly performed manually on the content review platform, and a single pre-trained model is used to identify illegal content. However, manual review is inefficient and easily affected by subjective factors, and the recognition accuracy and generalization ability of a single model are limited, which usually cannot meet the real-time and accuracy requirements of live content review. In addition, existing models mainly rely on large-scale supervised learning and high-quality training data. If there is noisy data, it is easy to cause unexpected biased predictions, false detections and missed detections, and the recall ability of illegal content is weak. Summary of the invention

[0004] The main purpose of the present invention is to provide a method, device, equipment and storage medium for identifying the category of live content, aiming to solve the technical problem that the existing live content review has a weak ability to recall illegal content.

[0005] A first aspect of the present invention provides a method for identifying a category of live content, the method comprising:

[0006] Get screenshots of live broadcast content;

[0007] Classify and identify the live content screenshots based on multiple preset classification models to obtain multiple category results, wherein the multiple preset classification models include at least one first classification model obtained by training a convolutional neural network model based on sample data sets with different classification focuses, and at least one second classification model obtained by fine-tuning a small amount of sample data or processing a large pre-trained classification model based on zero-sample data;

[0008] Mapping the multiple category result categories to preset categories to obtain multiple pending categories, wherein the preset categories at least include a violation category and a normal category;

[0009] A weighted vote is performed on the multiple pending categories according to the preset categories to determine the category of the live content screenshot.

[0010] Optionally, in a first implementation of the first aspect of the present invention, the first classification model includes:

[0011] The ultra-large-scale category classification model is obtained by training a first sub-convolutional neural network model based on a first sample training data set including violation class sample data, special class sample data, and normal class sample data;

[0012] A high-accuracy category classification model is obtained by training a second sub-convolutional neural network model based on a second sample data set including violation class sample data, suspected violation class sample data, special class sample data, and normal class sample data;

[0013] A high recall category classification model is obtained by training a third sub-convolutional neural network model based on a third sample data set including real-life violation sample data, cartoon violation sample data, special class sample data, and normal class sample data;

[0014] The highly fine-grained category classification model is obtained by training the fourth sub-convolutional neural network model based on a fourth sample data set including a variety of serious violation sample data, a variety of borderline violation sample data, and normal sample data.

[0015] Optionally, in a second implementation of the first aspect of the present invention, the second classification model includes:

[0016] A first small sample category classification model is obtained by fine-tuning the text-side features of the large pre-trained classification model based on a fifth sample data set including violation class sample data, special class sample data, and normal class sample data;

[0017] A second small sample category classification model is obtained by fine-tuning the text-side features of the large pre-trained classification model based on a sixth sample data set including real-life violation sample data, cartoon violation sample data, special class sample data, and normal class sample data;

[0018] The zero-sample category classification model is used to classify and identify the screenshots of the live content based on category labels of violation category, special category and normal category.

[0019] Optionally, in a third implementation of the first aspect of the present invention, the preset categories include a violation category, a special category, and a normal category;

[0020] The mapping of the multiple category result categories to preset categories to obtain multiple pending categories includes:

[0021] Mapping the category results of the suspected violation class, the category results of the real person violation class, the category results of the animation violation class, and the category results of the serious violation class to the preset category of the violation class to obtain the pending category of the violation class;

[0022] The category result of the borderline violation class is mapped to the preset category of the special class to obtain the pending category of the special class.

[0023] Optionally, in a fourth implementation manner of the first aspect of the present invention, performing weighted voting on the multiple to-be-determined categories according to the preset categories to determine the category of the live content screenshot includes:

[0024] The category of the live content screenshot is determined based on the following formula:

[0025] final_result = α·(w1M cnn1 +w2M cnn2 +w3M cnn3 +w4M cnn4 )+β·(w5M fs1 +w6M fs2 +w7M zs );

[0026] Wherein, final_result is the category of the live content screenshot, α is the first comprehensive weight value of the first classification model, β is the second comprehensive weight value of the second classification model, w1 is the first weight value of the ultra-large-scale category classification model, w2 is the second weight value of the high-accuracy category classification model, w3 is the third weight value of the high-recall category classification model, w4 is the fourth weight value of the high-granularity category classification model, w5 is the fifth weight value of the first few-sample category classification model, w6 is the sixth weight value of the second few-sample category classification model, w7 is the seventh weight value of the zero-sample category classification model, M cnn1 is the score value of the first category result identified by the ultra-large-scale category classification model, M cnn2 is the score value of the second category result identified by the high accuracy category classification model, M cnn3 is the score value of the third category result identified by the high recall category classification model, M cnn4 is the score value of the fourth category result identified by the highly fine-grained category classification model, M fs1 is the score value of the fifth category result identified by the first few-sample category classification model, M fs2 is the score value of the sixth category result identified by the second few-sample category classification model, M zs The result score value of the seventh category identified by the zero-sample category classification model.

[0027] Optionally, in a fifth implementation manner of the first aspect of the present invention, the second weight value and the third weight value are both greater than the first weight value and the fourth weight value; the first comprehensive weight value is 0.6, the second comprehensive weight value is 0.4, the first weight value is 0.9, the second weight value is 1.1, the third weight value is 1.1, and the fourth weight value is 0.8.

[0028] Optionally, in a sixth implementation of the first aspect of the present invention, after obtaining the screenshot of the live content, the method includes:

[0029] Obtain a user identifier of the anchor corresponding to the live content screenshot;

[0030] After performing weighted voting on the plurality of pending categories according to the preset categories to determine the category of the live content screenshot, the method further includes:

[0031] Acquire the categories of a plurality of live content screenshots of a target live broadcast at a preset frequency within a preset time period;

[0032] If the multiple categories include the category of violation, recalling the target live broadcast based on the user identifier;

[0033] If the number of the special categories among the multiple categories is greater than a preset recall threshold, the target live broadcast is recalled based on the user identifier, or the target live broadcast is fed back to the content review platform.

[0034] A second aspect of the present invention further provides a device for identifying a category of live content, the device comprising:

[0035] The acquisition module is used to obtain screenshots of live content;

[0036] An identification module, configured to classify and identify the live content screenshots based on a plurality of preset classification models to obtain a plurality of category results, wherein the plurality of preset classification models include at least one first classification model obtained by training a convolutional neural network model based on sample data sets with different classification focuses, and at least one second classification model obtained by fine-tuning a small amount of sample data or processing a large pre-trained classification model based on zero-sample data;

[0037] A mapping module, used for mapping the multiple category result categories to preset categories to obtain multiple pending categories, wherein the preset categories at least include a violation category and a normal category;

[0038] The settlement module is used to perform weighted voting on the multiple pending categories according to the preset categories to determine the category of the live content screenshot.

[0039] The third aspect of the present invention also provides a computer device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the computer device executes the live content category identification method as described above.

[0040] A fourth aspect of the present invention further provides a computer-readable storage medium, on which instructions are stored, and when the instructions are executed by a processor, the method for identifying the category of live content as described above is implemented.

[0041] The embodiments of the present invention provide a method, apparatus, device and storage medium for identifying the category of live content. A screenshot of the live content is first obtained, and then the screenshot of the live content is classified and identified through multiple classification models, and the results of the classification and identification are mapped to preset categories, and weighted voting is performed according to the preset categories to determine the category of the screenshot of the live content. This application improves the accuracy and robustness of illegal content identification through the combination of multiple classification models, reduces the false alarm rate and missed alarm rate, and by using a weighted voting mechanism, it can also assign corresponding weights according to the performance characteristics and focus of different models, so that the final classification results are more reliable. Through the use of few-sample and zero-sample models, it can effectively solve the problem of sample scarcity in the real world, effectively improve the generalization ability of the model, reduce the amount of calculation, and ensure the comprehensiveness and diversity of category identification, so that more live content screenshots can be processed at the same time, realizing real-time monitoring and automatic identification of live content, reducing the pressure of manual review, and improving the efficiency of identification and review and the ability to recall illegal content, which can ensure the green and healthy live broadcast, is conducive to maintaining a good network environment and live broadcast experience, improve the management efficiency of live broadcast content and the security and legality of live broadcast content, and maintain a healthy corporate image and user trust. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a schematic flow chart of a first embodiment of a method for identifying a category of live content in an embodiment of the present invention;

[0043] Figure 2 for Figure 1 A schematic flow chart of an embodiment of step 103 in the embodiment;

[0044] Figure 3 for Figure 1 A schematic diagram of an embodiment flow chart after step 104 in the embodiment;

[0045] Figure 4 This is a functional module diagram of an embodiment of a device for identifying a category of live content according to an embodiment of the present invention;

[0046] Figure 5 The figure is a functional module diagram of an embodiment of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0048] For ease of understanding, the specific process of the method for identifying the category of live content in an embodiment of the present invention is described below. Figure 1 , Figure 1 1 is a flow chart of a first embodiment of a method for identifying a category of live content in an embodiment of the present invention. In this embodiment, the method for identifying a category of live content includes:

[0049] 101. Get screenshots of live broadcast content;

[0050] In this embodiment, the live content screenshot is a key frame image of the live screen. The live content screenshot can reflect the scene, characters, and behaviors in the live content, so as to facilitate subsequent identification and analysis. At the same time, the live content screenshot can also be associated with the live broadcast information. The nickname of the anchor, the anchor user identifier (User Identifier, UID) and the anchor's IP address (Internet Protocol Address) and other information can be directly or indirectly obtained through the live content screenshot. The live content screenshot can be obtained by capturing the key frame image in the video stream according to the interface provided by the live broadcast platform, or by regularly sampling and capturing the live content through the background monitoring system. It can also be obtained through the user reporting mechanism, when the user finds suspected illegal content, the screenshot uploaded is obtained, etc. In addition, after obtaining the live content screenshot, appropriate preprocessing can be performed to improve the recognition of the screenshot and improve the accuracy of subsequent identification. The preprocessing includes but is not limited to image resizing, image cropping, contrast enhancement, binarization and denoising, etc., and this application does not make specific restrictions on this.

[0051] In an optional embodiment, after obtaining the live content screenshot, the method includes: obtaining a user identifier of a host corresponding to the live content screenshot.

[0052] In this optional embodiment, the user identifier is an identifier that uniquely identifies the identity of the anchor in the live broadcast content. The user identifier is assigned to the anchor by the live broadcast platform, or is modified by the anchor and confirmed by the platform for the convenience of platform management and user identification. Obtaining the user identifier helps to quickly and timely recall the live broadcast conducted by the corresponding anchor user in the future to avoid the leakage of illegal content.

[0053] 102. Classify and identify the live content screenshots based on multiple preset classification models to obtain multiple category results;

[0054] In this embodiment, the multiple preset classification models specifically include at least one first classification model obtained by training a convolutional neural network model (CNN) based on a sample data set with different classification focuses, and at least one second classification model obtained by fine-tuning a small amount of sample data or processing a large pre-trained classification model based on zero sample data. Among them, convolutional neural network models usually need to be trained with the support of a large amount of labeled data, and can effectively identify illegal content in screenshots of live content. Large pre-trained classification models refer to models that are pre-trained with large data sets and can perform classification tasks. They can be models dedicated to image classification or multimodal large language models. They usually use network models based on the Transformer architecture, such as CLIP (Contrastive Language-Image Pre-training), GPT (Generative Pre-trained Transformer) models, BERT (Bidirectional Encoder Representations from Transformers), etc. These models usually learn a large number of common features and knowledge representations, which can be used for initialization solutions for various related tasks, and are adapted and optimized through transfer learning or fine-tuning, so as to perform well in specific classification tasks.

[0055] Preferably, the large pre-trained classification model adopts a CLIP model.

[0056] CLIP (Contrastive Language-Image Pre-training) is a model that uses unsupervised learning to pre-train image-text matching. Its training process covers 400 million image-text pairs, with a wide range of knowledge coverage and excellent generalization capabilities. The CLIP model can be fine-tuned based on a small amount of sample data or understand and analyze the text information and contextual relationships in live content screenshots without relying on sample data, thereby assisting in identifying illegal content. These models can classify and identify live content screenshots from different angles, so that the subsequent integration of the category results identified by multiple models can obtain more accurate and robust category results.

[0057] In an optional embodiment, the first classification model specifically also includes: an ultra-large-scale category classification model, which is obtained by training the first sub-convolutional neural network model based on a first sample training data set including violation class sample data, special class sample data and normal class sample data.

[0058] In this optional embodiment, the first sub-convolutional neural network model needs to have a model with strong computing power and complex network structure to adapt to the classification task of super-large-scale categories, so as to process and distinguish a large number of sample data of different categories. The training data set of the super-large-scale category classification model includes illegal class sample data, special class sample data and normal class sample data. These data sets cover various categories that may appear in the live content, so that the model can more accurately identify and classify different scenes in the live content. By using a super-large-scale training data set, the model can learn more subtle and complex features and improve the recognition accuracy of illegal content. For example, the Inception model, ResNet (ResidualNetwork) model, VGG (Visual Geometry Group) model, DenseNet (Dense Connectivity) model, etc. These models can automatically extract and learn key features in images through deep learning technology, thereby realizing efficient recognition of scenes in live content.

[0059] Preferably, the first sub-convolutional neural network model adopts the Inception model.

[0060] In this preferred embodiment, the Inception model is a convolutional neural network model architecture with low computational cost and powerful image recognition performance, which aims to improve the accuracy and efficiency of image recognition by increasing the depth and width of the network. The model architecture uses convolution kernels of multiple different scales for feature extraction, and fuses feature maps of different scales through parallel connection, so as to capture richer and more complex feature information in the image. The first sub-convolutional neural network model can further specifically use the Inception-v4 model, where Inception-v4 is a model further evolved from the Inception model architecture. It maintains the advantages of the original Inception model and introduces more depth and width to further improve the accuracy and efficiency of image recognition, making the model perform better when processing complex images.

[0061] In this optional embodiment, the violation sample data refers to sample data that clearly contains elements that are not suitable for public viewing. Special sample data refers to sample data that may contain a certain degree of borderline elements, but has not reached the level of violation. Normal sample data refers to sample data that does not contain any violation or special content and belongs to normal scenes such as daily life, work, and study. These sample data can be provided by establishing a database, or can be collected legally and compliantly from the Internet through web crawler technology, or can be obtained through cooperation with third-party data providers. This application does not make specific restrictions on this. These sample data usually need to be strictly screened and labeled to ensure their accuracy, representativeness and diversity, so as to cover as wide a range of violations, special and normal situations as possible, so as to construct the first sample data set required for training the ultra-large-scale category classification model, provide the model with more comprehensive and accurate learning materials, and improve the accuracy and robustness of the model in identifying illegal content. Among them, the data volume of the first sample data set needs to reach the million level. In actual testing, it is determined that when the first sample data set exceeds 4 million, a more accurate ultra-large-scale category classification model can be trained.

[0062] In this embodiment, the training of the Inception-v4 model using the first sample training data set can adopt the existing Inception model framework, or use a deep learning framework such as TensorFlow or PyTorch to build and train the Inception-v4 model, and preprocess the sample data in the first sample data set to make it meet the input requirements of the Inception-v4 model, so that the sample data in the first sample data set can be input into the Inception-v4 model to train the model. During the training process, the model calculates the prediction result through forward propagation, and adjusts the model parameters through back propagation to minimize the error between the prediction result and the actual label. Through multiple iterations, the performance area of ​​the model on the verification set is stable, and the expected recognition accuracy is achieved, and the ultra-large-scale category classification model is obtained. Through the ultra-large-scale category classification model, the live content screenshots can be classified and identified, and the category results of the violation class corresponding to the sample data in the first sample data of the training, the category results of the special class, and the category results of the normal class are obtained.

[0063] In an optional embodiment, the first classification model specifically also includes: a high-accuracy category classification model, which is obtained by training the second sub-convolutional neural network model based on a second sample data set including violation class sample data, suspected violation class sample data, special class sample data and normal class sample data.

[0064] In this optional embodiment, the second sub-convolutional neural network model is adapted to the classification task of high-accuracy categories, and also needs to have a model with strong computing power and complex network structure. It can be selected from convolutional neural network models including Inception model, ResNet model, VGG model, DenseNet model, etc.

[0065] Preferably, the second sub-convolutional neural network model adopts a ResNet model.

[0066] In this preferred embodiment, the ResNet model is a deep convolutional neural network that is used to solve the gradient vanishing and gradient exploding problems in deep neural network training. The structure of ResNet enables the network to learn the residual mapping between input and output, train hundreds or even thousands of deep networks, significantly improve the speed and accuracy of network training, and improve the performance of the model. The second sub-convolutional neural network model can further specifically use the ResNet101 model, where ResNet101 is a deep ResNet model with a 101-layer deep structure. ResNet101 further enhances the network's expressive power by introducing more residual blocks, allowing the model to capture more complex and subtle features. This deep structure performs well in computer vision tasks such as image recognition, target detection, and semantic segmentation. The ResNet101 model can more easily train very deep neural networks, which is suitable for training more distinctive models for feature segmentation.

[0067] In this optional embodiment, the aforementioned violation sample data, special sample data, and normal sample data are not described in detail. The suspected violation sample data refers to sample data that may contain vague, obscure, or suggestive content, which may be considered inappropriate in some cases, but is not as obvious as the violation sample data. The addition of this type of data helps to improve the model's ability to identify edge cases, thereby reducing misjudgments in practical applications.

[0068] In this embodiment, the construction of the corresponding second sample data set also needs to undergo strict screening and labeling to ensure its accuracy, representativeness and diversity, so as to improve the generalization ability of the trained model. The high-accuracy category classification model can achieve a high recognition accuracy through the characteristics of these sample data. When constructing the second sample data set, the violation class and the suspected violation class are deliberately divided so that the model can better distinguish clear violation content and improve accuracy. In addition, the amount of data of the normal class in the second sample data set is usually set to more than 60%, so that the model can see more normal data and improve the accuracy of category classification. In actual testing, the amount of data in the second sample data set also needs to reach the million level, and it is determined that when the second sample data set reaches about 850,000 data, a more accurate high-accuracy category classification model can be trained.

[0069] In this optional embodiment, when training the ResNet101 model, a method similar to the Inception-v4 model can be used. First, the sample data in the second sample data set is preprocessed to meet the input requirements of the ResNet101 model. Then, the preprocessed sample data is input into the ResNet101 model for training. During the training process, the prediction results are calculated by forward propagation, and the model parameters are adjusted by back propagation to minimize the error between the prediction results and the actual labels. Through multiple iterations, the performance of the model on the validation set is stable, the expected recognition accuracy is achieved, and the high-accuracy category classification model is obtained. The live content screenshots can be classified and identified by the ultra-large-scale category classification model, and the category results of the violation class corresponding to the sample data in the second sample data set for training, the category results of the suspected violation class, the category results of the special class, and the category results of the normal class are obtained.

[0070] In an optional embodiment, the first classification model specifically also includes: a high recall category classification model, which is obtained by training a third sub-convolutional neural network model based on a third sample data set including real-person violation sample data, cartoon violation sample data, special class sample data and normal class sample data.

[0071] In this optional embodiment, the performance requirements and the available models of the third sub-convolutional neural network are not described in detail here. The third sub-convolutional neural network model also preferably adopts the ResNet model, and further adopts the ResNet101 model. Its training process is similar to the aforementioned embodiment, but the third sample data set pays special attention to the diversity of real-life violation sample data and animation violation sample data when constructing. The real-life violation sample data covers various shooting angles, scenes and character features, while the animation violation sample data includes animation works of different styles and themes. This diverse sample data helps to improve the model's ability to identify different types of violation content, thereby improving the recall rate in practical applications. In addition to the real-life violation class and animation violation class sample data, the third sample data set also includes special class sample data and normal class sample data to maintain the comprehensiveness of the covered recognition results. It is not described in detail here. The amount of normal class data in the model is maintained at about 40%. Through this diverse sample data, the model can better learn and distinguish normal content, special content and violation content.

[0072] In this optional embodiment, the data volume of the third sample data set also needs to reach the million level. In actual testing, through multiple iterations and optimizations, when the third sample data set reaches a data volume of about 2.97 million, a relatively accurate high-recall category classification model can be trained. Through this model, the screenshots of the live content can be classified and identified, and the category results of the real person violation class, the category results of the animation violation class, the category results of the special class, and the category results of the normal class corresponding to the sample data in the trained third sample data set can be obtained.

[0073] In an optional embodiment, the first classification model specifically also includes: a highly fine-grained category classification model, which is obtained by training a fourth sub-convolutional neural network model based on a fourth sample data set including a variety of serious violation sample data, a variety of borderline violation sample data, and normal sample data.

[0074] In this optional embodiment, there are multiple optional implementation methods of the fourth sub-convolutional neural network for the construction of a high-grained category classification model. For details, please refer to the selection of the aforementioned sub-convolutional neural network model, which will not be repeated here. For example, the high-grained category classification model can be obtained through in-depth training and adjustment of the ResNet model to achieve accurate recognition and distinction of various subtle illegal content, thereby achieving more diverse recognition results. In the highly fine-grained recognition process, the image style, body part characteristics, and behavioral characteristics in the screenshots of the live content will be analyzed and examined, and normal sample data is also provided to ensure that the model can fully learn and understand the boundaries between violations, special and normal.

[0075] In this optional embodiment, it should be noted that the various types of serious violation sample data and the various types of borderline violation sample data mentioned are all sample data of violation content and special content that appear at a high rate in live broadcasts. By subdividing the sample data, the identification of these scene contents can be achieved more accurately and meticulously in the future.

[0076] In this optional embodiment, specifically in model training, one or more of the sample data including these categories can be used, the categories of sample data can be appropriately increased, or certain sample data categories can be redefined to update the sample data of the corresponding categories. For example, there can also be violation and special sample data for different people, violation and special sample data for different countries, violation and special sample data for different scenarios, and other items in the scene can also be combined to identify illegal behaviors including violations, such as detecting elements such as drugs, instruments, and scars in screenshots to determine whether the violation content is with the help of illegal drugs or instruments, or violation content that causes physical harm. Once these situations are identified, an alarm mechanism can be collaboratively set up to deal with the relevant anchors in accordance with the law. Through this multi-dimensional classification and refinement of sample data, the training of the model will be more comprehensive and accurate, thereby improving the diversity and accuracy of its recognition in practical applications.

[0077] In this optional embodiment, the data volume of the fourth sample data set also needs to reach the million level. In actual testing, through multiple iterations and optimizations, when the fourth sample data set reaches a data volume of 1 million, a relatively accurate high-grained category classification model can be trained. Through this model, the live content screenshots can be classified and identified, and the category results of the subdivided violation class, the subdivided special class, and the normal class corresponding to the sample data in the trained fourth sample data set can be obtained.

[0078] In this optional embodiment, it should be understood that the ultra-large-scale category classification model, the high-accuracy category classification model, the high-recall category classification model and the high-granularity category classification model are all models based on large-scale supervised learning, and each model can learn excellent representation capabilities and high accuracy. However, these models are more dependent on high-quality training data, have limited generalization capabilities in complex scenarios, and have weak recall capabilities for unknown positive samples. In order to make up for this deficiency, the present invention combines a second classification model to ensure strong representation capabilities, high accuracy, high recall and high granularity of recognition, while making the final recognition result more generalizable and having a stronger recall capability for positive samples.

[0079] In an optional embodiment, the second classification model includes: a first few-sample category classification model, which is obtained by fine-tuning the text-side features of a large pre-trained classification model based on a fifth sample data set including violation class sample data, special class sample data and normal class sample data.

[0080] In this optional embodiment, the preferred CLIP model is also used as an example of the large pre-trained classification model. Since the CLIP model performs well in processing the association task between images and texts, the sample data category in the fifth sample data set that can be set by the CLIP model can be obtained by fine-tuning the text-side features of the CLIP model, thereby obtaining a first small sample category classification model. The semantic information of samples of different categories can be understood by this model, so that a high classification performance can be maintained even in the case of a small number of samples.

[0081] In this optional embodiment, in the process of fine-tuning the CLIP model, the CoOp (Context Optimization) method can be used, and the model structure of CLIP, such as Vision Transformer-b16 and weight parameters, can be fine-tuned and optimized. Specifically, the CLIP model can be pre-trained by the CoOp method first, so that it has the basic ability to process image and text association tasks. Then, by fine-tuning on the fifth sample data set, the model can be better adapted to the classification task of the category. In the fine-tuning process, the CoOp method dynamically adjusts the weight parameters of the CLIP model according to the sample data category in the fifth sample data set, and enables the model to continuously learn the numerical value of the feature vector by gradient descent. For example, for illegal sample data, the CoOp method optimizes the model to enhance its illegality and recognition ability for illegal content; for special sample data, the CoOp method adjusts the model to identify those special contents that may have fuzzy boundaries; and for normal sample data, the CoOp method optimizes the model to ensure that it does not mistakenly classify normal content as a category.

[0082] In the actual test of this optional embodiment, 87 sample data of violation classes, 80 sample data of special classes and 86 sample data of normal classes were specifically selected, and a total of 253 sample data can achieve a good classification and recognition effect. These numbers can be adjusted and changed appropriately, and this application does not impose specific restrictions on this, but it should be understood that through the use of this model, only hundreds of orders of magnitude are needed to accurately distinguish the live broadcast content of violation classes, special classes and normal classes. The use of this first few-sample category classification model can extract text features in advance and store them as global variables, so that no redundant text feature calculations are required each time image-text matching is performed, which significantly reduces the amount of calculation, has a strong generalization ability, and reduces dependence on manual operation annotation.

[0083] In an optional embodiment, the second classification model includes: a second few-sample category classification model, which is obtained by fine-tuning the text-side features of a large pre-trained classification model based on a sixth sample data set including real-life violation sample data, cartoon violation sample data, special class sample data and normal class sample data.

[0084] In this optional embodiment, the second few-sample category classification model is similar to the first few-sample category classification model, but its sample data set is richer, covering violation sample data from two different sources: real people and animation. Such a classification model can better adapt to illegal content from different sources and improve the generalization ability and recognition accuracy of the model. In practical applications, the second few-sample category classification model can effectively distinguish between real-person violations, animation violations, special and normal categories of content by fine-tuning the text-side features of the large pre-trained classification model. For example, in live broadcast content, real-person violation content and animation violation content often have different visual features and semantic information, and the second few-sample category classification model can accurately identify these differences, thereby providing a more accurate supervision tool for the live broadcast platform.

[0085] In this optional embodiment, the second small sample category classification model can also be optimized using the CoOp method during the fine-tuning process. By fine-tuning on the sixth sample data set, the model can better adapt to the classification task of illegal content from different sources. During the fine-tuning process, the CoOp method dynamically adjusts the weight parameters of the large pre-trained classification model according to the sample data category, so that the model can continuously learn and optimize the value of the feature vector, thereby improving the degree of violation and recognition of different illegal content.

[0086] In the actual test of this optional embodiment, 99 real-life violation sample data, 39 animation violation sample data, 49 special sample data and 86 normal sample data were selected, totaling 273 sample data to achieve a good classification and recognition effect. These numbers can also be adjusted and changed appropriately, and this application does not make specific restrictions on this, but it should be understood that through the use of this model, only hundreds of orders of magnitude are needed to accurately distinguish between different sources of violation, special and normal live broadcast content.

[0087] In an optional embodiment, the second classification model includes: a zero-sample category classification model, which is used to classify and identify screenshots of live content based on category labels of violation category, special category and normal category.

[0088] In this optional embodiment, unlike the first few-sample category classification model and the second few-sample category classification model, the zero-sample category classification model does not require any sample data. It utilizes the pre-training capability of the large pre-trained classification model and defines clear category labels without any sample data or any fine-tuning, so that the model can directly classify and identify screenshots of live content. Specifically, the zero-sample category classification model constructs a category label library containing violation categories, special categories, and normal categories. It only needs to set corresponding category prompt words, such as [a illegal image, a specific image, a normal image], so that the large pre-trained classification model can directly classify the image content through semantic understanding. The resulting zero-sample category classification model can quickly identify screenshots of live content and provide instant classification and recognition results.

[0089] In this embodiment, the first few-sample category classification model, the second few-sample category classification model and the zero-sample category recognition model can reuse the features of the large pre-trained classification image encoding, so that the three models only need to perform image feature calculation once. At the same time, since the text features are extracted in advance and stored as global variables, there is no need to perform redundant text feature calculations each time the image and text matching is performed, which significantly reduces the amount of calculation. In addition, the first few-sample category classification model, the second few-sample category classification model and the zero-sample model also make full use of the large and universal characteristics of the large pre-trained classification model. On the one hand, the zero-sample category recognition model is used for rapid deployment to cover a wide range of detection. On the other hand, by fine-tuning the first few-sample category classification model and the second first few-sample category classification model, they are optimized for specific tasks to improve detection accuracy, which not only ensures the wide applicability of the model, but also takes into account the performance requirements of specific tasks.

[0090] 103. Mapping multiple category result categories to preset categories to obtain multiple pending categories;

[0091] In this embodiment, category mapping (Label Mapping) is a technology that maps multiple category results to preset categories, and its purpose is to convert the category results output by the classification model into a unified category label for subsequent processing and decision-making. Among them, the preset categories specifically include at least violation categories and normal categories. Violation categories and normal categories are the two most important categories that need to be distinguished in content identification, which are directly related to whether the live content can be displayed and broadcasted. At the same time, other preset categories can also be appropriately added, such as using special categories to summarize the category results of special categories that are not very similar to violation categories and normal categories, and are between the two.

[0092] In this embodiment, the implementation of category mapping usually requires first defining a preset category label set, such as: {violation, special, normal}. These labels will be used as the output of the final classification result. Secondly, a mapping relationship table is established to correspond each category result output by the classification model with the preset category label. For example, if the result output by the classification model is a category result of a violation class, a category result of a suspected violation class, a category result of a real person violation class, or a category result of an animation violation class, it is mapped to the pending category of the "violation" label. If the result output by the classification model is a category result of a special class, it is mapped to the pending category of the "special" label, etc.

[0093] See also Figure 2 In an optional embodiment, multiple category result categories are mapped to preset categories to obtain multiple pending categories, including:

[0094] 1031. Map the category results of the suspected violation class, the category results of the real person violation class, the category results of the animation violation class, and the category results of the serious violation class to the preset category of the violation class to obtain the pending category of the violation class;

[0095] 1032. From the preset categories of borderline violation categories, a pending category of special category is obtained.

[0096] In this optional embodiment, the category mapping method of the category results identified by the above-mentioned seven classification models, namely, the ultra-large-scale category classification model, the high-accuracy category classification model, the high-recall category classification model, the high-granularity category classification model, the first few-sample category classification model, and the second few-sample category classification model is specifically described. In addition to the relatively unified category results of the normal class, the other category results are unified into the pending category of the violation class and the pending category of the special class. Through the above-mentioned category identification and category mapping with different focuses, it is possible to take into account the refined management and effective classification of various complex illegal contents, and realize accurate identification and efficient processing of different types of illegal contents.

[0097] 104. Perform weighted voting on multiple pending categories according to preset categories to determine the category of the live content screenshot.

[0098] In this embodiment, weighted voting is a soft voting method that takes into account the credibility and importance of different pending categories. By assigning a weight to each category, and then voting on multiple pending categories according to these weights, the category of the live content screenshot is finally determined. Specifically, the weights can be set according to the performance of each classification model, such as the accuracy and recall rate, and the rationality of the category mapping. For example, a higher weight can be assigned to the pending category output by a classification model with a high accuracy rate; while a lower weight can be assigned to categories that may be misjudged, such as some marginal special categories of content.

[0099] In this embodiment, during the specific operation, the pending categories output by each classification model can be summarized first, and the weighted score of each category can be calculated, wherein the weighted score can be obtained by adding all the weights pointing to each pending category and the probability of being considered to be the category, and then, the weighted scores of the three categories are compared, and the category with the highest score is the category of the live content screenshot that is finally determined. For example, for the same live content screenshot, the A model with a weight of 0.8 believes that the violation score is 0.8 and the special score is 0.2, the B model with a weight of 1.2 believes that the violation score is 0.6, the special score is 0.3, and the normal score is 0.1, and the C model with a weight of 1 believes that the violation score is 0.6, the special score is 0.2, and the normal score is 0.2. The calculated violation score is 0.8×0.8+1.2×0.6+0.4=2.34, or the score can be divided by the sum of the weights for normalization to obtain a final score of 0.78. The scores of the special category and the normal category are not repeated here. Through weighted voting, we can fully utilize the advantages of different classification models and comprehensively consider the information of multiple pending categories, so as to more accurately determine the category of the live content screenshots and provide strong support for subsequent review and processing.

[0100] In an optional embodiment, weighted voting is performed on multiple pending categories according to preset categories to determine the category of the live content screenshot, including:

[0101] The category of the live content screenshot is determined based on the following formula:

[0102] final_result = α·(w1M cnn1 +w2M cnn2 +w3M cnn3 +w4M cnn4 )+β·(w5M fs1 +w6M fs2 +w7M zs );

[0103] Among them, final_result is the category of the live content screenshot, α is the first comprehensive weight value of the first classification model, β is the second comprehensive weight value of the second classification model, w1 is the first weight value of the ultra-large-scale category classification model, w2 is the second weight value of the high-accuracy category classification model, w3 is the third weight value of the high-recall category classification model, w4 is the fourth weight value of the high-grained category classification model, w5 is the fifth weight value of the first few-sample category classification model, w6 is the sixth weight value of the second few-sample category classification model, w7 is the seventh weight value of the zero-sample category classification model, M cnn1 is the score value of the first category result identified by the ultra-large-scale category classification model, M cnn2 M is the score value of the second category result identified by the high accuracy category classification model.cnn3 is the score value of the third category result identified by the high recall category classification model, M cnn4 is the score value of the fourth category result identified by the highly fine-grained category classification model, M fs1 is the score value of the fifth category result identified by the first few-sample category classification model, M fs2 M is the score value of the sixth category result identified by the second few-sample category classification model, zs This is the score value of the seventh category result identified by the zero-shot class classification model.

[0104] In this optional embodiment, a weighted fusion algorithm is specifically provided for the aforementioned first classification model, the second classification model and the specific seven categories of models. In this formula, the first classification model and the second classification model are respectively assigned different weights, and the seven categories of specific models also have their own weights, so that the corresponding weights can be adjusted more diversely, making the final category recognition more accurate and reliable.

[0105] In an optional embodiment, the second weight value and the third weight value are both greater than the first weight value and the fourth weight value; the first comprehensive weight value is 0.6, the second comprehensive weight value is 0.4, the first weight value is 0.9, the second weight value is 1.1, the third weight value is 1.1, and the fourth weight value is 0.8.

[0106] In this optional embodiment, the values ​​of these weight values ​​are all iteratively determined values ​​that can achieve more accurate category recognition results, and can achieve a recall rate of 100% in actual measurements. The setting of weight values ​​reflects the importance of different classification models in the comprehensive voting process. Specifically, because a high-accuracy model can more accurately identify illegal content, and a high-recall model can reduce the possibility of missed detection, ensuring that illegal content is not missed. Therefore, the second weight value and the third weight value can be set to be greater than the first weight value and the fourth weight value, so that classification models with high accuracy and high recall are given higher weights.

[0107] In this embodiment, when the two main classification models are comprehensively considered, the first comprehensive weight value is set to 0.6 and the second comprehensive weight value is set to 0.4. Among them, the first classification model occupies a larger proportion in the overall vote, and the accuracy of the large-scale supervision model is higher. And because the first classification model is obtained by combining four classification models with different focuses, it is also more robust and reliable. The second classification model uses the CLIP model of few samples and zero samples. Although it has more generalization ability and computational efficiency, it is still slightly lower than the convolutional neural network model based on large-scale supervision in the first classification model in terms of accuracy and universality. Therefore, the first comprehensive weight value is set to be greater than the second comprehensive weight value.

[0108] In this embodiment, the first weight value is 0.9, the second weight value is 1.1, the third weight value is 1.1, and the fourth weight value is 0.8, which further refines the importance of each specific model in the voting process. The high weight values ​​of high accuracy and high recall models are higher, which means that they have a greater influence in the final vote and help improve the accuracy and recall of the overall classification. The relatively low weight values ​​of the ultra-large-scale category classification model and the highly fine-grained category classification model are due to their over-reliance on the amount of data or over-refinement in some cases, resulting in certain limitations in practical applications. Through this comprehensive weighted voting method, the advantages of different classification models can be fully utilized while balancing their limitations, thereby achieving higher accuracy and reliability when determining the category of live content screenshots. This diversified weight setting method not only improves the accuracy of classification, but also provides a more solid foundation for subsequent review and processing.

[0109] See also Figure 3 In an optional embodiment, after performing weighted voting on a plurality of pending categories according to preset categories to determine the category of the live content screenshot, the method further includes:

[0110] 1051. Obtain categories of multiple live content screenshots of a target live broadcast at a preset frequency within a preset time period;

[0111] 1052. If the multiple categories include a violation category, recall the target live broadcast based on the user identifier;

[0112] 1053. If the number of special categories in multiple categories is greater than a preset recall threshold, the target live broadcast is recalled based on the user identifier, or the target live broadcast is fed back to the content review platform.

[0113] In this optional embodiment, by acquiring the categories of multiple live content screenshots of the target live broadcast at a preset frequency within a preset time period, the target live broadcast can be monitored in real time or periodically. By acquiring at a preset frequency within a preset time period, such as acquiring six screenshots per minute, acquiring one screenshot per minute, acquiring one screenshot every five minutes, and acquiring one screenshot per hour, multiple content screenshots during the live broadcast can be acquired and analyzed continuously. Therefore, the screenshots can be processed in sequence according to the aforementioned steps 101, 102, 103, and 104 to determine the categories of the multiple live content screenshots.

[0114] In this optional embodiment, if any of the analyzed screenshots is identified as illegal content, the live broadcast platform can recall the target live broadcast based on the user identifier through the live broadcast removal mechanism or a dedicated command control system, that is, stop broadcasting the live broadcast, and may further review or trigger it. At the same time, the anchor can also be restricted from broadcasting again based on the user identifier to ensure the green health of the live broadcast platform.

[0115] In this optional embodiment, a more detailed approach is adopted for special content. Since special content may be somewhat ambiguous and controversial, the system will not recall the live broadcast immediately upon discovery. Instead, it will count the number of special categories within a preset time period and compare it with the preset recall threshold. If the number of special categories exceeds this threshold, the system may choose to recall the live broadcast, or more cautiously feedback the live broadcast to a dedicated content review platform for further manual review. This approach can ensure that obvious illegal content is handled quickly and effectively, while avoiding excessive suppression of special content that may be controversial, thereby ensuring the compliance of platform content while also taking into account a certain content diversity and user experience.

[0116] This embodiment improves the accuracy and robustness of illegal content identification and reduces the false alarm rate and missed alarm rate through the combination of multiple classification models. By using a weighted voting mechanism, it is also possible to assign corresponding weights according to the performance characteristics and focus of different models, so that the final classification result is more reliable. By using few-sample and zero-sample models, it can effectively solve the problem of sample scarcity in the real world, effectively improve the generalization ability of the model, reduce the amount of calculation, and ensure the comprehensiveness and diversity of category identification, so that more live content screenshots can be processed at the same time, realizing real-time monitoring and automatic identification of live content, reducing the pressure of manual review, and improving the efficiency of identification and review and the ability to recall illegal content, which can ensure the green and healthy live broadcast, is conducive to maintaining a good network environment and live broadcast experience, improves the management efficiency of live content and the security and legality of live content, and maintains a healthy corporate image and user trust.

[0117] In order to execute the corresponding steps in the above method embodiment and each possible implementation method, the following provides an implementation method of a live content category identification device. Figure 4 , Figure 4 1 is a functional module diagram of an embodiment of a device for identifying the category of live content in an embodiment of the present invention. In this embodiment, the device for identifying the category of live content is applied to an electronic device, and the device for identifying the category of live content includes:

[0118] Acquisition module 201, used to acquire screenshots of live content;

[0119] The identification module 202 is used to classify and identify the live content screenshots based on multiple preset classification models to obtain multiple category results, wherein the multiple preset classification models include at least one first classification model obtained by training a convolutional neural network model based on sample data sets with different classification focuses, and at least one second classification model obtained by fine-tuning a small amount of sample data or processing a large pre-trained classification model based on zero sample data;

[0120] A mapping module 203 is used to map the multiple category result categories to preset categories to obtain multiple pending categories, wherein the preset categories at least include a violation category and a normal category;

[0121] The settlement module 204 is used to perform weighted voting on multiple pending categories according to preset categories to determine the category of the live content screenshot.

[0122] Optionally, in one embodiment, the acquisition module 201 is further specifically used to: obtain a user identifier of a host corresponding to the live content screenshot.

[0123] Optionally, in one embodiment, the multiple preset classification models in the recognition module 202 specifically include at least one first classification model obtained by training a convolutional neural network model based on a sample data set with different classification focuses, and at least one second classification model obtained by fine-tuning a small amount of sample data or processing a large pre-trained classification model based on zero sample data.

[0124] Optionally, in one embodiment, the first classification model specifically includes: an ultra-large-scale category classification model, which is obtained by training a first sub-convolutional neural network model based on a first sample training data set including violation class sample data, special class sample data and normal class sample data; a high-accuracy category classification model, which is obtained by training a second sub-convolutional neural network model based on a second sample data set including violation class sample data, suspected violation class sample data, special class sample data and normal class sample data; a high-recall category classification model, which is obtained by training a third sub-convolutional neural network model based on a third sample data set including real-person violation class sample data, cartoon violation class sample data, special class sample data and normal class sample data; a highly fine-grained category classification model, which is obtained by training a fourth sub-convolutional neural network model based on a fourth sample data set including a variety of serious violation class sample data, a variety of borderline violation class sample data and normal class sample data.

[0125] Optionally, in one embodiment, the second classification model specifically includes: a first few-sample category classification model, which is obtained by fine-tuning the text-side features of a large pre-trained classification model based on a fifth sample data set including violation class sample data, special class sample data and normal class sample data; a second few-sample category classification model, which is obtained by fine-tuning the text-side features of a large pre-trained classification model based on a sixth sample data set including real-person violation class sample data, cartoon violation class sample data, special class sample data and normal class sample data; a zero-sample category classification model, which is used to classify and identify screenshots of live content based on category labels of violation class, special class and normal class.

[0126] Optionally, in one embodiment, the mapping module 203 is specifically used to: map the category results of the suspected violation class, the category results of the real-person violation class, the category results of the cartoon violation class, and the category results of the serious violation class to the preset category of the violation class to obtain the pending category of the violation class; and obtain the pending category of the special class from the preset category of the borderline violation class.

[0127] Optionally, in one embodiment, the settlement module 204 determines the category of the live content screenshot based on the following formula:

[0128] final_result = α·(w1M cnn1 +w2M cnn2 +w3M cnn3 +w4M cnn4 )+β·(w5M fs1 +w6M fs2 +w7M zs );

[0129] Among them, final_result is the category of the live content screenshot, α is the first comprehensive weight value of the first classification model, β is the second comprehensive weight value of the second classification model, w1 is the first weight value of the ultra-large-scale category classification model, w2 is the second weight value of the high-accuracy category classification model, w3 is the third weight value of the high-recall category classification model, w4 is the fourth weight value of the high-grained category classification model, w5 is the fifth weight value of the first few-sample category classification model, w6 is the sixth weight value of the second few-sample category classification model, w7 is the seventh weight value of the zero-sample category classification model, M cnn1 is the score value of the first category result identified by the ultra-large-scale category classification model, M cnn2 M is the score value of the second category result identified by the high accuracy category classification model. cnn3 is the score value of the third category result identified by the high recall category classification model, M cnn4 is the score value of the fourth category result identified by the highly fine-grained category classification model, M fs1is the score value of the fifth category result identified by the first few-sample category classification model, M fs2 M is the score value of the sixth category result identified by the second few-sample category classification model, zs This is the score value of the seventh category result identified by the zero-shot class classification model.

[0130] Optionally, in one embodiment, the second weight value and the third weight value are both greater than the first weight value and the second weight value, the first comprehensive weight value is 0.6, the second comprehensive weight value is 0.4, the first weight value is 0.9, the second weight value is 1.1, the third weight value is 1.1, and the fourth weight value is 0.8.

[0131] Optionally, in one embodiment, the settlement module 204 is specifically used to: obtain categories of multiple live content screenshots of the target live broadcast at a preset frequency within a preset time period; if the multiple categories include a violation category, recall the target live broadcast based on the user identifier; if the number of special categories in the multiple categories is greater than a preset recall threshold, recall the target live broadcast based on the user identifier, or feed back the target live broadcast to the content review platform.

[0132] Since the embodiments of the device part correspond to the embodiments of the above-mentioned method, please refer to the above-mentioned method embodiments for the introduction of the live content category identification device provided in the embodiments of the present invention. The embodiments of the present invention will not be repeated here, and it has the same beneficial effects as the above-mentioned live content category identification method.

[0133] The present invention also provides a computer device, which includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the above-mentioned method for identifying the category of live content. Figure 5 3 is a functional module diagram of a computer device provided by an embodiment of the present invention. The computer device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage devices) storing application programs 333 or data 332. Among them, the memory 320 and the storage medium 330 can be temporary storage or permanent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the computer device 300. Furthermore, the processor 310 can be configured to communicate with the storage medium 330 to execute a series of instruction operations in the storage medium 330 on the computer device 300.

[0134] The computer device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 5 The illustrated computer device structure does not constitute a limitation on the computer device, and may include more or fewer components than illustrated, or combine certain components, or arrange the components differently.

[0135] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the method for identifying the category of live content as described above.

[0136] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. If the integrated module or unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0137] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying the category of live content, characterized in that: The category identification method comprises: Get screenshots of live broadcast content; Classify and identify the live content screenshots based on multiple preset classification models to obtain multiple category results, wherein the multiple preset classification models include at least one first classification model obtained by training a convolutional neural network model based on sample data sets with different classification focuses, and at least one second classification model obtained by fine-tuning a small amount of sample data or processing a large pre-trained classification model based on zero-sample data; Mapping the multiple category result categories to preset categories to obtain multiple pending categories, wherein the preset categories at least include a violation category and a normal category; A weighted vote is performed on the multiple pending categories according to the preset categories to determine the category of the live content screenshot.

2. The method for identifying the category of live content according to claim 1, characterized in that: The first classification model includes: The ultra-large-scale category classification model is obtained by training a first sub-convolutional neural network model based on a first sample training data set including violation class sample data, special class sample data, and normal class sample data; A high-accuracy category classification model is obtained by training a second sub-convolutional neural network model based on a second sample data set including violation class sample data, suspected violation class sample data, special class sample data, and normal class sample data; A high recall category classification model is obtained by training a third sub-convolutional neural network model based on a third sample data set including real-person violation sample data, cartoon violation sample data, special class sample data, and normal class sample data; The highly fine-grained category classification model is obtained by training the fourth sub-convolutional neural network model based on a fourth sample data set including a variety of serious violation sample data, a variety of borderline violation sample data, and normal sample data.

3. The method for identifying the category of live content according to claim 1 or 2, characterized in that: The second classification model includes: A first small sample category classification model is obtained by fine-tuning the text-side features of the large pre-trained classification model based on a fifth sample data set including violation class sample data, special class sample data, and normal class sample data; A second small sample category classification model is obtained by fine-tuning the text-side features of the large pre-trained classification model based on a sixth sample data set including real-life violation sample data, cartoon violation sample data, special class sample data, and normal class sample data; The zero-sample category classification model is used to classify and identify the screenshots of the live content based on category labels of violation category, special category and normal category.

4. The method for identifying the category of live content according to claim 3, characterized in that: The preset categories include violation category, special category and normal category; The mapping of the multiple category result categories to preset categories to obtain multiple pending categories includes: Mapping the category results of the suspected violation class, the category results of the real person violation class, the category results of the animation violation class, and the category results of the serious violation class to the preset category of the violation class to obtain the pending category of the violation class; The category result of the borderline violation class is mapped to the preset category of the special class to obtain the pending category of the special class.

5. The method for identifying the category of live content according to claim 3, characterized in that: The performing weighted voting on the plurality of pending categories according to the preset categories to determine the category of the live content screenshot includes: The category of the live content screenshot is determined based on the following formula: final_result=α·(w1M cnn1 +w2M cnn2 +w3M cnn3 +w4M cnn4 )+β·(w5M fs1 +w6M fs2 +w7M zs ); Wherein, final_result is the category of the live content screenshot, α is the first comprehensive weight value of the first classification model, β is the second comprehensive weight value of the second classification model, w1 is the first weight value of the ultra-large-scale category classification model, w2 is the second weight value of the high-accuracy category classification model, w3 is the third weight value of the high-recall category classification model, w4 is the fourth weight value of the high-granularity category classification model, w5 is the fifth weight value of the first few-sample category classification model, w6 is the sixth weight value of the second few-sample category classification model, w7 is the seventh weight value of the zero-sample category classification model, M cnn1 is the score value of the first category result identified by the ultra-large-scale category classification model, M cnn2 is the score value of the second category result identified by the high accuracy category classification model, M cnn3 is the score value of the third category result identified by the high recall category classification model, M cnn4 is the score value of the fourth category result identified by the highly fine-grained category classification model, M fs1 is the score value of the fifth category result identified by the first few-sample category classification model, M fs2 is the score value of the sixth category result identified by the second few-sample category classification model, M zs The result score value of the seventh category identified by the zero-sample category classification model.

6. The method for identifying the category of live content according to claim 5, characterized in that: The second weight value and the third weight value are both greater than the first weight value and the fourth weight value; the first comprehensive weight value is 0.6, the second comprehensive weight value is 0.4, the first weight value is 0.9, the second weight value is 1.1, the third weight value is 1.1, and the fourth weight value is 0.

8.

7. The method for identifying the category of live content according to claim 1, characterized in that: After obtaining the screenshot of the live content, the method includes: Obtain a user identifier of the anchor corresponding to the live content screenshot; After performing weighted voting on the plurality of pending categories according to the preset categories to determine the category of the live content screenshot, the method further includes: Acquire the categories of a plurality of live content screenshots of a target live broadcast at a preset frequency within a preset time period; If the multiple categories include the category of violation, recalling the target live broadcast based on the user identifier; If the number of the special categories among the multiple categories is greater than a preset recall threshold, the target live broadcast is recalled based on the user identifier, or the target live broadcast is fed back to the content review platform.

8. A device for identifying the category of live content, characterized in that: The category identification device comprises: The acquisition module is used to obtain screenshots of live content; An identification module, configured to classify and identify the live content screenshots based on a plurality of preset classification models to obtain a plurality of category results, wherein the plurality of preset classification models include at least one first classification model obtained by training a convolutional neural network model based on sample data sets with different classification focuses, and at least one second classification model obtained by fine-tuning a small amount of sample data or processing a large pre-trained classification model based on zero-sample data; A mapping module, used for mapping the multiple category result categories to preset categories to obtain multiple pending categories, wherein the preset categories at least include a violation category and a normal category; The settlement module is used to perform weighted voting on the multiple pending categories according to the preset categories to determine the category of the live content screenshot.

9. A computer device, characterized in that: The computer device comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the computer device to execute the method for identifying the category of live content according to any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the method for identifying the category of live content as described in any one of claims 1-7 is implemented.

Images