Method, medium, and computing device for identifying rule violation images

Abstract

Embodiments of the present disclosure provide a method for identifying a non-compliant picture. The method comprises: inputting a first picture to be identified into an identification model to determine compliance of the first picture based on an output result of the identification model; wherein the identification model comprises a plurality of independent supervised models and a plurality of independent semi-supervised models; and the identification model is obtained by training the plurality of supervised models and the plurality of semi-supervised models. The identification model for identifying non-compliant pictures can be trained by the plurality of independent supervised models and the plurality of independent semi-supervised models, which overcomes the problem of weak generalization ability of a single training model, thereby significantly improving the identification accuracy of the identification model for non-compliant pictures, and the identification precision of non-compliant pictures is high. In addition, the present disclosure provides a medium and a computing device.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of image recognition, and more particularly, embodiments of the present disclosure relate to a method, medium and computing device for identifying illegal pictures. BACKGROUND

[0002] This section is intended to provide background information to the embodiments of the present disclosure recited in the claims. The description herein does not constitute admission of prior art.

[0003] With the increasing demand for network purification, more efforts are needed to remove illegal pictures on the network. Illegal pictures can be found through an identification model.

[0004] In an exemplary technology, the identification model is obtained through supervised model learning. When performing supervised learning, illegal pictures need to be used as labeled samples for learning. Compared with normal pictures, the proportion of illegal pictures is small, which makes the generalization ability of the trained identification model weak, reduces the identification accuracy of the identification model for illegal pictures, and the identification accuracy of illegal pictures is low. SUMMARY

[0005] The present disclosure provides a method, medium and computing device for identifying illegal pictures to solve the problem of low identification accuracy of illegal pictures.

[0006] In a first aspect of the embodiments of the present disclosure, a method for identifying illegal pictures is provided, comprising: inputting a first picture to be identified into an identification model to determine the compliance of the first picture based on the output result of the identification model; wherein the identification model comprises a plurality of independent supervised models and a plurality of independent semi-supervised models; the identification model is obtained by training the plurality of supervised models and the plurality of semi-supervised models.

[0007] In an embodiment of the present disclosure, the identification model is obtained by training the plurality of supervised models and the plurality of semi-supervised models, comprising: training the plurality of supervised models and the plurality of semi-supervised models through sample data in a sample data pool to obtain the identification model; wherein the sample data includes first unlabeled data appearing in a real illegal picture identification service, and second unlabeled data collected from a public picture resource pool

[0008] In another embodiment of the present disclosure, before the step of training the plurality of supervised models and the plurality of semi-supervised models by sample data in a sample data pool to obtain the identification model, the method further comprises: pre-training the plurality of supervised models and the plurality of semi-supervised models by sample data in a cold start data pool to make the corresponding plurality of supervised models and the corresponding plurality of semi-supervised models have label prediction capability; wherein the sample data in the cold start data pool is the first labeled data artificially labeled, and the sample quantity of the first labeled data is less than a preset quantity.

[0009] In another embodiment of the present disclosure, the step of training the plurality of supervised models and the plurality of semi-supervised models by sample data in a sample data pool further comprises: adding labels to the first unlabeled data; and adding labels to the second unlabeled data.

[0010] In another embodiment of the present disclosure, the step of adding labels to the first unlabeled data comprises: inputting the first unlabeled data into the plurality of supervised models to obtain a first probability prediction value of a violation category to which each supervised model belongs for a first sample picture in the first unlabeled data; in response to the first probability prediction value of the violation category to which the supervised model belongs for the first sample picture being greater than a preset first probability threshold, adding a first violation category label determined by the supervised model to the first sample picture; determining a first prediction label of the first sample picture from the first violation category labels determined by each of the supervised models based on a voting mechanism, and adding the first prediction label to the first sample picture belonging to the first unlabeled data to obtain second labeled data.

[0011] In another embodiment of the present disclosure, the step of adding labels to the second unlabeled data comprises: adding labels of positive samples to a first part of the second unlabeled data; and adding labels of negative samples to a second part of the plurality of second unlabeled data.

[0012] In another embodiment of the present disclosure, the adding positive sample labels to the second portion of the second unlabeled data comprises: inputting the second unlabeled data into the plurality of supervised models to obtain a second probability prediction value of a violation category to which each of the supervised models predicts a second sample picture in the second unlabeled data to belong; in response to the second probability prediction value of the violation category to which the supervised model predicts the second sample picture to belong being greater than a second preset probability threshold and less than a third preset probability threshold, adding a second violation category label determined by the supervised model to the second sample picture; determining a second prediction label of the corresponding sample picture from the second violation category labels determined by each of the supervised models based on a voting mechanism, and adding the second prediction label to the second unlabeled data to which the second sample picture belongs, the second preset label being a label of a positive sample.

[0013] In another embodiment of the present disclosure, the adding negative sample labels to the second portion of the plurality of second unlabeled data comprises: retraining the plurality of supervised models according to each of the second labeled data; in response to the plurality of supervised models satisfying a stop training condition, adding negative sample labels to the second portion of the plurality of second unlabeled data.

[0014] In another embodiment of the present disclosure, the retraining the plurality of supervised models according to each of the second labeled data comprises: in response to the plurality of supervised models not satisfying the stop training condition, returning to perform the step of adding labels to the first unlabeled data.

[0015] In another embodiment of the present disclosure, the retraining the plurality of supervised models according to each of the second labeled data comprises: filtering noise data in the second labeled data by a confident learning algorithm, and retraining the plurality of supervised models based on the remaining second labeled data after filtering.

[0016] In another embodiment of the present disclosure, the retraining the plurality of supervised models according to each of the second labeled data by the confident learning algorithm comprises: determining noise labels and true labels in the second labeled data, and determining noise data in the second labeled data based on a joint distribution between the noise labels and the true labels, the noise data being sample data incorrectly labeled by the supervised model; filtering out the noise data in the second labeled data, and retraining the plurality of supervised models based on the remaining second labeled data.

[0017] In another embodiment of the present disclosure, the first probability threshold of each of the supervised models is different, the second probability threshold of each of the supervised models is different, and the third probability threshold of each of the supervised models is different.

[0018] In another embodiment of the present disclosure, the adding of the label of the negative sample to the second unlabeled data of the second part comprises: inputting the second unlabeled data into the plurality of supervised models to obtain third unlabeled data that appears to be misjudged; performing semi-supervised learning on each of the third unlabeled data and the second labeled data by using each of the semi-supervised models to obtain a semi-supervised model that appears to be misjudged in a violation category to which the third unlabeled data belongs; inputting the second unlabeled data into the semi-supervised model that appears to be misjudged in the violation category to which the third unlabeled data belongs, to obtain a third probability prediction value of the semi-supervised model for a violation category to which a second sample picture in the second unlabeled data that appears to be misjudged should belong; and adding a label of a negative sample to the second unlabeled data whose third probability prediction value is greater than a fourth preset probability threshold.

[0019] In another embodiment of the present disclosure, after the step of adding the label of the negative sample to the second unlabeled data whose third probability prediction value is greater than the fourth preset probability threshold, the method further comprises: training the plurality of supervised models by using the first unlabeled data with the added label, the second unlabeled data with the added label of the positive sample, and the second unlabeled data with the added label of the negative sample to obtain a recognition model.

[0020] In another embodiment of the present disclosure, after the step of adding the label of the negative sample to the second unlabeled data whose third probability prediction value is greater than the fourth preset probability threshold, the method further comprises: training the plurality of supervised models by using the first unlabeled data with the added label, the second unlabeled data with the added label of the positive sample, and the second unlabeled data with the added label of the negative sample; inputting the second unlabeled data into the plurality of supervised models that have been trained to obtain a fifth probability prediction value of each of the supervised models for a violation category to which a second sample picture in the second unlabeled data should belong; obtaining second unlabeled data whose fifth probability prediction value is located in a preset probability interval for manual labeling of the label; and retraining the plurality of supervised models that have been trained by using the first unlabeled data with the added label, the second unlabeled data with the added label of the positive sample, the second unlabeled data with the added label of the negative sample, and the second unlabeled data with the manual label to obtain a recognition model.

[0021] In another embodiment of the present disclosure, the first unlabeled data is obtained from text, images, and videos.

[0022] In another embodiment of the present disclosure, the initial training parameters of the supervised model and the semi-supervised model are training parameters of the base model after the base model is trained by data in real violation business.

[0023] In another embodiment of the present disclosure, the training of the base model by data in real violation business comprises: processing a third sample picture of data in real violation business to obtain a fourth sample picture and auxiliary information, the processing comprising rotation, scaling and / or coloring, and the auxiliary information being a relationship between the third sample picture and the fourth sample picture; and training the base model by the third sample picture, the fourth sample picture and the auxiliary information.

[0024] In a second aspect of the embodiments of the present disclosure, there is provided an identification device for violation pictures, comprising: an input module configured to input a first picture to be identified into an identification model to determine compliance of the first picture based on an output result of the identification model; wherein the identification model comprises a plurality of independent supervised models and a plurality of independent semi-supervised models; and the identification model is obtained by training the plurality of supervised models and the plurality of semi-supervised models.

[0025] In one embodiment of the present disclosure, the identification device for violation pictures comprises: a training module configured to train the plurality of supervised models and the plurality of semi-supervised models by sample data in a sample data pool to obtain the identification model; wherein the sample data comprises: first unlabeled data occurring in real violation picture identification business, and second unlabeled data collected from a public picture resource pool.

[0026] In another embodiment of the present disclosure, the identification device for violation pictures comprises: the training module is further configured to pre-train the plurality of supervised models and the plurality of semi-supervised models by sample data in a cold start data pool to enable the corresponding plurality of supervised models and the corresponding plurality of semi-supervised models to have label prediction capability; wherein the sample data in the cold start data pool is first labeled data artificially labeled, and a sample quantity of the first labeled data is less than a preset quantity.

[0027] In another embodiment of the present disclosure, the identification device for violation pictures comprises: an adding module configured to add labels to the first unlabeled data; and the adding module is further configured to add labels to the second unlabeled data.

[0028] In another embodiment of the present disclosure, the device for identifying pictures in violation comprises: an input module configured to input the first unlabeled data into the plurality of supervised models to obtain a first probability prediction value of each supervised model for a first sample picture in the first unlabeled data belonging to a first violation category; the adding module is further configured to add a first violation category label determined by the supervised model to the first sample picture in response to the first probability prediction value of the supervised model for the first sample picture belonging to the first violation category being greater than a preset first probability threshold; and a determination module configured to determine a first prediction label of the first sample picture from the first violation category labels determined by each of the supervised models based on a voting mechanism, and add the first prediction label to the first unlabeled data to which the first sample picture belongs to obtain second labeled data.

[0029] In another embodiment of the present disclosure, the device for identifying pictures in violation comprises: the adding module is further configured to add a label of a positive sample to a first part of the second unlabeled data; and the adding module is further configured to add a label of a negative sample to a second part of the second unlabeled data.

[0030] In another embodiment of the present disclosure, the device for identifying pictures in violation comprises: the input module is further configured to input the second unlabeled data into the plurality of supervised models to obtain a second probability prediction value of each supervised model for a second sample picture in the second unlabeled data belonging to a second violation category; the adding module is further configured to add a second violation category label determined by the supervised model to the second sample picture in response to the second probability prediction value of the supervised model for the second sample picture belonging to the second violation category being greater than a preset second probability threshold and less than a preset third probability threshold; and the determination module is further configured to determine a second prediction label of the corresponding sample picture from the second violation category labels determined by each of the supervised models based on the voting mechanism, and add the second prediction label to the second unlabeled data to which the second sample picture belongs to, wherein the second preset label is a label of a positive sample.

[0031] In another embodiment of the present disclosure, the device for identifying pictures in violation comprises: the training module is further configured to retrain the plurality of supervised models according to each of the second labeled data; and the adding module is further configured to add a label of a negative sample to a second part of the second unlabeled data in response to the plurality of supervised models satisfying a stop training condition.

[0032] In one embodiment of the present disclosure, the device for identifying pictures in violation comprises: an execution module configured to return to execute the step of adding a label to the first unlabeled data in response to the plurality of supervised models not satisfying the stop training condition.

[0033] In another embodiment of the present disclosure, the device for identifying illegal pictures comprises a filtering module configured to filter noise data in the second labeled data by a confidence learning algorithm, and retrain the plurality of supervised models based on the second labeled data remaining after filtering.

[0034] In another embodiment of the present disclosure, the device for identifying illegal pictures comprises that the determining module is further configured to determine noise labels and real labels in the second labeled data, and determine noise data in the second labeled data based on a joint distribution between the noise labels and the real labels, the noise data being sample data mislabeled by the supervised models; and the filtering module is further configured to filter out the noise data in the second labeled data, and retrain the plurality of supervised models based on the second labeled data remaining after filtering.

[0035] In another embodiment of the present disclosure, the first probability threshold of each of the plurality of supervised models in the device for identifying illegal pictures is different, the second probability threshold of each of the plurality of supervised models is different, and the third probability threshold of each of the plurality of supervised models is different.

[0036] In another embodiment of the present disclosure, the device for identifying illegal pictures comprises that the input module is further configured to input the second unlabeled data into the plurality of supervised models to obtain third unlabeled data with misjudgment; the learning module is configured to perform semi-supervised learning on each of the third unlabeled data and the second labeled data by each of the semi-supervised models to obtain a semi-supervised model with misjudgment of a violation category to which the third unlabeled data belongs; the input module is further configured to input the second unlabeled data into the semi-supervised model with misjudgment of the violation category to which the third unlabeled data belongs to obtain a third probability prediction value of a violation category to which a second sample picture in the second unlabeled data with misjudgment should belong according to the semi-supervised model; and the adding module is further configured to add a label of a negative sample to the second unlabeled data with a third probability prediction value greater than a fourth probability threshold.

[0037] In another embodiment of the present disclosure, the device for identifying illegal pictures comprises that the adding module is further configured to train the plurality of supervised models by adding the first unlabeled data with a label, the second unlabeled data with a label of a positive sample, and the second unlabeled data with a label of a negative sample to obtain an identification model.

[0038] In another embodiment of the present disclosure, the device for identifying the illegal picture comprises: the adding module is further configured to train the plurality of supervised models by adding the first unlabeled data with labels, the second unlabeled data with labels of positive samples, and the second unlabeled data with labels of negative samples; the inputting module is further configured to input the second unlabeled data into the plurality of trained supervised models to obtain fifth probability prediction values of the illegal categories to which the second sample pictures in the second unlabeled data belong according to each of the supervised models; the obtaining module is configured to obtain the second unlabeled data with the fifth probability prediction values in the preset probability interval for manual labeling; and the adding module is further configured to re-train the plurality of trained supervised models by adding the first unlabeled data with labels, the second unlabeled data with labels of positive samples, the second unlabeled data with labels of negative samples, and the second unlabeled data with manual labels to obtain the identification model.

[0039] In another embodiment of the present disclosure, the first unlabeled data is obtained from texts, images, and videos.

[0040] In another embodiment of the present disclosure, the initial training parameters of the supervised models and the semi-supervised models in the device for identifying the illegal picture are training parameters of a basic model after the basic model is trained by data in real illegal businesses.

[0041] In another embodiment of the present disclosure, the device for identifying the illegal picture comprises: a processing module configured to process third sample pictures in data in real illegal businesses to obtain fourth sample pictures and auxiliary information, wherein the processing includes rotation, scaling, and / or coloring, and the auxiliary information is a relationship between the third sample pictures and the fourth sample pictures; and the training module is further configured to train the basic model by the third sample pictures, the fourth sample pictures, and the auxiliary information.

[0042] In a third aspect of the embodiments of the present disclosure, a medium is provided, comprising: computer-executable instructions configured to cause a processor to implement the method for identifying the illegal picture.

[0043] In a fourth aspect of the embodiments of the present disclosure, a computing device is provided, comprising: a memory and a processor; the memory stores computer-executable instructions; and the processor executes the computer-executable instructions stored in the memory, so that the processor implements the method for identifying the illegal picture.

[0044] The plurality of mutually independent supervised models and the plurality of mutually independent semi-supervised models according to the embodiments of the present disclosure can train an identification module for identifying the rule violation image, overcome the problem of weak generalization ability of a single training model, and thus significantly improve the identification accuracy of the identification module on the rule violation image, and the identification precision of the rule violation image is high. BRIEF DESCRIPTION OF DRAWINGS

[0045] The above and other objects, features and advantages of the present disclosure exemplary embodiments will be more apparent from the following detailed description read in conjunction with the accompanying drawings, in which:

[0046] FIG. 1 A training schematic diagram of an identification model according to an embodiment of the present disclosure is schematically shown;

[0047] FIG. 2 A flow schematic diagram according to an embodiment of the present disclosure is schematically shown;

[0048] FIG. 3 A flow schematic diagram according to another embodiment of the present disclosure is schematically shown;

[0049] FIG. 4 A flow schematic diagram according to still another embodiment of the present disclosure is schematically shown;

[0050] FIG. 5 A flow schematic diagram according to yet another embodiment of the present disclosure is schematically shown;

[0051] FIG. 6 A schematic diagram of a program product provided according to an embodiment of the present disclosure is schematically shown;

[0052] FIG. 7 A structural schematic diagram of an identification device of a rule violation image provided according to an embodiment of the present disclosure is schematically shown;

[0053] FIG. 8 A structural schematic diagram of a computing device provided according to an embodiment of the present disclosure is schematically shown.

[0054] In the drawings, identical or corresponding labels represent identical or corresponding parts. DETAILED DESCRIPTION

[0055] The principles and spirits of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are given only to enable those skilled in the art to better understand and implement the present disclosure, and do not limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0056] Those skilled in the art understand that the embodiments of the present disclosure can be implemented as a system, a device, an apparatus, a method or a computer program product. Therefore, the present disclosure can be embodied as a complete hardware, a complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0057] According to the embodiments of the present disclosure, a method, a medium and a computing device for illegal pictures are provided.

[0058] In this document, it is understood that the terms involved and the meanings of the terms are as follows:

[0059] Illegal pictures: refer to pictures containing uncivilized behavior, terrorism and other garbage information;

[0060] Identification model: a model for identifying illegal pictures;

[0061] Supervised model: a model trained using labeled data;

[0062] Semi-supervised model: a model trained using labeled data and unlabeled data;

[0063] First unlabeled data: is the unlabeled data in the real illegal picture identification service, including pictures;

[0064] Real illegal picture identification service: refers to a self-defined data pool, and the data in the self-defined data pool is obtained through keyword search, keyword association search, image search and video search;

[0065] Second unlabeled data: unlabeled data in a public picture resource pool;

[0066] Public picture resource pool: refers to an online random data pool, and the data in the online random data pool is obtained randomly online;

[0067] Cold start data pool: refers to a data pool containing a small amount of labeled data, which is used to start the training of the identification model;

[0068] Confidence learning: used to identify label errors and describe label noise to eliminate noise data from the data;

[0069] Self-supervised learning: using auxiliary tasks to mine supervision information from large-scale unsupervised data, and training the network through the supervision information constructed in this way, so as to learn the representation valuable to the downstream task.

[0070] Semi-supervised learning: the model performance in the supervised process is improved by using unlabeled data. Specifically, under the guidance of a small amount of sample labels, the learning performance of the model is improved by making full use of the data hidden in a large amount of unlabeled data, reducing the waste of data resources, and solving the problems of weak generalization ability of supervised learning method when there are few labeled data and inaccuracy of unsupervised learning method when there is no sample label guidance.

[0071] Active learning: the unlabeled samples with the most value and information are selected by a certain algorithm, and are marked by experts, and then the data marked by artificial marking is used for training by using a supervised learning model or a semi-supervised learning model, so as to gradually improve the effect of the model. The artificial experience is integrated into the model, so that the amount of labeled data required for model learning is effectively reduced.

[0072] In addition, the number of any elements in the drawings is used for example and not limitation, and any naming is only used for distinction and does not have any limiting meaning.

[0073] The principles and spirits of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure. SUMMARY

[0075] The present inventors found that the proportion of illegal data in actual massive data is small, and according to statistics, the illegal data accounts for only about 0.05% of the actual massive data;

[0076] The labeled data used for training the recognition model needs to be manually labeled, resulting in high labeling cost;

[0077] The precision and generalization ability of the recognition model are limited. In the actual labeling process, the training data usually has noise data due to the labeling quality. When the noise data is serious and the amount of labeled data is small, it has a great influence on the model precision and recall. If such noise samples are directly removed, the available samples for supervised learning are further reduced. In addition, when the training samples are few, the generalization ability of the recognition model will be affected because the domain range it covers is small.

[0078] A batch of independent supervised models are trained by using the labeled data in the cold start data pool, the unlabeled data is labeled by using the supervised models, the labeled data is used to train the supervised models, and after multiple iterations, a plurality of supervised models with good recognition ability for illegal pictures are obtained. On this basis, the unlabeled data provided by the supervised models is subjected to semi-supervised learning by using a plurality of semi-supervised models, to obtain labeled data as negative samples, and finally the supervised models are learned by using the labeled data of the negative samples, so as to obtain a recognition model with high recognition precision and generalization ability.

[0079] After introducing the basic principles of the present disclosure, the various non-limiting embodiments of the present disclosure will be specifically introduced below.

[0080] Overview of Application Scenarios

[0081] Reference is first made to FIG. 1 , FIG. 1 is a schematic diagram of the training process of the identification model of the embodiment of the present disclosure, including multiple stages such as multi-modal fusion data starting and data iteration process, and the specific process is as follows:

[0082] 1. Use self-supervised learning to mine supervision information from large-scale unsupervised data, train the basic model through the supervision information and label data, and then set the network parameters of the trained basic model as the initial training parameters of multiple models.

[0083] 2. Train the multiple training models with the initial training parameters through a small amount of labeled data, that is, multiple supervised models perform supervised learning to obtain multiple supervised models with prediction ability. The labeled data is cold start data, and the cold start data is a small amount of multi-modal fusion of artificial labeled data. The sources of the labeled data include OCR (Optical Character Recognition), image search, keyword crawler, and video stream. The multi-modal fusion of the cold start data refers to the combination of the labeled data obtained from the search of text, picture, and video at multiple levels.

[0084] 3. Use the multiple supervised models with prediction ability to predict the unlabeled data, and obtain supervised data and semi-supervised data based on the fusion system of multiple supervised models. The unlabeled data is derived from OCR, image search, keyword crawler, video stream, and online randomly obtained data. The fusion system refers to a mechanism system that combines the prediction results of multiple supervised models on the unlabeled data. Part of the supervised data is derived from the labeling of the supervised model and manual labeling. The multiple supervised models can identify the illegal pictures in the unlabeled data, use the data cleaning device to clean the illegal pictures, input the cleaned illegal pictures into the supervised model, and the supervised model can label the cleaned illegal pictures. Multiple supervised models can identify difficult-to-classify unlabeled data through active learning. These unlabeled data obtain supervised data through manual labeling. Multiple supervised models can identify valuable unlabeled data from the unlabeled data as semi-supervised data. The valuable unlabeled data refers to the unlabeled data that can improve the generalization ability of the model, and the valuable unlabeled data is, for example, a picture that is difficult to determine as illegal.

[0085] 4. Multiple supervised models can be retrained using supervised data. These retrained models then acquire supervised data from unlabeled data and undergo another round of training using this newly acquired supervised data. This means multiple rounds of training are required until all supervised models demonstrate a good ability to recognize inappropriate images. After each supervised model completes multiple rounds of training, it selects a portion of the unlabeled data as semi-supervised data. One or more semi-supervised models then perform semi-supervised learning using both supervised and semi-supervised data to fine-tune the multiple supervised models, thereby obtaining the final recognition model.

[0086] Exemplary Method

[0087] The following is combined FIG. 1 Application scenarios, refer to FIGS. 2-5 This document describes a method for identifying illegal images according to exemplary embodiments of the present disclosure. It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in any way. Rather, the embodiments of the present disclosure can be applied to any applicable scenario.

[0088] For example, the execution device in the embodiments of this disclosure may be a terminal with data processing capabilities.

[0089] refer to FIG. 2 , FIG. 2 An exemplary flowchart of an embodiment of the illegal image recognition method provided in this disclosure is shown. The illegal image recognition method includes:

[0090] Step S201: Input the first image to be identified into the recognition model to determine the compliance of the first image based on the output of the recognition model; wherein, the recognition model includes multiple independent supervised models and multiple independent semi-supervised models; the recognition model is obtained by training multiple supervised models and multiple semi-supervised models.

[0091] In this embodiment, the terminal is provided with an identification model, which includes a plurality of mutually independent supervised models and a plurality of mutually independent semi-supervised models. The plurality of mutually independent supervised models refer to supervised models with different network structures, and the plurality of mutually independent semi-supervised models refer to semi-supervised models with different network structures. The supervised model refers to a model obtained through supervised learning of labeled data, and the semi-supervised model refers to a model obtained through semi-supervised learning of labeled data and unlabeled data. It should be noted that the labeled data refers to picture data with labels, and the unlabeled data refers to picture data without labels. The label is used to determine the compliance of the picture, that is, to indicate the violation of the picture data. In this embodiment, the violation indicated by the label includes: violation, specific category of violation, or no violation.

[0092] The terminal obtains a first picture, which is a picture currently requiring identification of a violation. The first picture can be uploaded by a user. The terminal inputs the first picture to be identified into the identification model to obtain an output result of the identification model. The terminal can determine the compliance of the first picture based on the output result. Specifically, the output result includes three results of violation, specific category of violation, and no violation. If the output result is violation or specific category of violation, the first picture is a violation picture. If the output result is no violation, the first picture is a compliant picture.

[0093] The identification model is trained by the plurality of supervised models and the plurality of semi-supervised models. Each of the supervised models and the semi-supervised models has identification capability for the first picture, and the terminal can obtain a plurality of identification results. The terminal can vote based on the plurality of identification results of each supervised model and each semi-supervised model, and can obtain a final output result by synthesizing the votes of each supervised model and each semi-supervised model.

[0094] In this embodiment, the plurality of mutually independent supervised models and the plurality of mutually independent semi-supervised models can train an identification model for identifying violation pictures, overcome the problem of weak generalization ability of a single training model, and significantly improve the identification accuracy of the identification model for violation pictures, thereby improving the identification precision of the violation pictures.

[0095] Reference FIG. 3 , FIG. 3 This is a flowchart of another embodiment of the violation picture identification method of the present disclosure, based on the embodiment shown in FIG. 2 Before step S201, the embodiment further includes:

[0096] In step S202, the plurality of supervised models and the plurality of semi-supervised models are trained by sample data in a sample data pool to obtain a recognition model; wherein the sample data includes first unlabeled data appearing in a real violation picture recognition service and second unlabeled data collected from a public picture resource pool.

[0097] In the embodiment, the plurality of supervised models and the plurality of semi-supervised models are trained by sample data in a sample data pool. The sample data includes first unlabeled data and second unlabeled data. The first unlabeled data is data appearing in a real violation picture recognition service. The second unlabeled data is data collected from a public picture resource pool. The data in the public picture resource pool is randomly obtained online, and the second unlabeled data in the public picture resource pool is updated regularly.

[0098] The first unlabeled data is data obtained by keyword search, keyword association search, and picture search from text, pictures, and videos. Most of the first unlabeled data is violation pictures, and only a small amount of the first unlabeled data is non-violation pictures. The non-violation pictures are caused by inaccurate search. In addition, there are more difficult example data in the first unlabeled data. The difficult example data refers to data that is difficult for the supervised model to classify, that is, the supervised model is difficult to identify whether the picture is a compliance picture. Through learning of the difficult example data, the recognition ability of the supervised model can be improved.

[0099] The second unlabeled data in the public picture resource pool is randomly obtained, and the second unlabeled data is obtained multiple times. Therefore, the second unlabeled data is massive. The second unlabeled data is mainly used to enable the recognition model to recognize more online data, and the learning domain of the recognition model can match the distribution range of the online data, that is, the recognition range of the recognition model is improved.

[0100] To ensure the balance between speed and accuracy, when training the supervised model, the picture input into the supervised model is scaled in size. The specific scaling scale is determined according to actual task requirements, and the default is usually 224 pixels x 224 pixels. Before the picture is input into the network of the supervised model, the data needs to be normalized to ensure consistent data distribution in the model training process. In this regard, all sample data in the sample data pool can be preprocessed, and the image preprocessing includes scaling operation and data normalization, so that the sample data in the sample data pool is directly used to train the supervised model and the semi-supervised model.

[0101] In the embodiment, the plurality of supervised models and the plurality of semi-supervised models are trained by sample data in a sample data pool. The sample data includes first unlabeled data and second unlabeled data. The first unlabeled data is data appearing in a real violation picture recognition service. The second unlabeled data is collected from a public picture resource pool. The data in the public picture resource pool is randomly obtained online, and the second unlabeled data in the public picture resource pool is updated regularly.

[0102] Referring to FIG. 4 , FIG. 4 is a flowchart of another embodiment of the method for identifying a rule-violating picture of the present disclosure, based on FIG. 3 As shown in the embodiment, before step S202, the method further includes:

[0103] In step S203, the plurality of supervised models and the plurality of semi-supervised models are pre-trained by using the sample data in the cold start data pool, so that the plurality of supervised models and the plurality of semi-supervised models have label prediction capability; wherein the sample data in the cold start data pool is the first labeled data artificially labeled, and the sample quantity of the first labeled data is less than a preset quantity.

[0104] In the embodiment, the cold start data pool includes a plurality of sample data, and the sample data in the cold start data pool is the first labeled data artificially labeled. The first labeled data is a small amount, that is, the sample quantity of the first labeled data is less than a preset quantity, and the preset quantity is a small value, for example, the preset quantity is 500.

[0105] The first labeled data is used to train the plurality of supervised models and the plurality of semi-supervised models, so that the plurality of supervised models and the plurality of semi-supervised models have label prediction capability. The label prediction capability refers to that the supervised models and the semi-supervised models make a prediction on the label category to which the unlabeled data belongs.

[0106] In an example, the supervised models and the semi-supervised models can set default initial training parameters, and then use the first labeled data for training.

[0107] In another example, the initial training parameters of the supervised models and the semi-supervised models are set to the training parameters obtained by self-supervised learning. The self-supervised learning is to obtain supervised information from large-scale unsupervised data by using auxiliary tasks, and the auxiliary tasks include image selection, image coloring, etc., and the supervised information refers to the relationship between an image and an image processed by an auxiliary task, for example, image A is rotated by 90 degrees to obtain image B, and the relationship between the two images A and B is that one is not rotated and the other is rotated by 90 degrees, and the supervised information is that image A is not rotated and image B is rotated by 90 degrees relative to image A. Based on the supervised information of the self-supervised learning, the supervised models and the semi-supervised models can predict how many degrees the two images are rotated. Training the network of the supervised models and the semi-supervised models by using the supervised information can make the supervised models and the semi-supervised models learn valuable data.

[0108] In yet another example, the base model can be trained by data in real violation business, and the base model is self-supervised learned by supervision information, and the training parameters of the base model are used as initial training parameters of the supervised model and the semi-supervised model after learning, so as to further improve the recognition accuracy and generalization ability of the supervised model and the semi-supervised model. Wherein, the training of the base model by data in real violation business includes the following steps:

[0109] 1. Process the third sample picture in the real violation business data to obtain a fourth sample picture and auxiliary information (the auxiliary information is the supervision information described above), and the corresponding processing mode includes rotation, scaling and / or coloring. The auxiliary information is the relationship between the third sample picture and the fourth sample picture.

[0110] 2. Train the base model by the third sample picture, the fourth sample picture and the auxiliary information.

[0111] In this embodiment, a plurality of supervised models and semi-supervised models with label prediction ability are trained by a small amount of first labeled data marked by artificial labeling, so that the plurality of semi-supervised models and the plurality of supervised models learn based on the label prediction ability obtained by training, and lay a good foundation for the training of the recognition model with high recognition accuracy.

[0112] Referring to FIG. 5 , FIG. 5 The flowchart of another embodiment of the violation picture recognition method of the present disclosure is based on FIG. 4 As shown in the embodiment of FIG. 2, step S203 includes:

[0113] Step S501, add labels to the first unlabeled data.

[0114] In this embodiment, the first unlabeled data is labeled during the training of the plurality of supervised models, that is, labels are added to the first unlabeled data. Most of the first unlabeled data is violation pictures, and the labels added to the first unlabeled data are used to represent that the first unlabeled data is a violation picture.

[0115] In one example, the step of adding labels to the first unlabeled data is as follows:

[0116] 1. Input the first unlabeled data into the plurality of supervised models to obtain a first probability prediction value of the violation category to which each supervised model should belong to the first sample picture in the first unlabeled data.

[0117] The network structures of the plurality of supervised models are different, and the greater the difference in network structure is, the higher the recognition accuracy of the final recognition model will be. The supervised models are trained by the first labeled data, and the label prediction capability is obtained. The sample picture in the first unlabeled data is defined as the first sample picture, and the probability prediction value is defined as the first prediction probability value. The violation categories are, for example, pictures of bloodshed, pictures of guns, and the like. For example, the prediction result of the supervised model A for the sample picture a is that the probability prediction value of the bloodshed picture is 95%, and the prediction result of the supervised model B for the sample picture a is that the probability prediction value of the gun picture is 90%.

[0118] 2. In response to the first probability prediction value of the violation category to which the first sample picture belongs being greater than the preset first probability threshold value, the first sample picture is added with the first violation category label determined by the supervised model.

[0119] The first probability threshold value is set in each supervised model. If the first probability prediction value of the first sample picture by the supervised model A is greater than the first probability threshold value set by the supervised model A, the first sample picture is considered to be the first unlabeled picture by the supervised model A and should be labeled. The network structures of the supervised models are different, and thus the first probability threshold values set by different supervised models are different.

[0120] The terminal determines the first sample pictures to be labeled, and the first probability prediction value of the violation category to which the first sample picture belongs is greater than the first probability threshold value. The terminal adds the first violation category label determined by the supervised model to the sample picture. For example, the prediction result of the supervised model A for the first sample picture a is that the first probability prediction value of the bloodshed picture is 95%, the first probability prediction value of the gun picture is 80%, and the first probability threshold value set by the supervised model A is 90%, and thus the supervised model A adds the bloodshed label to the first sample picture a.

[0121] 3. The first prediction label of the first sample picture is determined from the first violation category labels determined by the supervised models based on a voting mechanism, and the first prediction label is added to the first unlabeled data to which the first sample picture belongs, and the second labeled data is obtained.

[0122] Different supervised models add different first violation category labels to the first sample picture, and thus the first prediction label of the first sample picture can be determined from the first violation category labels of the first sample picture based on the voting mechanism. For example, there are five supervised models, the supervised models A, B, and C add the bloodshed label to the first sample picture, and the supervised models D and E add the gun label to the first sample picture. Based on the principle of minority submission to majority of the voting mechanism, the first prediction label is the bloodshed label. The terminal adds the first prediction label to the first unlabeled data to which the first sample picture belongs, and thus the second labeled data is obtained.

[0123] In another example, adding labels to the first unlabeled data includes the following process:

[0124] 1. inputting the first unlabeled data into a plurality of supervised models to obtain first probability prediction values of each supervised model for a first sample picture in the first unlabeled data.

[0125] Each supervised model predicts the first unlabeled data to obtain a first probability prediction value of each supervised model. Each supervised model is provided with a different first probability threshold value.

[0126] 2. determining a first number of first probability prediction values greater than the first probability threshold value set by the supervised model.

[0127] There are multiple supervised models, and each supervised model predicts the first sample picture, so that the first sample picture has multiple first probability prediction values. The terminal determines the number of first probability prediction values greater than the first probability threshold value. For example, the first probability threshold value set by the supervised model A is 90%, the first probability threshold value set by the supervised model B is 85%, the first probability threshold value set by the supervised model C is 92%, the first probability threshold value set by the supervised model D is 87%, and the first probability threshold value set by the supervised model E is 98%; the first probability prediction value of the supervised model A for the first sample picture is 95%, the first probability prediction value of the supervised model B for the first sample picture is 92%, the first probability prediction value of the supervised model C for the first sample picture is 88%, the first probability prediction value of the supervised model D for the first sample picture is 93%, and the first probability prediction value of the supervised model E for the first sample picture is 99%, then the first probability prediction value corresponding to the supervised model A is 95% which is greater than the first probability threshold value 90%, the first probability prediction value corresponding to the supervised model B is 92% which is greater than the first probability threshold value 85%, the first probability prediction value corresponding to the supervised model C is 88% which is less than the first probability threshold value 92%, the first probability prediction value corresponding to the supervised model D is 93% which is greater than the first probability threshold value 87%, and the first probability prediction value corresponding to the supervised model E is 99% which is greater than the first probability threshold value 98%, then the first number is 4.

[0128] 3. determining a first ratio between the first number and the total number of supervised models.

[0129] 4. in response to the first ratio being greater than a first preset ratio, adding a first predicted label to the first unlabeled data to which the first sample picture belongs to obtain second labeled data.

[0130] The terminal calculates a first ratio between the first ratio and a total number of the supervision models. If the first ratio is greater than a first preset ratio, the first sample picture is a rule violation picture. The terminal then adds a first prediction label to the first no-label data to which the first sample picture belongs to obtain second labeled data. The first prediction label refers to a label of the rule violation picture. The first preset ratio can be any suitable value, for example, the first preset ratio is 0.5.

[0131] Step S502, adding a label to the second no-label data.

[0132] After adding the label to the first no-label data, in order to improve the recognition ability of the recognition model on online data, the second no-label data needs to be labeled. The labeling of the second no-label data can include: adding a label of a positive sample to the first part of the second no-label data and adding a label of a negative sample to the second part of the plurality of second no-label data. In the embodiment, each second no-label data that needs to add a label of a positive sample is defined as the first part of the second no-label data, and the remaining second no-label data in the public picture resource pool is defined as the second part of the no-label data.

[0133] It should be noted that after the first no-label data is labeled, the plurality of supervision models are trained by the labeled first no-label data, and the labeled first no-label data is a positive sample, so that the plurality of supervision models have the recognition ability of the positive sample data after training, but lack the recognition ability of the negative sample data. Thus, the label of the positive sample of the second no-label data is added by the plurality of supervision models, and the label of the negative sample of the second no-label data is added by the plurality of semi-supervised models.

[0134] In the embodiment, by adding labels to the first no-label data and the second no-label data, a large amount of sample data is provided for training of the recognition model with high recognition accuracy.

[0135] In an embodiment, adding the label of the positive sample to the first part of the second no-label data includes:

[0136] 1. inputting the second no-label data into the plurality of supervision models to obtain a second probability prediction value of a rule violation category to which a second sample picture in the second no-label data belongs.

[0137] In the embodiment, the number of the second no-label data is massive, and there are a large number of rule violation pictures. The rule violation pictures as sample data can be obtained from the second no-label data.

[0138] The supervised model is trained by the first labeled data to obtain the label prediction ability. The unlabeled data is input into the multiple supervised models to obtain a second probability value of each supervised model for a second sample picture belonging to a second rule category in the second unlabeled data. The rule category is, for example, a blood picture, a gun picture, and the like. For example, the prediction result of the supervised model A for the sample picture A is that the probability prediction value of the blood picture is 95%, and the prediction result of the supervised model B for the sample picture A is that the probability prediction value of the gun picture is 90%.

[0139] 2. In response to the second probability prediction value of the supervised model for the rule category to which the second sample picture belongs being greater than the second probability threshold value and less than the third probability threshold value, a second rule category label determined by the supervised model is added to the second sample picture.

[0140] Since the online data is continuously generated, there are many simple examples (i.e., the prediction score of the data by the supervised model is high) in the random data (the second unlabeled data) on the line. The gain of the model performance improvement of this part of simple examples is almost 0, so this part of data (high score) needs to be removed, so two threshold values need to be selected. For this purpose, the second probability threshold value and the third probability threshold value are set in each supervised model, and the third probability threshold value is greater than the second probability threshold value. Different supervised models have different second probability threshold values and third probability threshold values.

[0141] In the self-defined data (in the first unlabeled data), since it is a targeted mining, the obtained data is relatively difficult to classify, and the amount of mining of this part of data is also not much, so this part of data only needs to set one threshold value, that is, for the labeling of the first unlabeled data, the supervised model only needs to set the first probability threshold value.

[0142] If the second probability prediction value of the supervised model A for the second sample picture is greater than the second probability threshold value set by the supervised model A and less than the third probability threshold value, the supervised model A considers that the second unlabeled picture to which the second sample picture belongs should be labeled.

[0143] The terminal determines each second sample picture to be labeled, and a second probability prediction value of a violation category to which the second sample picture to be labeled belongs is greater than a second probability threshold and less than a third probability threshold. For example, the second probability threshold of the violation category of a gun set by the supervision model is 90%, the third probability threshold is 95%, and if a probability of a picture being a gun picture predicted by the supervision model is 92%, the probability 92% is greater than the second probability threshold 90% and less than the third probability threshold 95%, the picture is the second sample picture to be labeled. The terminal adds a second violation category label judged by the supervision model to the sample picture. For example, the prediction result of the second sample picture A by the supervision model A is that the second probability prediction value of the bloody picture is 95%, and the second probability threshold set by the supervision model A is 90% and the third probability threshold is 98%, and the supervision model A adds a bloody label to the second sample picture A.

[0144] 3. A second prediction label of the corresponding sample picture is determined from the second violation category labels judged by the supervision models based on a voting mechanism, and a second prediction label is added to the second unlabeled data to which the second sample picture belongs, and the second preset label is a label of a positive sample. The picture of the positive sample label refers to a violation picture, and the picture of the negative sample label refers to a compliance picture.

[0145] Different supervision models add different second violation category labels to the second sample picture, and the second prediction label of the second sample picture can be determined from each second violation category label of the second sample picture based on the voting mechanism. For example, there are five supervision models, the supervision models A, B and C add a bloody label to the second sample picture, and the supervision models D and E add a gun label to the second sample picture, and the second prediction label is the bloody label based on the principle of minority submission to majority of the voting mechanism. The terminal adds the second prediction label to the second unlabeled data to which the second sample picture belongs, and the second labeled data is obtained. In this embodiment, the second prediction label is a label of a positive sample.

[0146] In this embodiment, the terminal labels the second unlabeled data through the plurality of supervision models, and obtains sample data of the positive sample required for training of the recognition model.

[0147] In an embodiment, adding the label of the positive sample to the first part of the second unlabeled data comprises:

[0148] 1. The second unlabeled data is input into the plurality of supervision models to obtain a second probability prediction value of each supervision model to the second sample picture in the second unlabeled data.

[0149] Each supervision model predicts the second unlabeled data to obtain a second probability prediction value of each supervision model. Each supervision model sets a different second probability threshold and a different third probability value.

[0150] 2. Determine a second number of second probability thresholds that are greater than the second probability threshold set by the supervised model and less than the third probability threshold.

[0151] There are multiple supervised models, each predicting the second sample image, resulting in multiple predicted second probability values ​​for the second sample image. The terminal determines the number of predicted second probability values ​​that are greater than a second probability threshold and less than a third probability threshold. For example, supervised model A sets its second and third probability thresholds to 90% and 95%, respectively; supervised model B sets them to 85% and 90%, respectively; supervised model C sets them to 92% and 96%, respectively; supervised model D sets them to 87% and 95%, respectively; and supervised model E sets them to 95% and 98%, respectively. The predicted second probability value for the second sample image by supervised model A is 93%, by supervised model B it is 92%, and by supervised model C it is 88%. If the second probability prediction value of supervised model D for the second sample image is 93%, and the second probability prediction value of supervised model E for the second sample image is 99%, then the first probability prediction value of supervised model A (93%) is greater than the second probability threshold of 90% and less than the third probability threshold of 95%. The second probability prediction value of supervised model B (92%) is greater than the third probability threshold of 90%. The second probability prediction value of supervised model C (88%) is less than the second probability threshold of 92%. The second probability prediction value of supervised model D (93%) is greater than the second probability threshold of 87% and less than the third probability threshold of 95%. The second probability prediction value of supervised model E (99%) is greater than the third probability threshold of 98%. Therefore, the number of second probability prediction values ​​is 3.

[0152] 3. Determine the second ratio between the second quantity and the total quantity of the supervised model.

[0153] 4. In response to the second ratio being greater than the second preset ratio, add the first predicted label to the second unlabeled data to which the second sample image belongs, and obtain the second labeled data.

[0154] The terminal calculates a second ratio between the second ratio and the total number of supervised models. If the second ratio is greater than a second preset ratio, the second sample image is a violation image. The terminal then assigns a first predicted label to the second unlabeled data to which the second sample image belongs, thus obtaining the second labeled data. The first predicted label refers to the label of the violation image.

[0155] In this embodiment, the terminal labels the second unlabeled data using multiple supervised models to obtain sample data of positive samples required for training the recognition model.

[0156] In an embodiment, the labels of the negative samples are added to the second part of the plurality of second unlabeled data, comprising:

[0157] The plurality of supervised models are retrained according to the respective second labeled data.

[0158] In the embodiment, after the labels are added to the first unlabeled data and the labels of the positive samples are added to the first part of the second unlabeled data, the second labeled data is obtained, the supervised models are retrained by the second labeled data, and the respective supervised models with improved generalization ability are obtained.

[0159] The terminal is provided with a stop training condition, for example, the number of training times of the plurality of supervised models reaches a set number, or the training parameters of the respective supervised models no longer continue to converge.

[0160] It should be noted that after the plurality of supervised models are labeled on the first unlabeled data in each round, it is necessary to determine whether there is an error in the identification of online random data by the respective supervised models, that is, the supervised models are used to identify part of the second unlabeled data, and if the number of correctly identified second unlabeled data is greater than a set number, the plurality of supervised models use the labeled first unlabeled data for the next round of training. If the number of correctly identified second unlabeled data is less than or equal to the set number, the plurality of supervised models need to add labels to the second unlabeled data, and in the next round of training, the labeled first unlabeled data and the second unlabeled data are used for training, so as to make up for the problem of low identification accuracy of online data by the supervised models. In addition, after the last but one round of supervised models are labeled on the first unlabeled data, the supervised models need to add labels to the second unlabeled data, and finally the labeled first unlabeled data and the second unlabeled data are used for training to obtain the final supervised models. It can be understood that in each round of training before the last but one round of training, if the supervised models have the problem of low identification accuracy of online data, the second unlabeled data needs to be labeled for the next round of training of the supervised models.

[0161] In response to the plurality of supervised models satisfying the stop training condition, labels of negative samples are added to part of the second unlabeled data.

[0162] When the plurality of supervised models satisfy the stop training condition, the labels of the negative samples are added to the second part of the plurality of second unlabeled data. In response to the plurality of supervised models not satisfying the stop training condition, the step of adding labels to the first unlabeled data is returned, that is, the plurality of supervised models are trained for a new round.

[0163] Specifically, the process of adding labels of negative samples to the second unlabeled data is as follows:

[0164] 1. input the second unlabeled data into the plurality of supervised models to obtain third unlabeled data with misjudgment.

[0165] The supervised model labels the first unlabeled data, mainly to complete the selection of positive samples. However, negative samples are needed in the training process of the supervised model. However, the first unlabeled data is obtained based on the features of the violation pictures. The pictures obtained in a targeted manner may not be accurate, and there may be compliant pictures, but the first unlabeled data is still dominated by positive samples (violation pictures are positive samples). Therefore, the number of negative samples that can be mined from each first unlabeled data is small. If the supervised model directly mines negative samples from the public resource data pool, the supervised model may misjudge a certain type of picture, and the supervised model has a high misjudgment score for a certain type, and the supervised model is difficult to directly obtain a large number of negative samples from each second unlabeled data by threshold. To this end, in this embodiment, through semi-supervised learning of the semi-supervised model, the misjudgment samples of this type are learned, so that the semi-supervised model "mistakes" to learn the misjudgment ability of the samples of this type. The semi-supervised model selects the misjudgment samples from the second unlabeled data as negative samples through the learned misjudgment ability, that is, the semi-supervised model can obtain a large number of negative samples from the vast amount of second unlabeled data, reducing the difficulty of obtaining a sufficient number of negative samples, so that the supervised model can be trained by a sufficient number of negative samples, thereby improving the recognition accuracy of the recognition model obtained by the supervised model.

[0166] After the supervised learning, that is, the iteration training of the plurality of supervised models is completed, each supervised model has the ability to improve the recognition accuracy of violation pictures. However, the supervised model still has the problem of recognition error, that is, the situation that the supervised model will confirm the pictures that are not violations as violation pictures. The terminal inputs the plurality of second unlabeled data into the plurality of supervised models to obtain the second unlabeled data with recognition results, and manually reviews each second unlabeled data with recognition results to obtain second unlabeled data with inaccurate recognition, that is, third unlabeled data with recognition error. The third unlabeled data is valuable unlabeled data. For example, the supervised model identifies a hot pot picture (non-violation picture) as a roulette picture (violation picture).

[0167] 2. each semi-supervised model is used to perform semi-supervised learning on each third unlabeled data and second labeled data to obtain a semi-supervised model with misjudgment ability

[0168] The terminal performs semi-supervised learning on the third unlabeled data and the second labeled data by using each semi-supervised model, so that the semi-supervised model learns the misjudgment ability. For example, the third unlabeled data is a hot pot picture itself, and the third unlabeled data is identified as a roulette picture by the supervised model, and the misjudgment ability learned by the semi-supervised model refers to the ability to misjudge the hot pot picture as a roulette picture.

[0169] It should be noted that the semi-supervised model can perform multiple semi-supervised learning, so that the semi-supervised model obtains very accurate misjudgment ability. For example, 98% of hot pot pictures can be misjudged as roulette pictures.

[0170] 3. Input the second unlabeled data into the semi-supervised model that misjudges the illegal category to which the third unlabeled data belongs, and obtain a third probability prediction value of the semi-supervised model for the illegal category to which the second sample picture in the second unlabeled data that misjudges appears.

[0171] After the semi-supervised model learns the misjudgment ability, the second unlabeled data that misjudges is obtained by inputting the second unlabeled data into the semi-supervised model, and the third probability prediction value of the second unlabeled data that misjudges is obtained.

[0172] 4. Add a label of a negative sample to the second unlabeled data with a third probability prediction value greater than a preset fourth probability threshold.

[0173] The semi-supervised model is provided with a fourth probability threshold, and the fourth probability threshold is large, for example, the fourth probability threshold is 99%. The semi-supervised model obtains a third probability prediction value of the second unlabeled data by predicting the second unlabeled data, and if the third probability prediction value is greater than the preset fourth probability threshold, the semi-supervised model adds a label of a negative sample to the second unlabeled data.

[0174] The terminal trains a plurality of supervised models by using the first unlabeled data with the added label, the second unlabeled data with the added label of a positive sample, and the second unlabeled data with the added label of a negative sample to obtain a recognition model.

[0175] It can be understood that the recognition error of a certain type of picture by the supervised model enables the semi-supervised model to learn the misjudgment ability for this type of picture, and the semi-supervised misjudgment ability and the high fourth probability threshold are used to screen this type of picture from a large amount of second unlabeled data as a negative sample, and the supervised model is trained by using the data of the negative sample and the second labeled data, so as to eliminate the misjudgment of the supervised model for this type of picture, and improve the recognition accuracy of the supervised model for the picture. This strategy is a make mistakes strategy.

[0176] In an embodiment, retraining a plurality of supervised models according to each second labeled data comprises:

[0177] The noise data in the second labeled data is filtered by the confident learning algorithm, and the plurality of supervised models are retrained based on the remaining second labeled data after filtering.

[0178] In this embodiment, after the plurality of supervised models obtain the second labeled data, it is necessary to remove the noise of the second labeled data, that is, to remove the second labeled data with recognition errors.

[0179] The terminal filters the noise data in the second labeled data by the confident learning algorithm, and re-trains the plurality of supervised models based on the remaining second labeled data after filtering, so as to obtain the supervised models with improved recognition accuracy.

[0180] Specifically, the noise data in the second labeled data is filtered by the confident learning algorithm, and the plurality of supervised models are retrained based on the remaining second labeled data after filtering, including:

[0181] 1. Determine the noise label and the real label in the second labeled data, and determine the noise data in the second labeled data based on the joint distribution between the noise label and the real label. The noise data is the sample data incorrectly labeled by the supervised model.

[0182] The terminal determines the noise label and the real label in the second labeled data, and there are multiple noise labels in one second labeled data. The terminal determines the difference value between each noise label and the real label, and the difference values are plotted in the coordinate system to form a joint distribution. If most of the difference values are located in a small area (area in the coordinate system), the second labeled data is not noise data. If each difference value is scattered in the coordinate system, rather than concentrated in a small area, the second labeled data is noise data.

[0183] In an example, the terminal can divide the multiple second labeled data into K parts, and first predict K-1 parts and 1 part of the second labeled data respectively, so that each second labeled data obtains a noise label; and then predict K-2 parts and 2 parts of the second labeled data respectively, so that each second labeled data obtains a noise label; in this way, until 1 part and K-1 parts of the second labeled data are predicted respectively, each second labeled data obtains a last noise label. The above noise label is obtained only for the convenience of description, and in the actual process, the K-1 parts and 1 part, the K-2 parts and 2 parts, and the 1 part and K-1 part of the second labeled data are predicted simultaneously. For example, the multiple second labeled data are divided into three parts of second labeled data with serial numbers 1, 2 and 3, the second labeled data with serial numbers 1 and 2 are predicted, and the second labeled data with serial number 3 is predicted; the second labeled data with serial numbers 2 and 3 are predicted, and the second labeled data with serial number 1 is predicted.

[0184] 2. Filtering noise data in the second labeled data, and retraining the multiple supervised models based on the remaining second labeled data.

[0185] The terminal filters the noise data, and can retrain the multiple supervised models based on the remaining second labeled data.

[0186] It should be noted that the terminal can extract part of the second labeled data from all the second labeled data, identify the extracted second labeled data, and if the ratio of the number of correctly identified second labeled data to the extracted second labeled data is greater than a set ratio, it is determined that the identification accuracy of the second labeled data of the supervised model is higher than a set accuracy, and confidence learning is not required; if the ratio is less than or equal to the set ratio, it is determined that the identification accuracy of the second labeled data of the supervised model is lower than the set accuracy, and confidence learning is required.

[0187] In the embodiment, the noise data is removed from the second labeled data through confidence learning, so as to improve the identification accuracy of the supervised model, and further improve the identification accuracy of the identification model for the illegal pictures.

[0188] In an embodiment, after the step of adding the label of the negative sample to the second unlabeled data with the third probability prediction value greater than the fourth probability threshold, the method further includes:

[0189] 1. Training the multiple supervised models by using the first unlabeled data with the added label, the second unlabeled data with the added label of the positive sample, and the second unlabeled data with the added label of the negative sample.

[0190] In the embodiment, after obtaining the second unlabeled data with added negative sample labels, the terminal trains each supervised model through each second labeled data and the second unlabeled data with added negative sample labels, so as to eliminate the misjudgment of the supervised model on a certain type of picture.

[0191] 2. The second unlabeled data is input into the trained multiple supervised models, and the fifth probability prediction value of the rule category to which each second sample picture in the second unlabeled data belongs is obtained.

[0192] The remaining second unlabeled data in the public picture resource pool is difficult to classify after being mined by the multiple supervised models and the multiple semi-supervised models. At this time, the idea of active learning can be used to obtain the second unlabeled data that is difficult to classify by the model for manual labeling, and then the second unlabeled data labeled manually is used for learning to further improve the recognition accuracy and generalization ability of the model.

[0193] To this end, the remaining second unlabeled data in the public picture resource pool is input into the trained multiple supervised models, and the supervised model at this time refers to the supervised model after eliminating the misjudgment of one or more types of pictures. The terminal can obtain the fifth probability prediction value of the rule category to which each second sample in the second unlabeled data belongs.

[0194] 3. The second unlabeled data with the fifth probability prediction value located in the preset probability interval is obtained for manual labeling of the label.

[0195] The preset probability interval is set in the terminal, and the second unlabeled data with the fifth probability prediction value located in the preset probability interval is difficult to classify. The preset probability interval can be any suitable interval, for example, the preset probability interval can be [0.3, 0.8]. These screened second unlabeled data can be manually labeled.

[0196] 4. The trained multiple supervised models are retrained by the first unlabeled data with added labels, the second unlabeled data with added positive sample labels, the second unlabeled data with added negative sample labels, and the second unlabeled data with manual labels, to obtain a recognition model.

[0197] The terminal re-trains the trained multiple supervised models by adding the first unlabeled data with added labels, the second unlabeled data with added positive sample labels, the second unlabeled data with added negative sample labels, and the second unlabeled data with manual labels, to obtain a recognition model with high recognition accuracy.

[0198] Based on the above embodiment, the method for identifying rule violation pictures provided by the present disclosure has the following beneficial effects:

[0199] 1. Multiple independent supervised models and multiple independent semi-supervised models can be used to train a recognition module for identifying illegal images, overcoming the problem of weak generalization ability of a single training model, thereby significantly improving the recognition accuracy of the recognition model for illegal images, and the recognition accuracy of illegal images is high.

[0200] 2. Training the supervised model with the first unlabeled data obtained through targeted acquisition can improve the recognition model's ability to identify illegal images, and training the supervised model with the second unlabeled data obtained through random acquisition can improve the recognition range of the recognition model.

[0201] 3. By pre-training multiple semi-supervised and supervised models with a small amount of manually labeled data, these models can be trained to have a certain ability to predict labels on unlabeled data. This allows for the training of a recognition model with good recognition capabilities. In other words, a recognition model with good recognition capabilities can be trained with a small amount of manually labeled data, reducing the manual labeling cost of the recognition model.

[0202] 4. The initial training parameters of multiple supervised models and multiple semi-supervised models are the parameters after training the base model. The base model is trained with data from real-world violation business scenarios, which improves the recognition accuracy of multiple supervised models and multiple semi-supervised models for violation images in real-world violation business scenarios.

[0203] 5. After obtaining labeled data from multiple supervised models, confidence learning is used to remove misidentified data from the labeled data, i.e., to remove noisy data. Then, the labeled data after removing noisy data is used to train multiple supervised models, thereby obtaining supervised models with higher recognition accuracy.

[0204] 6. After multiple supervised models have good recognition capabilities, negative samples are provided to the supervised models through semi-supervised learning of multiple semi-supervised models. This enables multiple supervised models to eliminate items in one or more classes of images based on negative samples, thereby improving the recognition accuracy and generalization ability of the recognition models trained by supervised and semi-supervised models.

[0205] 7. After the supervised model is trained based on negative samples, the trained supervised model obtains unlabeled data that is difficult to identify from the remaining second unlabeled data and manually labels it. Then, the manually labeled unlabeled data is used to train the supervised model, which improves the recognition model's ability to identify images that are difficult to classify.

[0206] Exemplary Medium

[0207] After introducing the methods of exemplary embodiments of this disclosure, the following references are made. FIG. 6A storage medium of an exemplary embodiment of the present disclosure is described.

[0208] Reference is made to FIG. 6 As shown, the storage medium 60 stores therein a program product for implementing the above-described method according to an embodiment of the present disclosure, which can take a portable compact disc read-only memory (CD-ROM) and include computer-executable instructions for causing a computing device to perform the method of identifying a violation picture provided by the present disclosure. However, the program product of the present disclosure is not limited thereto.

[0209] The program product can take any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0210] The readable signal medium can include a data signal carried in a baseband or as part of a carrier wave, in which computer-executable instructions are carried. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The readable signal medium can also be any readable medium other than the readable storage medium.

[0211] The computer-executable instructions for carrying out the operations of the present disclosure can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-executable instructions can execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of the latter, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN).

[0212] Exemplary Apparatus

[0213] After introducing the medium of the exemplary embodiment of the present disclosure, next, reference is made to FIG. 7An apparatus for identifying a non-compliant picture according to an exemplary embodiment of the present disclosure is described, which is used to implement the method in any of the method embodiments described above, and has similar implementation principles and technical effects, which will not be described here.

[0214] Reference FIG. 7 , FIG. 7 A structural schematic diagram of an apparatus for identifying a non-compliant picture according to an embodiment of the present disclosure is schematically shown. As shown in FIG. 7 The apparatus for identifying a non-compliant picture includes:

[0215] An input module 701 is configured to input a first picture to be identified into an identification model, so as to determine the compliance of the first picture based on an output result of the identification model; wherein the identification model includes a plurality of mutually independent supervised models and a plurality of mutually independent semi-supervised models; the identification model is obtained by training the plurality of supervised models and the plurality of semi-supervised models.

[0216] In an embodiment of the present disclosure, the apparatus for identifying a non-compliant picture includes:

[0217] A training module 702 is configured to train the plurality of supervised models and the plurality of semi-supervised models by sample data in a sample data pool, so as to obtain the identification model; wherein the sample data includes first unlabeled data appearing in a real non-compliant picture identification service, and second unlabeled data collected from a public picture resource pool.

[0218] In another embodiment of the present disclosure, the apparatus for identifying a non-compliant picture includes:

[0219] The training module 702 is further configured to pre-train the plurality of supervised models and the plurality of semi-supervised models by sample data in a cold start data pool, so as to enable the corresponding plurality of supervised models and the corresponding plurality of semi-supervised models to have a label prediction capability; wherein the sample data in the cold start data pool is first labeled data artificially labeled, and the sample quantity of the first labeled data is less than a preset quantity.

[0220] In another embodiment of the present disclosure, the apparatus for identifying a non-compliant picture includes:

[0221] An adding module 703 is configured to add a label for the first unlabeled data.

[0222] The adding module 703 is further configured to add a label for the second unlabeled data.

[0223] In another embodiment of the present disclosure, the apparatus for identifying a non-compliant picture includes:

[0224] The input module 701 is configured to input the first unlabeled data into the plurality of supervised models to obtain a first probability prediction value of a violation category to which a first sample picture in the first unlabeled data should belong according to each supervised model.

[0225] The adding module 703 is further configured to add a first violation category label determined by the supervised model to the first sample picture, in response to the first probability prediction value of the violation category to which the first sample picture should belong according to the supervised model being greater than a preset first probability threshold.

[0226] The determining module 704 is configured to determine a first prediction label of the first sample picture from the first violation category labels determined by the supervised models based on a voting mechanism, and add the first prediction label to the first unlabeled data to which the first sample picture belongs, to obtain second labeled data.

[0227] In another embodiment of the present disclosure, the violation picture identification device comprises:

[0228] The adding module 703 is further configured to add the label of the positive sample to the first part of the second unlabeled data.

[0229] The adding module 703 is further configured to add the label of the negative sample to the second part of the plurality of second unlabeled data.

[0230] In another embodiment of the present disclosure, the violation picture identification device comprises:

[0231] The input module 701 is further configured to input the second unlabeled data into the plurality of supervised models to obtain a second probability prediction value of a violation category to which a second sample picture in the second unlabeled data should belong according to each supervised model.

[0232] The adding module 703 is further configured to add a second violation category label determined by the supervised model to the second sample picture, in response to the second probability prediction value of the violation category to which the second sample picture should belong according to the supervised model being greater than a preset second probability threshold and less than a preset third probability threshold.

[0233] The determining module 704 is further configured to determine a second prediction label of the corresponding sample picture from the second violation category labels determined by the supervised models based on the voting mechanism, and add the second prediction label to the second unlabeled data to which the second sample picture belongs, the second preset label being the label of the positive sample.

[0234] In another embodiment of the present disclosure, the violation picture identification device comprises:

[0235] The training module 702 is further configured to retrain the plurality of supervised models according to each second labeled data.

[0236] The adding module 703 is further configured to add labels of negative samples to the second part of the plurality of second unlabeled data in response to the plurality of supervised models satisfying the stop training condition.

[0237] The executing module 705 is configured to return to the step of adding labels to the first unlabeled data in response to the plurality of supervised models not satisfying the stop training condition.

[0238] In another embodiment of the present disclosure, the device for identifying illegal pictures comprises:

[0239] The filtering module 706 is configured to filter noise data in the second labeled data by a confidence learning algorithm, and retrain the plurality of supervised models based on the remaining second labeled data after filtering.

[0240] In another embodiment of the present disclosure, the device for identifying illegal pictures comprises:

[0241] The determining module 704 is further configured to determine noise labels and real labels in the second labeled data, and determine noise data in the second labeled data based on a joint distribution between the noise labels and the real labels, the noise data being sample data mislabeled by the supervised models;

[0242] The filtering module 706 is further configured to filter out the noise data in the second labeled data, and retrain the plurality of supervised models based on the remaining second labeled data.

[0243] In another embodiment of the present disclosure, the device for identifying illegal pictures comprises: different first probability thresholds preset by the respective supervised models in the device for identifying illegal pictures, different second probability thresholds preset by the respective supervised models, and different third probability thresholds preset by the respective supervised models.

[0244] In another embodiment of the present disclosure, the device for identifying illegal pictures comprises:

[0245] The input module 701 is further configured to input the second unlabeled data into the plurality of supervised models to obtain third unlabeled data with misjudgment;

[0246] The learning module 707 is configured to perform semi-supervised learning on the respective third unlabeled data and the second labeled data by using the respective semi-supervised models to obtain semi-supervised models with misjudgment of illegal categories to which the third unlabeled data belong.

[0247] The input module 701 is further configured to input the second unlabeled data into the semi-supervised models with misjudgment of illegal categories to which the third unlabeled data belong, to obtain third probability prediction values of illegal categories to which second sample pictures in the second unlabeled data with misjudgment should belong according to the semi-supervised models.

[0248] The adding module 703 is configured to add labels of negative samples to the second unlabeled data with the third probability prediction value greater than the fourth preset probability threshold.

[0249] In another embodiment of the present disclosure, the device for identifying the illegal picture comprises:

[0250] The adding module 703 is further configured to train the plurality of supervised models by adding the first unlabeled data with the labels, the second unlabeled data with the labels of the positive samples, and the second unlabeled data with the labels of the negative samples, to obtain the identification model.

[0251] In another embodiment of the present disclosure, the device for identifying the illegal picture comprises:

[0252] The adding module 703 is further configured to train the plurality of supervised models by adding the first unlabeled data with the labels, the second unlabeled data with the labels of the positive samples, and the second unlabeled data with the labels of the negative samples;

[0253] The input module 701 is further configured to input the second unlabeled data into the plurality of supervised models trained, to obtain fifth probability prediction values of the illegal categories to which the second sample pictures in the second unlabeled data belong according to each supervised model;

[0254] The obtaining module 708 is configured to obtain the second unlabeled data with the fifth probability prediction values in the preset probability interval, for manual labeling.

[0255] The adding module 703 is further configured to re-train the plurality of supervised models trained by adding the first unlabeled data with the labels, the second unlabeled data with the labels of the positive samples, the second unlabeled data with the labels of the negative samples, and the second unlabeled data with the manual labels, to obtain the identification model.

[0256] In another embodiment of the present disclosure, the first unlabeled data is obtained from texts, images, and videos.

[0257] In another embodiment of the present disclosure, initial training parameters of the supervised models and the semi-supervised models in the device for identifying the illegal picture are training parameters of a basic model after the basic model is trained by data in a real illegal business.

[0258] In another embodiment of the present disclosure, the device for identifying the illegal picture comprises:

[0259] The processing module 709 is configured to process third sample pictures in the data in the real illegal business to obtain fourth sample pictures and auxiliary information, the processing including rotation, scaling, and / or coloring, and the auxiliary information being a relationship between the third sample pictures and the fourth sample pictures.

[0260] Training module 702 is also used to train the base model using third sample images, fourth sample images, and auxiliary information.

[0261] Exemplary Computing Device

[0262] Having described the methods, media, and apparatus of exemplary embodiments of this disclosure, the following references... FIG. 8 A computing device according to an exemplary embodiment of the present disclosure will be described.

[0263] FIG. 8 The computing device 80 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0264] like FIG. 8 As shown, the computing device 80 is presented in the form of a general-purpose computing device. The components of the computing device 80 may include, but are not limited to: at least one processing unit 801, at least one storage unit 802, and a bus 803 connecting different system components (including the processing unit 801 and the storage unit 802). The at least one storage unit 802 stores computer-executable instructions; the at least one processing unit 801 includes a processor that executes the computer-executable instructions to implement the methods described above.

[0265] The 803 bus includes a data bus, a control bus, and an address bus.

[0266] Storage unit 802 may include readable media in the form of volatile memory, such as random access memory (RAM) 8021 and / or cache memory 8022, and may further include readable media in the form of non-volatile memory, such as read-only memory (ROM) 8023.

[0267] Storage unit 802 may also include a program / utility 8025 having a set (at least one) of program modules 8028, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0268] The computing device 80 can also communicate with one or more external devices 808 (e.g., keyboard, pointing device, etc.). This communication can be performed via the input / output (I / O) interface 805. Furthermore, the computing device 80 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via a network adapter 806. FIG. 8As shown, the network adapter 806 communicates with other modules of the computing device 80 over the bus 803. It should be appreciated that although not shown, other hardware and / or software modules could be used in conjunction with the computing device 80. These include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0269] It should be noted that although in the foregoing detailed description several units / modules or sub-units / modules of the identification apparatus of the irregular picture are mentioned, such a division is merely exemplary and not mandatory. Indeed, according to embodiments of the present disclosure, features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, features and functions of one unit / module described above can be further divided into embodied by multiple units / modules.

[0270] Moreover, although operations of the methods of the present disclosure are described in a particular order in the figures, this is not required or implied in any manner, nor is it required that all of the illustrated operations be performed to achieve a desired result. Additionally or alternatively, certain steps can be omitted, combined into a single step, and / or broken into multiple steps.

[0271] While the spirit and principles of the present disclosure have been described with reference to several specific embodiments, it is to be understood that the disclosure is not limited to the specific embodiments disclosed and that the division of aspects is not meant to imply that features from these aspects cannot be combined to benefit. The division is merely for ease of presentation. The present disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the claims appended hereto.

Claims

1. A method of identifying a rule violation picture, characterized by, The method comprises the following steps: inputting a first picture to be identified into an identification model to determine compliance of the first picture based on an output result of the identification model; wherein the identification model comprises a plurality of mutually independent supervised models and a plurality of mutually independent semi-supervised models; and the method further comprises the following steps: training the plurality of supervised models and the plurality of semi-supervised models by sample data in a sample data pool to obtain the identification model; wherein the sample data comprises first unlabeled data occurring in a real violation picture identification service and second unlabeled data collected from a public picture resource pool; the step of training the plurality of supervised models and the plurality of semi-supervised models by sample data in the sample data pool comprises the following steps: adding labels to the first unlabeled data; adding labels to the second unlabeled data; the step of adding labels to the first unlabeled data comprises the following steps: inputting the first unlabeled data into the plurality of supervised models to obtain first probability prediction values of violation categories to which first sample pictures in the first unlabeled data belong according to respective supervised models; adding a first violation category label determined by the supervised model to the first sample picture in response to the first probability prediction value of the violation category to which the first sample picture belongs according to the supervised model being greater than a preset first probability threshold value; determining a first prediction label of the first sample picture from the first violation category labels determined by the respective supervised models based on a voting mechanism and adding the first prediction label to the first unlabeled data to which the first sample picture belongs to obtain second labeled data; the step of adding labels to the second unlabeled data comprises the following steps: adding labels of negative samples to a second part of the plurality of second unlabeled data; the step of adding labels of negative samples to the second part of the plurality of second unlabeled data comprises the following steps: inputting the second unlabeled data into the plurality of supervised models to obtain third unlabeled data in which misjudgment occurs; performing semi-supervised learning on the respective third unlabeled data and the second labeled data by using respective semi-supervised models to obtain semi-supervised models in which misjudgment of violation categories to which the third unlabeled data belongs occurs; inputting the second unlabeled data into the semi-supervised models in which misjudgment of violation categories to which the third unlabeled data belongs occurs to obtain third probability prediction values of violation categories to which second sample pictures in the second unlabeled data in which misjudgment occurs belong according to the semi-supervised models; adding labels of negative samples to the second unlabeled data in which the third probability prediction value is greater than a preset fourth probability threshold value.

2. The method of claim 1, wherein, before the step of training the plurality of supervised models and the plurality of semi-supervised models by sample data in the sample data pool to obtain the identification model, the method further comprises the following steps: Pre-train the plurality of supervised models and the plurality of semi-supervised models by sample data in a cold start data pool, so that the corresponding plurality of supervised models and the corresponding plurality of semi-supervised models have label prediction capability; wherein the sample data in the cold start data pool is first labeled data with labels, and the sample quantity of the first labeled data with labels is less than a preset quantity.

3. The method of claim 1, wherein, The adding labels for the second unlabeled data further includes: Adding labels for positive samples of the first part of the second unlabeled data.

4. The method of claim 3, wherein The adding labels for positive samples of the first part of the second unlabeled data includes: Inputting the second unlabeled data into the plurality of supervised models to obtain second probability prediction values of violation categories to which second sample pictures in the second unlabeled data belong according to each of the supervised models; In response to the second probability prediction values of violation categories to which the second sample pictures belong according to the supervised models being greater than a preset second probability threshold and less than a preset third probability threshold, adding second violation category labels determined by the supervised models to the second sample pictures; Determining second prediction labels of corresponding sample pictures from the second violation category labels determined by each of the supervised models based on a voting mechanism, and adding the second prediction labels to the second unlabeled data to which the second sample pictures belong, the second prediction labels being labels of positive samples.

5. The method of claim 4, wherein, The adding labels for negative samples of the second part of the plurality of second unlabeled data includes: Re-training the plurality of supervised models according to each of the second labeled data, the second labeled data being the first unlabeled data with added labels; In response to the plurality of supervised models satisfying a stop training condition, adding labels for negative samples of the second part of the plurality of second unlabeled data.

6. The method of claim 5, wherein, The re-training the plurality of supervised models according to each of the second labeled data includes: In response to the plurality of supervised models not satisfying the stop training condition, returning to execute the adding labels for the first unlabeled data.

7. The method of claim 5, wherein, The re-training the plurality of supervised models according to each of the second labeled data includes: Filtering noise data in the second labeled data by a confident learning algorithm, and re-training the plurality of supervised models based on the remaining second labeled data after filtering.

8. The method of claim 7, wherein, The re-training the plurality of supervised models based on the remaining second labeled data after filtering includes: Determining noise labels and real labels in the second labeled data, and determining noise data in the second labeled data based on a joint distribution between the noise labels and the real labels, the noise data being sample data incorrectly labeled by the supervised models; Filtering out the noise data in the second labeled data, and re-training the plurality of supervised models based on the remaining second labeled data.

9. The method of claim 4, wherein, The first probability threshold of each of the supervised models is different, the second probability threshold of each of the supervised models is different, and the third probability threshold of each of the supervised models is different.

10. The method of claim 1, wherein, The step of adding labels of negative samples to the second unlabeled data with the third probability prediction value greater than the fourth preset probability threshold further comprises: The plurality of supervised models are trained by the first unlabeled data with added labels, the second unlabeled data with added labels of positive samples, and the second unlabeled data with added labels of negative samples to obtain the identification model.

11. The method of claim 1, wherein, The step of adding labels of negative samples to the second unlabeled data with the third probability prediction value greater than the fourth preset probability threshold further comprises: The plurality of supervised models are trained by the first unlabeled data with added labels, the second unlabeled data with added labels of positive samples, and the second unlabeled data with added labels of negative samples; The second unlabeled data with the fifth probability prediction value of the violation category to which the second sample picture in the second unlabeled data belongs is obtained by inputting the second unlabeled data into the plurality of trained supervised models; The second unlabeled data with the fifth probability prediction value in the preset probability interval is obtained for manual labeling of labels; The plurality of trained supervised models are retrained by the first unlabeled data with added labels, the second unlabeled data with added labels of positive samples, the second unlabeled data with added labels of negative samples, and the second unlabeled data with manual labels to obtain the identification model.

12. The method of claim 2, wherein, The first unlabeled data is obtained from text, images, and videos.

13. The method of claim 1, wherein, The initial training parameters of the supervised models and the semi-supervised models are training parameters of a basic model after the basic model is trained by data in a real violation business.

14. The method of claim 13, wherein, The training of the basic model by data in a real violation business comprises: The third sample picture in the real violation business is processed to obtain a fourth sample picture and auxiliary information, the processing comprising rotation, scaling, and / or coloring, and the auxiliary information being a relationship between the third sample picture and the fourth sample picture; The basic model is trained by the third sample picture, the fourth sample picture, and the auxiliary information.

15. An apparatus for identifying a rule violation picture, characterized by Comprise: An input module is configured to input a first picture to be identified into an identification model to determine compliance of the first picture based on an output result of the identification model; wherein the identification model comprises a plurality of mutually independent supervised models and a plurality of mutually independent semi-supervised models; the identification model is obtained by training the plurality of supervised models and the plurality of semi-supervised models; A training module is configured to train the plurality of supervised models and the plurality of semi-supervised models by sample data in a sample data pool to obtain the identification model; wherein the sample data comprises first unlabeled data appearing in a real violation picture identification business and second unlabeled data collected from a public picture resource pool; An adding module is configured to add labels to the first unlabeled data and add labels to the second unlabeled data. The input module is further configured to input the first unlabeled data into the plurality of supervised models to obtain a first probability prediction value of a violation category to which a first sample picture in the first unlabeled data should belong according to each supervised model; The adding module is further configured to add a first violation category label determined by the supervised model to the first sample picture in response to the first probability prediction value of the violation category to which the first sample picture should belong according to the supervised model being greater than a preset first probability threshold value. The determining module is configured to determine a first prediction label of the first sample picture from the first violation category labels determined by the respective supervised models based on a voting mechanism, and add the first prediction label to the first unlabeled data to which the first sample picture belongs to obtain second labeled data. The adding module is further configured to add a negative sample label to a second part of the plurality of second unlabeled data. The input module is further configured to input the second unlabeled data into the plurality of supervised models to obtain third unlabeled data with misjudgment. The learning module is configured to perform semi-supervised learning on the respective third unlabeled data and the second labeled data by using each semi-supervised model to obtain a semi-supervised model with misjudgment of a violation category to which the third unlabeled data belongs. The input module is further configured to input the second unlabeled data into the semi-supervised model with misjudgment of the violation category to which the third unlabeled data belongs to obtain a third probability prediction value of a violation category to which a second sample picture in the second unlabeled data with misjudgment should belong according to the semi-supervised model. The adding module is further configured to add a negative sample label to the second unlabeled data with the third probability prediction value greater than a preset fourth probability threshold value.

16. The device for identifying a rule violation picture according to claim 15, characterized in that The training module is further configured to pre-train the plurality of supervised models and the plurality of semi-supervised models by using sample data in a cold start data pool to enable the respective plurality of supervised models and the respective plurality of semi-supervised models to have label prediction capability; wherein the sample data in the cold start data pool is first labeled data manually labeled, and a sample quantity of the first labeled data is less than a preset quantity. The adding module is further configured to add a positive sample label to a first part of the second unlabeled data.

17. The device for identifying a rule violation picture according to claim 15, characterized in that The input module is further configured to input the second unlabeled data into the plurality of supervised models to obtain a second probability prediction value of a violation category to which a second sample picture in the second unlabeled data should belong according to each supervised model; The adding module is further configured to add a second violation category label determined by the supervised model to the second sample picture in response to the second probability prediction value of the violation category to which the second sample picture should belong according to the supervised model being greater than a preset second probability threshold value and less than a preset third probability threshold value.

18. The device for identifying a rule violation picture according to claim 17, characterized in that ​ ​ ​ The determination module is further configured to determine a second prediction label of the corresponding sample picture from the second violation category labels determined by the respective supervision models based on a voting mechanism, and add the second prediction label to the second unlabeled data to which the second sample picture belongs, the second prediction label being a label of a positive sample.

19. The device for identifying a rule violation picture according to claim 18, characterized in that The method comprises: The training module is further configured to retrain the plurality of supervision models according to the respective second labeled data; The adding module is further configured to add a label of a negative sample to a second part of the plurality of second unlabeled data in response to the plurality of supervision models satisfying a stop training condition.

20. The device for identifying a rule violation picture according to claim 19, characterized in that The method comprises: The execution module is configured to return to performing the step of adding the label to the first unlabeled data in response to the plurality of supervision models not satisfying the stop training condition.

21. The device for identifying a rule violation picture according to claim 19, characterized in that The method comprises: The filtering module is configured to filter noise data in the second labeled data by a confident learning algorithm, and retrain the plurality of supervision models based on the remaining second labeled data after filtering.

22. The device for identifying a rule violation picture according to claim 21, characterized in that The method comprises: The determination module is further configured to determine noise labels and real labels in the second labeled data, and determine noise data in the second labeled data based on a joint distribution between the noise labels and the real labels, the noise data being sample data incorrectly labeled by the supervision models; The filtering module is further configured to filter out the noise data in the second labeled data, and retrain the plurality of supervision models based on the remaining second labeled data.

23. The apparatus for identifying a rule violation picture of claim 18, wherein, The first probability threshold of each of the supervision models in the violation picture identification device is different, the second probability threshold of each of the supervision models is different, and the third probability threshold of each of the supervision models is different.

24. The apparatus for identifying a rule-violation picture according to claim 15, wherein, The method comprises: The adding module is further configured to train the plurality of supervision models by adding the first unlabeled data with the label, the second unlabeled data with the label of the positive sample, and the second unlabeled data with the label of the negative sample, to obtain an identification model.

25. The apparatus for identifying a rule-violation picture according to claim 15, wherein, The method comprises: The adding module is further configured to train the plurality of supervision models by adding the first unlabeled data with the label, the second unlabeled data with the label of the positive sample, and the second unlabeled data with the label of the negative sample; The input module is further configured to input the second unlabeled data into the plurality of trained supervision models to obtain a fifth probability prediction value of a violation category to which a second sample picture in the second unlabeled data belongs according to each of the supervision models; The acquisition module is configured to acquire the second unlabeled data with the fifth probability prediction value in a preset probability interval for manual labeling. The adding module is further configured to retrain the plurality of trained supervision models by adding the first unlabeled data with the label, the second unlabeled data with the label of the positive sample, the second unlabeled data with the label of the negative sample, and the second unlabeled data with the manual label, to obtain an identification model.

26. The device for identifying a rule-violation picture according to claim 16, wherein The first unlabeled data is obtained from text, image, and video.

27. The apparatus for identifying a rule-violation picture according to claim 15, wherein, The initial training parameters of the supervision model and the semi-supervised model in the illegal picture identification device are training parameters of a basic model after the basic model is trained by data in real illegal businesses.

28. The apparatus for identifying a rule-violation picture according to claim 27, wherein Comprise: A processing module, configured to process a third sample picture of data in real illegal businesses to obtain a fourth sample picture and auxiliary information, the processing comprising rotation, scaling and / or coloring, and the auxiliary information being a relationship between the third sample picture and the fourth sample picture; The training module is further configured to train the basic model by the third sample picture, the fourth sample picture and the auxiliary information.

29. A medium characterized by: Comprise: Computer execution instructions, when executed by a processor, are used to implement the illegal picture identification method of any one of claims 1 to 14.

30. A computing device, comprising: Comprise: A memory and a processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the illegal picture identification method of any one of claims 1 to 14.

Images