Method and apparatus for training content review model

Abstract

The application provides a content review model training method and device, equipment, a storage medium and a computer program product; the method comprises the following steps: predicting compliance of at least one piece of content to be reviewed by a content review model to obtain a predicted review result of each piece of content to be reviewed; for each piece of content to be reviewed, determining a predicted difficulty score of the content to be reviewed based on the predicted review result; from at least one piece of content to be reviewed, a target review content is selected whose predicted difficulty score meets a difficulty score condition; the content review model is trained by taking the target review content as a sample and taking the actual review result of the target review content as a sample label of the sample, so as to update the content review model; by the application, the training cost of the content review model can be reduced, and the training speed and content review accuracy of the content review model can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a training method, apparatus, device, storage medium, and computer program product for a content moderation model. Background Technology

[0002] Artificial Intelligence (AI) is a comprehensive technology within computer science that studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities. AI technology is a multidisciplinary field encompassing a wide range of areas, including natural language processing and machine learning / deep learning. With technological advancements, AI will be applied in more fields and play an increasingly important role.

[0003] Content moderation is also an important application area of ​​artificial intelligence. In related technologies, pre-trained content moderation models are usually used for content moderation. However, pre-training a content moderation model requires building a large number of training samples and labeling them extensively. Moreover, obtaining samples of non-compliant content is also very difficult, resulting in high model training costs, long model iteration cycles, and the content moderation model is also fixed, leading to low accuracy in reviewing complex and ever-changing content. Summary of the Invention

[0004] This application provides a training method, apparatus, device, storage medium, and computer program product for a content moderation model, which can reduce the training cost of the content moderation model and improve the training speed and content moderation accuracy of the content moderation model.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a method for training a content moderation model, including:

[0007] By using a content moderation model, compliance prediction is performed on at least one piece of content to be reviewed, and the predicted review results for each piece of content to be reviewed are obtained.

[0008] For each of the items to be reviewed, a predicted difficulty score is determined based on the predicted review results;

[0009] The prediction difficulty score is used to indicate the degree of difficulty in making an accurate prediction of the content to be reviewed through the content review model.

[0010] From the at least one piece of content to be reviewed, target content that meets the predicted difficulty score condition is selected for review, and the actual review result of the target content is obtained.

[0011] Using the target content to be reviewed as a sample, and the actual review result of the target content to be reviewed as the sample label of the sample, the content review model is trained and updated.

[0012] In the above scheme, training the content review model using the target content to be reviewed as a sample and the actual review results of the target content to be reviewed as the sample labels includes:

[0013] Obtain a first review content sample and a second review content sample. The first review sample carries a tag, while the second review content sample does not carry a tag.

[0014] Using the target content to be reviewed as a sample, and the actual review result of the target content to be reviewed as the sample label of the sample, the content review model is trained in combination with the first content review sample to obtain an intermediate content review model;

[0015] Using the second sample of content to be reviewed, the intermediate content review model is trained for the target number of rounds.

[0016] In the above scheme, the target number of rounds is k, where k is an integer greater than 0. The step of using the second review content sample to train the intermediate content review model for the target number of rounds includes:

[0017] For the first round of training in the k rounds of training, the second content review sample is predicted by the intermediate content review model to obtain a first prediction result, and the first prediction result is used as the first sample label of the second content review sample;

[0018] Based on the second content sample carrying the first sample label, the intermediate content review model is trained to obtain the first content review model trained in the first round of k rounds of training.

[0019] For the nth training round in the k rounds of training, the second content review sample is predicted by the first content review model trained in the (n-1)th round to obtain the second prediction result, and the second prediction result is used as the second sample label of the second content review sample;

[0020] Based on the second content sample carrying the second sample label, the first content review model in the (n-1)th round of training is trained to obtain the first content review model in the (n-1)th round of training in the k rounds of training, where n is an integer greater than 1 and less than or equal to k;

[0021] By iterating through n, we obtain the first content moderation model trained in the kth round.

[0022] In the above scheme, the intermediate content review model includes multiple sub-content review models. The step of predicting the second review content sample using the intermediate content review model to obtain a first prediction result includes:

[0023] By using at least two target sub-content review models from the plurality of sub-content review models, the second review content sample is predicted respectively, and the sub-model prediction results of each target sub-content review model are obtained;

[0024] Obtain the weight values ​​of each target sub-content review model;

[0025] Based on the weight values ​​of each target sub-content review model, the sub-model prediction results of at least two target sub-content review models are weighted and averaged to obtain the first prediction result.

[0026] This application embodiment also provides a training device for a content moderation model, including:

[0027] The prediction module is used to predict the compliance of at least one piece of content to be reviewed through a content review model, and obtain the predicted review results for each piece of content to be reviewed.

[0028] The determination module is used to determine the prediction difficulty score of each of the contents to be reviewed, based on the predicted review result; wherein the prediction difficulty score is used to indicate the degree of difficulty in obtaining an accurate prediction result by predicting the contents to be reviewed through the content review model.

[0029] The filtering module is used to filter out target review content that meets the predicted difficulty score condition from the at least one piece of content to be reviewed, and obtain the actual review result of the target review content;

[0030] The training module is used to train the content review model using the target content to be reviewed as a sample and the actual review result of the target content to be reviewed as the sample label of the sample, so as to update the content review model.

[0031] In the above scheme, the prediction module is further used to obtain the generation time of each of the contents to be reviewed; input the at least one content to be reviewed into the content review model in batches according to the order of the generation time; and perform compliance prediction on each batch of contents to be reviewed through the content review model to obtain the predicted review result of each content to be reviewed, wherein the batch includes a target number of contents to be reviewed.

[0032] In the above scheme, the content review model includes multiple sub-content review models. The prediction module is further configured to perform the following processing for each of the contents to be reviewed: perform compliance prediction on the contents to be reviewed using each of the sub-content review models to obtain intermediate prediction review results for each sub-content review model; obtain the prediction accuracy score for each sub-content review model and use the prediction accuracy score as the weight value of the sub-content review model; wherein the prediction accuracy score is used to indicate the accuracy of the intermediate prediction review results of the corresponding sub-content review model; and perform weighted processing on the intermediate prediction review results of the multiple sub-content review models based on the weight values ​​of each sub-content review model to obtain the prediction review result of the contents to be reviewed.

[0033] In the above scheme, the content review model is used to perform classification prediction for at least two categories, and the predicted review result includes: the probability value of the content to be reviewed belonging to each of the categories; the determining module is further used to determine the information entropy value corresponding to the content to be reviewed based on the probability value of the content to be reviewed belonging to each of the categories, and use the information entropy value as the prediction difficulty score of the content to be reviewed.

[0034] In the above scheme, the content review model includes a first number of sub-content review models, and the predicted review result includes the intermediate predicted review result of each of the sub-content review models. The sub-content review model is used to perform classification prediction for at least two categories. The determining module is further used to obtain, for each category, a second number of sub-content review models whose intermediate predicted review results indicate the category. Based on the first number and the second number corresponding to each category, the predicted difficulty score of the content to be reviewed is determined.

[0035] In the above scheme, the filtering module is further used to select, from the at least one piece of content to be reviewed, the content whose predicted difficulty score reaches the first difficulty score threshold as the target content to be reviewed.

[0036] In the above scheme, the filtering module is further configured to sort the at least one piece of content to be reviewed in descending order based on the predicted difficulty score; and select the target number of content to be reviewed from the sorted at least one piece of content to be reviewed as the target content to be reviewed.

[0037] In the above scheme, when there are multiple target content items, the filtering module is further configured to determine the object review score of each target content item based on its predicted difficulty score. The object review score indicates the likelihood of object review of the target content item. Target content items whose object review scores reach a review score threshold are identified as object review content. Correspondingly, the filtering module is also configured to obtain the actual review results of the object review content. Correspondingly, the training module is also configured to train the content review model using the object review content as samples and the actual review results of the object review content as the sample labels of the samples.

[0038] In the above scheme, the filtering module is further configured to, when the predicted difficulty score is lower than the second difficulty score threshold, use the product of the predicted difficulty score and the object review ratio of the object review as the object review score of the target review content; when the predicted difficulty score is not lower than the second difficulty score threshold, determine the object review score of the target review content as the target score, and the target score is used to indicate the operation of determining the target review content as the object review content.

[0039] In the above scheme, the filtering module is further configured to obtain the planned content review volume and the remaining content review volume for the target time period, and use the ratio of the remaining content review volume to the planned content review volume as the time period review ratio for the target time period; correspondingly, the filtering module is further configured to use the product of the predicted difficulty score, the object review ratio, and the time period review ratio as the object review score of the target review content; correspondingly, the filtering module is further configured to determine the object review score of the target review content as the product of the target score and the time period review ratio.

[0040] In the above scheme, the training module is further configured to: determine a first review content from the target review content, wherein the predicted review result of the first review content is different from the actual review result of the first review content; perform data augmentation processing on the first review content to obtain a second review content, and use the actual review result of the first review content as the actual review result of the second review content; use the target review content and the second review content as the samples, and use the actual review result of the target review content and the actual review result of the second review content as the sample labels of the samples, to train the content review model.

[0041] In the above scheme, the training module is further used to obtain a first review content sample and a second review content sample, wherein the first review sample carries a label and the second review content sample does not carry a label; using the target review content as a sample and the actual review result of the target review content as the sample label of the sample, and combining the first review content sample, the content review model is trained to obtain an intermediate content review model; using the second review content sample, the intermediate content review model is trained for a target number of rounds.

[0042] In the above scheme, the target number of rounds is k, where k is an integer greater than 0. The training module is further configured to: predict the second content review sample using the intermediate content review model for the first round of training in the k rounds, obtain a first prediction result, and use the first prediction result as the first sample label of the second content review sample; train the intermediate content review model based on the second content review sample carrying the first sample label to obtain the first content review model trained in the first round of training in the k rounds; predict the second content review sample using the first content review model trained in the (n-1)th round of training in the k rounds, obtain a second prediction result, and use the second prediction result as the second sample label of the second content review sample; train the first content review model trained in the (n-1)th round based on the second content review sample carrying the second sample label to obtain the first content review model trained in the (n-1)th round of training in the k rounds, where n is an integer greater than 1 and less than or equal to k; and iterate through n to obtain the first content review model trained in the k rounds.

[0043] In the above scheme, the intermediate content review model includes multiple sub-content review models. The training module is further configured to predict the second review content sample using at least two target sub-content review models from the multiple sub-content review models, respectively, to obtain the sub-model prediction results of each target sub-content review model; obtain the weight values ​​of each target sub-content review model; and perform a weighted average of the sub-model prediction results of at least two target sub-content review models based on the weight values ​​of each target sub-content review model to obtain the first prediction result.

[0044] In the above scheme, the training module is further configured to acquire test content samples, which carry test sample labels; predict the test content samples using a target content review model obtained by training the content review model, and obtain test prediction results; determine the prediction accuracy score of the target content review model based on the test prediction results and the test sample labels, whereby the prediction accuracy score indicates the accuracy of the test prediction results; and update the content review model using the target content review model when the prediction accuracy score exceeds an accuracy score threshold.

[0045] This application also provides an electronic device, including:

[0046] Memory is used to store executable instructions for a computer;

[0047] The processor, when executing computer-executable instructions stored in the memory, implements the training method for the content moderation model provided in the embodiments of this application.

[0048] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the training method for the content moderation model provided in this application.

[0049] This application also provides a computer program product, including a computer program or computer executable instructions, which, when executed by a processor, implements the training method for the content moderation model provided in this application.

[0050] The embodiments of this application have the following beneficial effects:

[0051] First, a content moderation model is used to perform content moderation on the content to be reviewed, and the model is trained based on the target content within that content. This allows for simultaneous content moderation and model training without the need to construct an additional training set (including sample construction and labeling), thus reducing training costs.

[0052] Second, after the content to be reviewed is reviewed by the content moderation model, based on the predicted difficulty score of each piece of content to be reviewed by the content moderation model, target content that meets the difficulty score condition is selected from at least one piece of content to be reviewed. Thus, 1) in terms of content moderation, if the target content to be reviewed is manually reviewed to obtain the actual review results (i.e., manual review results), the amount of manual review can be reduced, and the cost of manual review can be lowered. That is, it can realize both machine review of the content to be reviewed and manual review of only the target content that meets the difficulty score condition. While reducing the amount of manual review, the cost of manual review is reduced and the efficiency of content moderation is improved through human-machine collaborative review; 2) in terms of model training, there is no need to label a large number of training samples. Only the target content that meets the difficulty score condition needs to be labeled, which improves the model training speed and timeliness.

[0053] Third, as the model training process progresses, the predictive ability of the obtained content review model becomes stronger and stronger, the prediction difficulty score for the content to be reviewed will decrease, and the target content to be reviewed based on the prediction difficulty score will become less and less, thereby further reducing the annotation cost (i.e., the target content to be reviewed that needs to obtain actual review results is reduced) and improving the model training speed.

[0054] Fourth, it can optimize and update the model based on the real-time content to be reviewed, making the trained content review model more flexible, adaptable to complex and ever-changing content to be reviewed, and improving the accuracy of model-based content review. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of the architecture of the training system 100 for the content moderation model provided in this application embodiment;

[0056] Figure 2 This is a schematic diagram of the structure of an electronic device 500 for implementing the training method of the content review model provided in this application embodiment;

[0057] Figure 3 This is a flowchart illustrating the training method of the content moderation model provided in this application embodiment;

[0058] Figure 4 This is a flowchart illustrating the training method of the content moderation model provided in this application embodiment;

[0059] Figure 5 This is a flowchart illustrating the training method of the content moderation model provided in this application embodiment;

[0060] Figure 6This is a flowchart illustrating the training method of the content moderation model provided in this application embodiment;

[0061] Figure 7 This is a flowchart illustrating the training method of the content moderation model provided in this application embodiment;

[0062] Figure 8 This is a schematic diagram of the human-computer collaborative content moderation framework provided in the embodiments of this application;

[0063] Figure 9 This is a training diagram of the content moderation model provided in the embodiments of this application. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0065] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0066] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0068] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0069] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0070] 1) Client: An application that runs in a terminal to provide various services, such as a content client.

[0071] 2) Responding to indicates the conditions or states on which the operation is performed depends. When the conditions or states on which it depends are met, one or more operations can be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which multiple operations are performed.

[0072] 3) Active Learning: This is a subfield of machine learning, also known as query learning or optimal experimental design in statistics. Active learning attempts to solve the bottleneck of sample labeling by proactively prioritizing the labeling of the most valuable unlabeled samples, thereby achieving the model's expected performance with as few labeled samples as possible.

[0073] 4) Uncertain samples: Sample data that is difficult to distinguish in the model, and sample data with low model confidence.

[0074] 5) Ensemble learning: also known as "multi-classifier system" or "committee-based learning". The basic idea is to combine multiple learners to obtain a learner with better generalization performance than a single learner.

[0075] 6) Semi-Supervised Learning (SSL): This is a key research area in pattern recognition and machine learning, combining supervised and unsupervised learning methods. SSL uses a large amount of unlabeled data, as well as labeled data simultaneously, to perform pattern recognition tasks. When using SSL, the goal is typically to minimize the number of personnel involved while still achieving relatively high accuracy.

[0076] 7) Distillation learning: By introducing a soft-target related to the teacher network (complex but with superior inference performance) as part of the total loss, the training of the student network (simple and low-complexity) is induced, thereby achieving knowledge transfer and resulting in a model with fewer model parameters and better generalization.

[0077] 8) Incremental learning: refers to a learning system that can continuously learn new knowledge from new samples and retain most of the knowledge that has been learned before.

[0078] 9) Model cold start: refers to the process of building a business model (such as a content moderation model) from scratch with little or no training data.

[0079] This application provides a training method, apparatus, device, storage medium, and computer program product for a content moderation model, which can reduce the training cost of the content moderation model and improve the training speed and content moderation accuracy of the content moderation model.

[0080] The following describes the implementation scenario of the training method for the content moderation model provided in the embodiments of this application. See also Figure 1 , Figure 1 This is a schematic diagram of the architecture of the training system 100 for the content moderation model provided in this application embodiment. In order to support an exemplary application, the terminal 400 connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two, and data transmission is achieved using wireless or wired links.

[0081] Terminal 400 (including at least one terminal, each terminal may be configured with a content publishing client) is used to send content to server 200 in response to a confirmation publishing instruction for the content;

[0082] Server 200 is configured to receive at least one piece of content sent by terminal 400 and identify the at least one piece of content as content to be reviewed; perform compliance prediction on the at least one piece of content to be reviewed using a content review model to obtain the predicted review result for each piece of content to be reviewed; for each piece of content to be reviewed, based on the predicted review result, determine the predicted difficulty score of the content to be reviewed, which indicates the difficulty of predicting the content to be reviewed through the content review model to obtain an accurate prediction result; select target content to be reviewed from the at least one piece of content to be reviewed that meets the difficulty score condition, and obtain the actual review result of the target content to be reviewed.

[0083] In practical applications, the predicted audit results and the actual audit results can be used as the audit results for the corresponding content to be audited. The audit results are used to indicate whether the content to be audited is compliant. The content to be audited is processed according to the audit results. For example, if the audit results indicate that the content to be audited is non-compliant, the content to be audited is blocked to prevent its publication and a non-compliance notification message is returned to terminal 400. If the audit results indicate that the content to be audited is compliant, the content to be audited is published and a compliance notification message is returned to terminal 400.

[0084] Server 200 is also used to train the content review model using the target content to be reviewed as samples and the actual review results of the target content to be reviewed as sample labels, thereby obtaining a target content review model; the target content review model is then used to update the content review model, so as to perform compliance prediction through the updated content review model. In practical applications, the training method of the content review model provided in this application embodiment can be repeatedly executed according to the real-time generated content to be reviewed, so as to iteratively train the content review model, improve the review accuracy of the content review model, and reduce the cost of manual review.

[0085] Terminal 400 is also used to receive and display notification messages returned by server 200.

[0086] In some embodiments, the training method for the content moderation model provided in this application can be implemented by various electronic devices. For example, it can be implemented by a terminal alone, by a server alone, or by a terminal and a server working together. For example, a terminal can execute the training method for the content moderation model provided in this application independently. This application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving.

[0087] In some embodiments, the electronic device used to train the content moderation model provided in this application can be various types of terminal devices or servers. The server (e.g., server 200) can be a standalone physical server, a server cluster consisting of multiple physical servers, or a distributed system. The terminal (e.g., terminal 400) can be a smartphone, tablet computer, laptop computer, desktop computer, smart voice interaction device (e.g., smart speaker), smart home appliance (e.g., smart TV), smartwatch, in-vehicle terminal, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations on this.

[0088] In some embodiments, the training method for the content moderation model provided in this application can be implemented using cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to achieve data computation, storage, processing, and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology applied based on the cloud computing business model. It can form a resource pool, be used on demand, and is flexible and convenient. Cloud computing technology will become an important support. The backend services of the technology network system require a large amount of computing and storage resources. As an example, a server (e.g., server 200) can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0089] In some embodiments, the terminal or server can implement the training method of the content moderation model provided in this application by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), that is, a program that needs to be installed in the operating system to run; it can also be a mini-program, that is, a program that only needs to be downloaded to the browser environment to run; or it can be a mini-program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plugin.

[0090] In some embodiments, multiple servers can form a blockchain, with each server being a node on the blockchain. Information connections can exist between each node in the blockchain, allowing for information transmission between nodes. Furthermore, the data related to the training method of the content moderation model provided in this application (e.g., content moderation model, samples, sample tags, etc.) can be stored on the blockchain.

[0091] The following describes an electronic device for training a content moderation model according to embodiments of this application. See also: Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device 500 that implements the training method for the content review model provided in this application embodiment. Taking the electronic device 500 as an example... Figure 1Taking the server shown as an example, the electronic device 500 for implementing the training method of the content moderation model provided in this application embodiment includes: at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. The various components in the electronic device 500 are coupled together through a bus system 540. It is understood that the bus system 540 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 540.

[0092] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0093] Memory 550 may be removable, non-removable, or a combination thereof. Memory 550 may optionally include one or more storage devices physically located away from processor 510. Memory 550 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 550 described in this application embodiment is intended to include any suitable type of memory.

[0094] In some embodiments, memory 550 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0095] Operating system 551 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0096] The network communication module 552 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0097] In some embodiments, the training apparatus for the content moderation model provided in this application can be implemented in software. Figure 2 A training device 553 for a content moderation model stored in memory 550 is shown. It can be software in the form of programs and plug-ins, including the following software modules: prediction module 5531, determination module 5532, filtering module 5533, training module 5534, and update module 5535. These modules are logical and can therefore be arbitrarily combined or further split according to the functions they implement. The functions of each module will be described below.

[0098] The following describes the training method for the content moderation model provided in the embodiments of this application. In some embodiments, the training method for the content moderation model provided in the embodiments of this application can be implemented by various electronic devices. For example, it can be implemented by a terminal alone, by a server alone, or by a terminal and a server working together. Taking server implementation as an example, see [link to documentation]. Figure 3 , Figure 3 This is a flowchart illustrating the training method for the content moderation model provided in this application embodiment. The training method for the content moderation model provided in this application embodiment includes:

[0099] Step 101: The server uses a content moderation model to predict the compliance of at least one piece of content to be reviewed, and obtains the predicted review results for each piece of content to be reviewed.

[0100] In step 101, the content moderation model can be an initially constructed content moderation model, or it can be trained based on a small number of content samples (carrying manually labeled tags, i.e., the results of manual review of the content samples). In actual implementation, a cold start process can be performed to create the content moderation model with no or few training samples.

[0101] In step 101, the at least one piece of content to be reviewed can be content sent by the user in real time or content sent by the user over a period of time. This content can be media information such as news, music, short videos, long videos, images, and text. Because the content contains a large amount of text, images, videos, audio, comments, and bullet comments, in practical applications, it is usually necessary to review the content to determine whether it complies with relevant regulations, thereby determining whether to proceed with content delivery, publication, or other operations.

[0102] In step 101, when the server receives at least one piece of content to be reviewed generated by the user, it performs a compliance prediction on at least one piece of content to be reviewed, obtaining the predicted review result for each piece of content to be reviewed. In practical applications, compliance refers to conformity to relevant regulations, which may be formulated regarding the legality, quality, theme, security, etc. of the content. The predicted review result can be seen in the following examples: 1) Blacklisted: The content to be reviewed is identified as containing harmful elements (i.e., non-compliant content); 2) Whitelisted: The content to be reviewed is identified as not containing harmful elements; 3) Graylisted: The content to be reviewed may contain harmful elements.

[0103] In some embodiments, the number of content awaiting review typically fluctuates, meaning it varies at different times. To ensure efficient use of model prediction computing resources, at least one piece of content awaiting review can be processed in batches, with a target number of pieces of content to be processed in each batch. Specifically, the server can use the content review model to predict the compliance of at least one piece of content awaiting review, obtaining the predicted review results for each piece of content awaiting review: obtaining the generation time of each piece of content awaiting review; inputting at least one piece of content awaiting review into the content review model in batches according to the order of generation time; and using the content review model to sequentially predict the compliance of each batch of content awaiting review, obtaining the predicted review results for each batch, where each batch includes the target number of pieces of content awaiting review.

[0104] In some embodiments, a content moderation model may include multiple sub-content moderation models. Based on this, see [link to relevant documentation]. Figure 4 , Figure 4 This is a flowchart illustrating the training method of the content moderation model provided in this application embodiment. Figure 4 It shows Figure 3 Step 101 can be implemented through steps 1011-1013: For each piece of content to be reviewed, the following processing is performed: Step 1011, using each sub-content review model, compliance prediction is performed on the content to be reviewed to obtain the intermediate prediction review results of each sub-content review model; Step 1012, for each sub-content review model, the prediction accuracy score of the sub-content review model is obtained, and the prediction accuracy score is used as the weight value of the sub-content review model; wherein, the prediction accuracy score is used to indicate the accuracy of the intermediate prediction review results of the corresponding sub-content review model; Step 1013, based on the weight values ​​of each sub-content review model, the intermediate prediction review results of multiple sub-content review models are weighted to obtain the prediction review results of the content to be reviewed.

[0105] In practical applications, the content moderation model can be constructed using ensemble learning, meaning the content moderation model is defined as comprising multiple sub-content moderation models. In step 1012, the prediction accuracy score can be represented by the accuracy of the sub-content moderation models. That is, when a prediction is made on a set of content samples (including multiple content samples, each carrying a sample label) based on the sub-content moderation model and the prediction result is obtained, the accuracy of the sub-content moderation model can be determined based on the prediction result and the sample label. For example, the ratio between the number of content samples with the same prediction result and sample label and the total number of content samples can be used as the accuracy of the sub-content moderation model. In step 1013, the weighting process can be either a weighted average or a weighted summation.

[0106] Step 102: For each item to be reviewed, determine the predicted difficulty score based on the predicted review results.

[0107] The prediction difficulty score indicates the degree of difficulty in obtaining accurate prediction results for the content to be reviewed through the content moderation model.

[0108] In step 102, for each piece of content to be reviewed, a prediction difficulty score is determined based on the predicted review result. This prediction difficulty score is used to determine the difficulty of obtaining an accurate prediction result for the content to be reviewed through the content review model. It should be noted that when the difference between the predicted review result and the actual review result of the content to be reviewed is less than a difference threshold, or when the predicted review result and the actual review result are the same, the predicted review result is considered an accurate prediction result. This prediction difficulty score can be represented by information entropy, or a corresponding difficulty score can be set for the difference, and the difficulty score corresponding to the difference between the predicted review result and the actual review result can be used as the prediction difficulty score.

[0109] In step 102, a higher prediction difficulty score indicates a greater difficulty in obtaining an accurate prediction result for the content to be reviewed using the content moderation model, and also indicates a higher degree of uncertainty regarding the content to be reviewed. Thus, when training the content moderation model based on the content to be reviewed, the content to be reviewed can be considered an uncertain sample for the content moderation model. Therefore, based on the prediction difficulty score, a portion of the content to be reviewed can be selected from at least one piece of content to be reviewed, and after manual review of the selected content, the content moderation model can be retrained based on the actual review results obtained from the manual review.

[0110] In some embodiments, the content moderation model is used to perform classification prediction for at least two categories, and the prediction result includes the probability value of the content to be reviewed belonging to each category; based on this, the server can determine the prediction difficulty score of the content to be reviewed based on the prediction result in such a way as follows: based on the probability value of the content to be reviewed belonging to each category, determine the information entropy value corresponding to the content to be reviewed, and use the information entropy value as the prediction difficulty score of the content to be reviewed.

[0111] In practical applications, the information entropy value corresponding to the content to be reviewed can be determined based on the probability value of the content belonging to each category, using the following formula (I):

[0112] ; (one)

[0113] in, The value of information entropy. To pass the content moderation model Predicted content to be reviewed For the first Category Possibility (i.e., the content to be reviewed belongs to the category of) (Probability value of the category).

[0114] In some embodiments, the content moderation model includes a first number of sub-content moderation models, and the predicted moderation result includes intermediate predicted moderation results of each sub-content moderation model. The sub-content moderation model is used to perform classification prediction for at least two categories. Based on this, the server can determine the prediction difficulty score of the content to be moderated based on the predicted moderation results in the following manner: for each category, obtain a second number of sub-content moderation models whose intermediate predicted moderation results indicate the category among the first number of sub-content moderation models; and determine the prediction difficulty score of the content to be moderated based on the first number and the second number corresponding to each category.

[0115] In practical applications, if multiple sub-content review models classify the content to be reviewed into the same category, it indicates that the content is easy to distinguish and has a low degree of uncertainty; conversely, if multiple sub-content review models classify the content into different categories, it indicates that the content is difficult to distinguish and has a high degree of uncertainty. Therefore, based on the first quantity and the second quantity corresponding to each category, the prediction difficulty score of the content to be reviewed can be determined using the following formula (II):

[0116] ; (two)

[0117] in, Indicates the difficulty score of the prediction. Indicates the first category, This indicates the intermediate predicted audit result indication for the content to be audited. The number of sub-content moderation models (i.e., the second quantity). This represents the total number of sub-content review models (i.e., the first number).

[0118] In practical applications, the server can also determine the predicted difficulty score of the content to be reviewed based on the first quantity and the second quantity corresponding to each category in the following way: For each category, determine the ratio of the second quantity to the first quantity corresponding to that category; determine the largest target ratio from the ratios corresponding to each category; obtain the correspondence between the largest ratio and the candidate predicted difficulty score (where the larger the largest ratio, the smaller the candidate predicted difficulty score); based on this correspondence, determine the candidate predicted difficulty score corresponding to the target ratio as the predicted difficulty score of the content to be reviewed.

[0119] Step 103: From at least one piece of content to be reviewed, select the target content that meets the difficulty score condition and obtain the actual review result of the target content.

[0120] In step 103, a higher prediction difficulty score indicates a greater difficulty in obtaining accurate prediction results for the content to be reviewed using the content review model. Simultaneously, it indicates a higher degree of uncertainty regarding the content to be reviewed, leading to less accurate prediction results. Therefore, based on this prediction difficulty score, content to be reviewed that meets the difficulty score criteria (e.g., content with a prediction difficulty score reaching a threshold, or content with prediction difficulty scores ranked high in descending order) can be selected as target content for review. The actual review result (i.e., the manual review result) of the target content is then obtained. This ensures the accuracy of content review and allows for training the content review model in step 104 using the target content as samples and the actual review results of the target content as sample labels, resulting in a target content review model with higher accuracy than the current model.

[0121] In some embodiments, the server may filter target content that meets the difficulty score condition by means of the following method: selecting content to be reviewed from at least one content to be reviewed that has a predicted difficulty score that reaches a first difficulty score threshold as target content to be reviewed.

[0122] In some embodiments, the server may also filter target review content that meets the difficulty score condition by the following method: sorting at least one piece of content to be reviewed in descending order based on the predicted difficulty score; selecting the top-ranked target number of content to be reviewed from the sorted content as target review content.

[0123] In some embodiments, when there are multiple target content items, after the server filters out target content items whose predicted difficulty scores meet the difficulty score conditions, it can further filter the obtained target content items before obtaining the actual review results, in order to reduce manual review costs. Specifically, the server can filter the obtained target content items in the following way: for each target content item, based on the predicted difficulty score of the target content item, determine the object review score of the target content item. This object review score is used to indicate the probability of object review of the target content item; determine the target content item whose object review score reaches the review score threshold as object review content; correspondingly, when the server obtains the actual review results of the target content item, it obtains the actual review results of the object review content; thus, when training the content review model, the object review content is used as the sample, and the actual review results of the object review content are used as the sample labels for the content review model.

[0124] In practical applications, when there are multiple items requiring review, to reduce manual review costs, the content can be filtered. For example, when the number of items reaches a threshold, further filtering can be performed. During this filtering, an object review score can be determined for each item. This score can be represented by the probability of performing an object review (i.e., manual review). This score threshold can be pre-set. When a determined object review score reaches the threshold, the item with that score is considered for manual review, yielding the actual review result.

[0125] Based on this, the server can obtain only the actual review results of the content to be reviewed, thereby further reducing the cost of manual review. Accordingly, the server uses the content to be reviewed as samples and the actual review results of the content to be reviewed as sample labels to train the content review model.

[0126] See Figure 5 , Figure 5This is a flowchart illustrating the training method of the content moderation model provided in this application embodiment, including: Step 301, selecting multiple target content items whose predicted difficulty scores meet the difficulty score conditions from at least one content item to be reviewed; Step 302, determining the object review score of each target content item based on its predicted difficulty score; Step 303, identifying target content items whose object review scores reach the review score threshold as object review content; Step 304, obtaining the actual review results of the object review content; Step 305, training the content moderation model using the object review content as samples and the actual review results of the object review content as sample labels to obtain the target content moderation model.

[0127] In some embodiments, the server may determine the object review score of the target review content in the following manner: when the predicted difficulty score is lower than the second difficulty score threshold, the product of the predicted difficulty score and the object review ratio of the object review is used as the object review score of the target review content; when the predicted difficulty score is not lower than the second difficulty score threshold, the object review score of the target review content is determined as the target score, which is used to indicate the operation of determining the target review content as object review content.

[0128] In practical applications, a second difficulty score threshold and an object review ratio (i.e., a manual sampling ratio) can be preset. It should be noted that this second difficulty score threshold can be greater than the first difficulty score threshold mentioned above. For target review content with a predicted difficulty score lower than the second difficulty score threshold, the product of the object review ratio and the predicted difficulty score can be used as the object review score for that target review content. For target review content with a predicted difficulty score not lower than the second difficulty score threshold, the object review score is determined as the target score, for example, 1. This target score is used to instruct the execution of the operation "determine the target review content with a predicted difficulty score not lower than the second difficulty score threshold as the target review content".

[0129] In practice, this can be achieved through the following formula (III): the target review score for the review content. for:

[0130] ; (three)

[0131] in, To predict the difficulty score, The ratio of random sampling for human review, This is the difficulty score threshold.

[0132] In some embodiments, the server can further control the cost of manual review based on time periods. The server can obtain the planned content review volume and the remaining content review volume for a target time period, and use the ratio of the remaining content review volume to the planned content review volume as the time period review ratio for the target time period. Here, the target time period can be 24 hours, 12 hours, one week, one month, etc.

[0133] Accordingly, the server can use the product of the predicted difficulty score, the object review ratio, and the time period review ratio as the object review score of the target content; the object review score of the target content is determined as the product of the target score and the time period review ratio.

[0134] In practical applications, taking a target time period of one day (24 hours) as an example, the number of manual reviews per day can be controlled to ensure the budget is covered and the review score of the target content is achieved. It can be calculated using the following formula (iv):

[0135] ; (Four)

[0136] Where B represents the daily planned content review volume, b represents the remaining content review volume for the day, and b / B is used to control the review volume to balance labor costs.

[0137] Step 104: Using the target content to be reviewed as a sample and the actual review results of the target content to be reviewed as the sample labels, train the content review model to update the content review model.

[0138] In step 104, the content moderation model is trained using the target content to be reviewed as samples and the actual review results of the target content as sample labels. The trained target content moderation model is then used to update the content moderation model, thereby updating the content moderation model and predicting the compliance of the content to be reviewed based on the updated model. This allows the content moderation model to be trained simultaneously with content review, eliminating the need for acquiring and labeling a large number of training samples. This reduces manual labeling costs and improves the training efficiency and speed of the content moderation model, enhancing its timeliness. It can be optimized and updated based on real-time content to be reviewed, giving the content moderation model better adaptability. Furthermore, a target content moderation model is obtained by training the content moderation model, and then the target content moderation model is used to update the content moderation model, enabling compliance prediction based on the updated model.

[0139] In some embodiments, see Figure 6 , Figure 6This is a flowchart illustrating the training method of the content moderation model provided in this application embodiment. Figure 6 It shows Figure 3 Step 104 can be implemented through steps 1041-1043: Step 1041, determine the first review content from the target review content, the predicted review result of the first review content is different from the actual review result of the first review content; Step 1042, perform data augmentation processing on the first review content to obtain the second review content, and use the actual review result of the first review content as the actual review result of the second review content; Step 1043, use the target review content and the second review content as samples, and use the actual review result of the target review content and the actual review result of the second review content as the sample labels of the samples to train the content review model and obtain the target content review model.

[0140] Here, data augmentation is performed on target content for which the predicted review results (from the content moderation model) differ from the actual human review results. This allows for thorough learning of the data with prediction discrepancies in the next stage. In practical applications, the data augmentation method can be determined based on the data modality (such as text, images, etc.). For example, text data augmentation methods could include resampling, synonym replacement, sentence order replacement, etc.; image data augmentation methods could include rotation, scaling, flipping, etc.

[0141] In some embodiments, see Figure 7 , Figure 7 This is a flowchart illustrating the training method of the content moderation model provided in this application embodiment. Figure 7 It shows Figure 3 Step 104 can also be implemented through steps 201-203: Step 201, obtain a first review content sample and a second review content sample, wherein the first review sample carries a label and the second review content sample does not carry a label; Step 202, using the target review content as a sample and the actual review result of the target review content as the sample label, and combining it with the first review content sample, train the content review model to obtain an intermediate content review model; Step 203, using the second review content sample, train the intermediate content review model for the target number of rounds to obtain the target content review model.

[0142] In step 104, a semi-supervised learning approach can be used to train the content moderation model. To expand the sample size of the training set, labeled first-level content samples can be obtained. Then, in step 202, the target content is used as the sample, and the actual review results of the target content are used as the sample labels. Combined with the first-level content samples, the content moderation model is trained to obtain an intermediate content moderation model. Then, in step 203, the intermediate content moderation model is iteratively trained using unlabeled second-level content samples to obtain the target content moderation model.

[0143] In some embodiments, the target number of rounds is k, where k is an integer greater than 0; step 203 can be implemented through steps 2031-2035: Step 2031, for the first round of training in k rounds, predict the second content review sample using the intermediate content review model to obtain a first prediction result, and use the first prediction result as the first sample label of the second content review sample; Step 2032, based on the second content review sample carrying the first sample label, train the intermediate content review model to obtain the first content review model trained in the first round of training in k rounds; Step 2033, for the nth round of training in k rounds, through the ( Step 2034: The first content moderation model trained in the (n-1)th round predicts the second content moderation sample to obtain the second prediction result, and uses the second prediction result as the second sample label of the second content moderation sample; Step 2035: Based on the second content moderation sample carrying the second sample label, the first content moderation model trained in the (n-1)th round is trained to obtain the first content moderation model trained in the (n-1)th round of training, where n is an integer greater than 1 and less than or equal to k; Step 2036: n is traversed to obtain the first content moderation model trained in the kth round, and the first content moderation model trained in the kth round is used as the target content moderation model.

[0144] In some embodiments, the intermediate content moderation model includes multiple sub-content moderation models. The server can predict the second content sample to obtain a first prediction result by using the intermediate content moderation model as follows: The second content sample is predicted using at least two target sub-content moderation models from the multiple sub-content moderation models, respectively, to obtain the sub-model prediction result of each target sub-content moderation model; the weight values ​​of each target sub-content moderation model are obtained; and a weighted average is performed on the sub-model prediction results of at least two target sub-content moderation models based on the weight values ​​of each target sub-content moderation model to obtain the first prediction result. This improves the labeling accuracy of unlabeled second content samples, thereby enhancing the effectiveness of model training using second content samples.

[0145] After training the content moderation model through the above steps to obtain the target content moderation model, the trained target content moderation model is used to update the content moderation model. The updated content moderation model is then used to predict the compliance of subsequently generated content to be reviewed. At the same time, steps 101-104 can be repeated to iteratively train and update the content moderation model during the content review process. This makes the content moderation model more and more accurate, reduces manual costs, and improves the training efficiency and speed of the content moderation model. It can be optimized and updated according to the real-time generated content to be reviewed, making the content moderation model more adaptable.

[0146] In some embodiments, the server may update the content moderation model using the target content moderation model as follows: obtain a test content moderation sample, the test content moderation sample carrying a test sample label; predict the test content moderation sample using the target content moderation model to obtain a test prediction result; determine the prediction accuracy score of the target content moderation model based on the test prediction result and the test sample label, the prediction accuracy score being used to indicate the accuracy of the test prediction result; when the prediction accuracy score exceeds an accuracy score threshold, update the content moderation model using the target content moderation model.

[0147] Applying the above embodiments of this application, firstly, the content moderation model performs content moderation on the content to be reviewed, and trains the content moderation model based on the target content to be reviewed within the content to be reviewed. In this way, the training of the content moderation model can be achieved simultaneously with content moderation, without the need to additionally construct the training set required for model training (including constructing samples and labeling sample tags), thus reducing model training costs.

[0148] Second, after the content to be reviewed is reviewed by the content moderation model, based on the predicted difficulty score of each piece of content to be reviewed by the content moderation model, target content that meets the difficulty score condition is selected from at least one piece of content to be reviewed. Thus, 1) in terms of content moderation, if the target content to be reviewed is manually reviewed to obtain the actual review results (i.e., manual review results), the amount of manual review can be reduced, and the cost of manual review can be lowered. That is, it can realize both machine review of the content to be reviewed and manual review of only the target content that meets the difficulty score condition. While reducing the amount of manual review, the cost of manual review is reduced and the efficiency of content moderation is improved through human-machine collaborative review; 2) in terms of model training, there is no need to label a large number of training samples. Only the target content that meets the difficulty score condition needs to be labeled, which improves the model training speed and timeliness.

[0149] Third, as the model training process progresses, the predictive ability of the obtained content review model becomes stronger and stronger, the prediction difficulty score for the content to be reviewed will decrease, and the target content to be reviewed based on the prediction difficulty score will become less and less, thereby further reducing the annotation cost (i.e., the target content to be reviewed that needs to obtain actual review results is reduced) and improving the model training speed.

[0150] Fourth, it can optimize and update the model based on the real-time content to be reviewed, making the trained content review model more flexible, adaptable to complex and ever-changing content to be reviewed, and improving the accuracy of model-based content review.

[0151] The following describes an exemplary application of the embodiments of this application in a real-world scenario. With the development and popularization of the internet, more and more content is presented to users via the internet. Content consumption products (such as news, music, short videos, long videos, and other media information) have become one of the primary channels for users to obtain information. This content contains a large amount of text, images, videos, audio, comments, and bullet comments. In practical applications, it is usually necessary to review the content to determine whether it complies with relevant regulations, thereby determining whether to proceed with content delivery, publication, or other operations.

[0152] In related technologies, human review and machine review are typically treated as two relatively separate parts. The process involves: summarizing human experience to develop labeling standards for machine review algorithms; organizing a labeling team to manually label data; and training the machine review algorithm model after data labeling. However, this approach suffers from several drawbacks: 1) the manual cost of developing and labeling data is enormous, and there is a lack of effective means to obtain a large number of target harmful samples (i.e., non-compliant content samples), thus requiring the labeling of a large amount of data; 2) the model iteration cycle is long, and it cannot efficiently optimize for uncertain samples; 3) the model threshold is difficult to determine, and the machine review algorithm model's definition of "gray area" (whether to submit to human review) is fixed, lacking a flexible way to balance the cost of human review and the risk of missed / false blocking. Consequently, related technologies often suffer from problems such as high training data labeling costs, difficulty in defining data labeling standards, poor offline model timeliness, difficulty in setting algorithm thresholds, and difficulty in balancing risk and cost.

[0153] Based on this, this application provides a training method for a content moderation model (i.e., a human-machine collaborative content moderation method based on active learning). It effectively integrates and unifies the lifecycles (training, iteration, online prediction) of human and machine moderation algorithms through active learning theory, thereby at least addressing the aforementioned problems. In this application embodiment, human and machine moderation are unified into a single framework. Human moderation results are used to supplement the training data of the machine moderation algorithm, and the machine moderation algorithm is used to select data requiring human review. This effectively balances the cost of human moderation and the risk of machine moderation, improving the response speed of the machine moderation algorithm model (i.e., the aforementioned content moderation model) to uncertain samples. In actual implementation, the machine moderation algorithm typically has three functions: 1) Blacklisting: identifying content to be reviewed containing harmful elements (i.e., non-compliant content) and prohibiting its publication; 2) Whitelisting: identifying content to be reviewed not containing harmful elements and releasing it; 3) Graylisting: identifying content to be reviewed that may contain harmful elements and sending it for human review, at which point human review costs are incurred.

[0154] See below. Figure 8 , Figure 8 This is a schematic diagram of the human-machine collaborative content moderation framework provided in this application embodiment. The human-machine collaborative content moderation framework includes: (1) an online machine moderation module; (2) a human-machine collaboration module; and (3) a model iteration module. Each module will be described in detail below.

[0155] (1) Online Machine Review Module: This is the core module for online machine review inference. It is responsible for predicting the compliance (i.e., the likelihood of harm) of real-time content awaiting review that is accessed online, and processing non-compliant content (i.e., harmful data) to prevent risk leakage. The processing flow of the online machine review module includes: (a) model cold start; (b) accessing real-time content awaiting review; (c) model prediction; and (d) model deployment and model quality inspection. Among them,

[0156] (a) Model Cold Start (i.e., the construction phase of the first version of the content moderation model). To quickly and effectively cold start the content moderation model, a training dataset was built using historical content as samples and corresponding human review results as labels to train the initial content moderation model. Here, the advantages of using human review results as labels are: 1) It simplifies the standardization process of human experience and omits the definition of label standards; 2) It simplifies the data acquisition process, acquiring a large amount of sample data while performing necessary human review work; 3) It automates data acquisition and allows for continuous streaming supplementation of training samples.

[0157] In practical applications, considering that content samples (such as samples containing non-compliant content) are usually complex and diverse, this application embodiment can construct a content moderation model through ensemble learning, that is, define the content moderation model as including multiple sub-content moderation models, namely... In practice, the choice of content moderation model can be determined based on the data modality (such as text, image, video, etc.). For example, text can be modeled using network models such as BERT, Albert, Electra, and Enrie, while images or videos can be modeled using network models such as Inception, VGG, and ResNet.

[0158] (b) Accessing real-time content awaiting review. Here, the content awaiting review generated in real time is accessed. Since the subsequent model prediction process relies on computing resources, and the real-time data stream in the business (i.e., the number of content awaiting review) often fluctuates periodically, it is necessary to limit the accessed real-time data stream so that the review prediction algorithm can be called based on the limited real-time data stream.

[0159] (c) Model Prediction. To improve the efficiency of the algorithm's prediction and make reasonable use of computing resources, a batch processing approach is adopted for predicting the content to be reviewed. When each batch contains n pieces of content to be reviewed and c sub-content review models, the content review model can predict n pieces of content to be reviewed at once. Each sub-content review model... For each of the n pending items, a compliance prediction is performed. For each pending item, the content moderation model predicts whether it is the nth item. The probability of a category can be represented in the following way (v):

[0160] ; (five)

[0161] in, For sub-content moderation model (for classification model) The prediction accuracy score (i.e., accuracy rate); To pass The content to be reviewed is predicted to be the first... The possibilities of categories.

[0162] In practical applications, this content moderation model can be used for the following three categories: Category 1: Blacklist, which identifies that the content to be reviewed contains harmful elements (i.e., non-compliant content); Category 2: Whitelist, which identifies that the content to be reviewed does not contain harmful elements; Category 3: It identifies that the content to be reviewed may contain harmful elements and needs to be sent for manual review.

[0163] (d) Model Deployment and Quality Inspection. After each iteration of the content review model, a test sample set is randomly selected from the historical content samples carrying the human review results. The new content review model obtained from the iteration is used to predict the compliance (i.e., the likelihood of harm) of the test sample set, and the prediction accuracy score (e.g., accuracy rate) of the new content review model is calculated. When the prediction accuracy score exceeds the specified score threshold, the new content review model is automatically deployed; otherwise, deployment is abandoned.

[0164] (2) Human-Machine Collaboration Module: This module plays a crucial role in balancing risk and cost, ensuring rapid and efficient model iteration through the reasonable selection of manually reviewed data, and transforming human experience into knowledge. The Human-Machine Collaboration Module includes: (a) Human Review Trigger; (b) Cost Controller; and (c) Human Review. Among these,

[0165] (a) Human review trigger. By using query functions in active learning, the most beneficial target review content for model iteration and review error prevention can be determined from a massive amount of content to be reviewed, thereby reducing human review costs and accelerating model iteration and optimization of uncertain samples.

[0166] In practical applications, entropy can be used to measure the uncertainty of a content moderation model in predicting content to be reviewed and obtaining accurate prediction results. A higher entropy indicates greater uncertainty, while a lower entropy indicates less uncertainty. Therefore, in single-model binary or multi-class classification scenarios, content with higher entropy can be selected as the target content for manual review. The entropy is calculated as shown in Formula (I):

[0167] ; (one)

[0168] in, For entropy, To pass Predicted content to be reviewed For the first The likelihood of a category (i.e., the predicted audit outcome).

[0169] Therefore, when extended to scenarios with multiple sub-content moderation models, entropy can be used to measure the difficulty of obtaining accurate prediction results for content to be reviewed by multiple sub-content moderation models. If multiple sub-content moderation models classify the content to be reviewed into the same category, it means that the content to be reviewed is easy to distinguish and the degree of uncertainty is small; if multiple sub-content moderation models classify the content to be reviewed into different categories, it means that the content to be reviewed is difficult to distinguish and the degree of uncertainty is large. Thus, the difficulty of obtaining accurate prediction results for content to be reviewed using a content moderation model (including multiple sub-content moderation models) (i.e., the prediction difficulty score mentioned above, represented by entropy) can be determined by the following formula (II):

[0170] ; (two)

[0171] in, Indicates the difficulty score of the prediction. Indicates the first category, This indicates the predicted review result for the content to be reviewed. The number of sub-content moderation models, This represents the total number of sub-content moderation models, i.e. .

[0172] You can either manually review content that has reached the predicted difficulty score threshold from multiple pending submissions, or you can manually review content with the highest predicted difficulty score.

[0173] (b) Cost Controller. When selecting target review content for manual review, the cost of manual review needs to be considered. The probability of submitting the content to be reviewed can be expressed by the following formula (iii):

[0174] ; (three)

[0175] in, The ratio of random sampling for human review, The above difficulty score threshold is used.

[0176] In practical applications, the number of manual reviews per day can also be controlled to ensure budget availability. The probability of submitting the content to be reviewed can be expressed by the following formula (iv):

[0177] ; (Four)

[0178] Where B is the total number of human reviewers per day, b is the remaining number of human reviewers for the day, and b / B is used to control the number of reviewers to balance labor costs.

[0179] c) Manual Review. Human review experts review the submitted content for review, obtaining the human review results. Then, using the target content as a sample and the corresponding human review results as sample labels, a human review sample is constructed for iterative updates to the content review model. In practical applications, if there are high timeliness requirements for this stage, the process can be broken down into sub-processes. For example, in the initial review stage, harmful content can be reviewed first, and harmless content can be released externally. Then, in the secondary review stage, harmful content can be annotated in detail to meet the model training requirements.

[0180] (3) Model Iteration Module: The content review model is trained based on human review samples. To facilitate rapid model iteration, incremental learning or model fine-tuning techniques are used to fine-tune the model and update it quickly. The model iteration module includes: (a) sample backflow and data augmentation; (b) supervised sample extraction and one round of model iteration; (c) unsupervised sample extraction and k rounds of model iteration; (d) model distillation. Among them,

[0181] (a) Sample Feedback and Data Augmentation. The human review samples output by the human-machine collaboration module (constructed using the target review content as the sample and the corresponding human review results as the sample labels) are fed back for data augmentation. Simultaneously, data augmentation is performed on the target review content whose predicted review results (from the content review model) differ from the human review results, so that the data with prediction deviations can be fully learned in the next stage. In practical applications, the data augmentation method can be determined according to the data modality (e.g., text, image, etc.). For example, text data augmentation methods can include resampling, synonym replacement, sentence order replacement, etc.; image data augmentation methods can include rotation, scaling, flipping, etc.

[0182] (b) Supervised sample extraction and one round of model iteration. Generally speaking, considering the cost of human review, the proportion of human-reviewed samples is relatively small. How to maximize the value of the limited human-reviewed samples is a challenge. In the embodiments of this application, a scheme combining semi-supervised learning and distillation learning is proposed.

[0183] See Figure 9 , Figure 9 This is a training diagram of the content moderation model provided in an embodiment of this application. Here, it is assumed that the content moderation model includes four sub-content moderation models. That is, C=4. During the first round of model iteration, in order to make full use of the human review samples d, the parts of the human review samples with conflicts (inconsistencies between predicted review results and human review results) can first be data augmented. Then, the content review model is trained for the first iteration by combining the data augmented human review samples and the labeled supervision samples in the historical content samples. Then, the four sub-content review models are fine-tuned according to the sample labels in the human review samples to obtain the sub-intermediate content review models obtained in the first round of iteration. .

[0184] c) Unsupervised sample extraction and k-round model iteration. Here, during the k-round model iteration, a massive amount of unlabeled historical content samples are randomly acquired, and the sub-intermediate content moderation models trained in the first round are iterated. In practice, at least two sub-intermediate content moderation models can be randomly selected to predict the massive amount of unsupervised samples, and the weighted average of the prediction results of these at least two sub-intermediate content moderation models (the weights can be the accuracy of the sub-intermediate content moderation models) is used as the label (i.e., soft label) of the unsupervised sample to obtain a new training dataset.

[0185] d) Model distillation. Here, during k rounds of model iteration, fine-tuning and model distillation are performed using a training dataset with soft labels. This process is repeated several times to perform knowledge distillation from a limited set of human review samples, resulting in a content moderation model with better generalization capabilities. See also... Figure 9 For example, adopting a sub-intermediate content review model. and Label the unsupervised samples to obtain new samples. Adopting a sub-intermediate content review model and Label the unsupervised samples to obtain new samples. Adopting a sub-intermediate content review model and Label the unsupervised samples to obtain new samples. Adopting a sub-intermediate content review model and Label the unsupervised samples to obtain new samples. Therefore, during the k rounds of model iteration, new samples are used... , , as well as Pair-to-Pair Content Review Model The first round of model distillation yields the sub-intermediate content review model. This process continues until the Kth round of model distillation is completed based on unsupervised samples, resulting in the final content moderation model. .

[0186] See above for further details. Figure 8 The processing flow of the human-machine collaborative content moderation framework provided in this application embodiment can be as follows: 1. Model cold start to obtain a content moderation model; 2. Using the content moderation model, perform model prediction on the real-time generated content to be reviewed to obtain a predicted review result; 3. Based on the predicted review result, determine whether to trigger manual review of the content to be reviewed through a human review trigger; 4. Determine the target content to be manually reviewed through a cost controller; 5. Perform manual review on the target content to obtain a manual review result; 6. Using the target content to be reviewed as a sample, and the manual review result of the target content to be reviewed as the sample label, construct a human review sample and perform sample feedback; 7. Perform data augmentation processing on the parts of the human review sample that have conflicts (inconsistencies between the predicted review result and the manual review result), to obtain... 8. Extract supervised samples from historical data and iterate the content moderation model once using the data-augmented human review samples and the extracted supervised samples; 9. Extract unsupervised samples from historical data and label the unsupervised samples using the content moderation model obtained from the first iteration to obtain new samples with labels (i.e., soft-labels); 10. Distill the content moderation model obtained from the first iteration using the new samples with labels to complete k iterations and obtain the target content moderation model; 11. Perform quality checks on the target content moderation model and obtain the quality check results; 12. If the quality check results indicate that the quality check is passed, the model is launched, that is, the content moderation model is updated using the target content moderation model to obtain the updated content moderation model. In practical applications, based on the real-time generated content to be reviewed, the above steps can be repeated to update the content moderation model multiple times to gradually obtain a content moderation model with more accurate review precision.

[0187] Therefore, the training logic for this content moderation model can be as follows. Wherein, the input is the human moderation sample d, and the content moderation model is... Assume the maximum number of iterations is m.

[0188] For i=1 to k do; / / Iterate through rounds 1 to k;

[0189] if i == 1; / / If it's the first iteration;

[0190] Training set = d; / / The training set uses human-reviewed samples d;

[0191] else

[0192] Training set = Di; / / If this is not the first iteration, the training set uses sample Di;

[0193] For j = 1 to c do; / / For model 1 to model c;

[0194] Based on training set pairs Conduct training;

[0195] End for; / / End the loop;

[0196] For j = 1 to c do; / / For model 1 to model c;

[0197] Randomly select several models to generate Di for unsupervised sample labeling, in order to perform model distillation;

[0198] End for; / / End the loop;

[0199] End for; / / End the loop;

[0200] For j = 1 to c do; / / For model 1 to model c;

[0201] D is generated by unsupervised sample labeling of all models. f To perform model distillation;

[0202] End for; / / End the loop and train to generate the final model.

[0203] In practical applications, there are various ways to select human review samples, such as uncertainty sampling query, committee-based query, model change expectation-based query, error reduction-based query, variance reduction-based query, density weight-based query, etc. For model updates after human review sample feedback, in addition to semi-supervised distillation methods, online learning and other technologies can also be used.

[0204] In practical applications, the human-machine collaborative content review method provided in this application embodiment can be applied to the following content review scenarios: 1) In terms of content risk control review, this method can effectively combine human review and machine review, balance costs and risks, and accelerate the optimization of uncertain samples. 2) In terms of user data review, similar to content risk control scenarios, this method can also reduce the workload of reviewing issues such as low-quality accounts, avatars, and data, and accelerate model iteration and improvement. 3) In other general review aspects, through reasonable modeling, this method can play a similar role in the review process of various scenarios such as loan risk control, expense reimbursement risk control, intelligent operation and maintenance risk control, and public opinion risk control.

[0205] The above embodiments mainly involve two main entities: machine review algorithm model and human review. Around these two entities, there are multiple data flow actions, and the main technologies involved include: active learning, ensemble learning, semi-supervised learning, distillation learning, incremental learning, etc.

[0206] By applying the embodiments described above, the content review process can effectively balance the costs of human review and the risk of omissions, while reducing the development costs of machine review models, demonstrating broad practical application prospects. In the security review of text, images, and interactive content, the solution provided by these embodiments can reduce the overall number of labeled samples from hundreds of thousands to thousands, and improve the accuracy of model review in multiple review processes.

[0207] It is understood that in the embodiments of this application, data such as user information are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0208] The following description continues to illustrate the exemplary structure of the training device 553 for the content moderation model provided in this application embodiment as a software module. In some embodiments, such as... Figure 2 As shown, the software modules in the training device 553 of the content moderation model stored in the memory 550 may include: a prediction module 5531, used to predict the compliance of at least one piece of content to be reviewed through the content moderation model, and obtain the predicted review result of each piece of content to be reviewed; a determination module 5532, used to determine the prediction difficulty score of each piece of content to be reviewed based on the predicted review result; wherein the prediction difficulty score is used to indicate the difficulty of predicting the content to be reviewed through the content moderation model to obtain an accurate prediction result; a filtering module 5533, used to filter target content that meets the difficulty score condition from the at least one piece of content to be reviewed, and obtain the actual review result of the target content; and a training module 5534, used to train the content moderation model with the target content as a sample and the actual review result of the target content as the sample label of the sample, so as to update the content moderation model.

[0209] In some embodiments, the prediction module 5531 is further configured to obtain the generation time of each of the contents to be reviewed; input the at least one content to be reviewed into the content review model in batches according to the order of the generation time; and perform compliance prediction on each batch of contents to be reviewed in sequence through the content review model to obtain the predicted review result of each content to be reviewed, wherein the batch includes a target number of contents to be reviewed.

[0210] In some embodiments, the content moderation model includes multiple sub-content moderation models, and the prediction module 5531 is further configured to perform the following processing for each of the contents to be reviewed: perform compliance prediction on the contents to be reviewed using each of the sub-content moderation models to obtain intermediate prediction review results of each sub-content moderation model; obtain the prediction accuracy score of each sub-content moderation model for each sub-content moderation model, and use the prediction accuracy score as the weight value of the sub-content moderation model; wherein, the prediction accuracy score is used to indicate the accuracy of the intermediate prediction review results of the corresponding sub-content moderation model; and perform weighted processing on the intermediate prediction review results of the multiple sub-content moderation models based on the weight values ​​of each sub-content moderation model to obtain the prediction review result of the contents to be reviewed.

[0211] In some embodiments, the content moderation model is used to perform classification prediction for at least two categories, and the predicted moderation result includes: the probability value of the content to be reviewed belonging to each of the categories; the determining module 5532 is further used to determine the information entropy value corresponding to the content to be reviewed based on the probability value of the content to be reviewed belonging to each of the categories, and use the information entropy value as the prediction difficulty score of the content to be reviewed.

[0212] In some embodiments, the content moderation model includes a first number of sub-content moderation models, and the predicted moderation result includes intermediate predicted moderation results of each of the sub-content moderation models. The sub-content moderation models are used to perform classification prediction for at least two categories. The determining module 5532 is further configured to, for each category, obtain a second number of sub-content moderation models whose intermediate predicted moderation results indicate the category among the first number of sub-content moderation models; and determine the prediction difficulty score of the content to be moderated based on the first number and the second number corresponding to each category.

[0213] In some embodiments, the filtering module 5533 is further configured to select, from the at least one piece of content to be reviewed, the content to be reviewed whose predicted difficulty score reaches a first difficulty score threshold as the target content to be reviewed.

[0214] In some embodiments, the filtering module 5533 is further configured to sort the at least one piece of content to be reviewed in descending order based on the predicted difficulty score; and select the target number of pieces of content to be reviewed that are ranked higher from the sorted at least one piece of content to be reviewed as the target content to be reviewed.

[0215] In some embodiments, when there are multiple target content items, the filtering module 5533 is further configured to determine the object review score of each target content item based on its predicted difficulty score, wherein the object review score indicates the likelihood of object review of the target content item; and to determine target content items whose object review scores reach a review score threshold as object review content. Correspondingly, the filtering module 5533 is also configured to obtain the actual review results of the object review content. Correspondingly, the training module 5534 is also configured to train the content review model using the object review content as samples and the actual review results of the object review content as the sample labels of the samples.

[0216] In some embodiments, the filtering module 5533 is further configured to, when the predicted difficulty score is lower than the second difficulty score threshold, use the product of the predicted difficulty score and the object review ratio of the object review as the object review score of the target review content; and when the predicted difficulty score is not lower than the second difficulty score threshold, determine the object review score of the target review content as the target score, wherein the target score is used to indicate the operation of determining the target review content as the object review content.

[0217] In some embodiments, the filtering module 5533 is further configured to obtain the planned content review volume and the remaining content review volume for a target time period, and use the ratio of the remaining content review volume to the planned content review volume as the time period review ratio for the target time period; correspondingly, the filtering module 5533 is further configured to use the product of the predicted difficulty score, the object review ratio, and the time period review ratio as the object review score of the target review content; correspondingly, the filtering module 5533 is further configured to determine the object review score of the target review content as the product of the target score and the time period review ratio.

[0218] In some embodiments, the training module 5534 is further configured to determine a first review content from the target review content, wherein the predicted review result of the first review content is different from the actual review result of the first review content; perform data augmentation processing on the first review content to obtain a second review content, and use the actual review result of the first review content as the actual review result of the second review content; train the content review model using the target review content and the second review content as the samples, and using the actual review result of the target review content and the actual review result of the second review content as the sample labels of the samples.

[0219] In some embodiments, the training module 5534 is further configured to obtain a first review content sample and a second review content sample, wherein the first review sample carries a label and the second review content sample does not carry a label; using the target review content as a sample and the actual review result of the target review content as the sample label of the sample, and combining the first review content sample, the content review model is trained to obtain an intermediate content review model; using the second review content sample, the intermediate content review model is trained for a target number of rounds.

[0220] In some embodiments, the target number of rounds is k, where k is an integer greater than 0. The training module 5534 is further configured to: predict the second content review sample using the intermediate content review model for the first round of training in the k rounds, obtain a first prediction result, and use the first prediction result as the first sample label of the second content review sample; train the intermediate content review model based on the second content review sample carrying the first sample label to obtain the first content review model trained in the first round of training in the k rounds; predict the second content review sample using the first content review model trained in the (n-1)th round of training in the k rounds, obtain a second prediction result, and use the second prediction result as the second sample label of the second content review sample; train the first content review model trained in the (n-1)th round based on the second content review sample carrying the second sample label to obtain the first content review model trained in the (n-1)th round of training in the k rounds, where n is an integer greater than 1 and less than or equal to k; and iterate through n to obtain the first content review model trained in the kth round.

[0221] In some embodiments, the intermediate content moderation model includes multiple sub-content moderation models. The training module 5534 is further configured to predict the second content sample to be reviewed by at least two target sub-content moderation models among the multiple sub-content moderation models, thereby obtaining the sub-model prediction results of each target sub-content moderation model; obtain the weight values ​​of each target sub-content moderation model; and perform a weighted average of the sub-model prediction results of at least two target sub-content moderation models based on the weight values ​​of each target sub-content moderation model to obtain the first prediction result.

[0222] In some embodiments, the training module 5534 is further configured to acquire test content samples, the test content samples carrying test sample labels; predict the test content samples using a target content review model obtained by training the content review model, and obtain test prediction results; determine the prediction accuracy score of the target content review model based on the test prediction results and the test sample labels, the prediction accuracy score indicating the accuracy of the test prediction results; and update the content review model using the target content review model when the prediction accuracy score exceeds an accuracy score threshold.

[0223] Applying the above embodiments of this application, firstly, the content moderation model performs content moderation on the content to be reviewed, and trains the content moderation model based on the target content to be reviewed within the content to be reviewed. In this way, the training of the content moderation model can be achieved simultaneously with content moderation, without the need to additionally construct the training set required for model training (including constructing samples and labeling sample tags), thus reducing model training costs.

[0224] Second, after the content to be reviewed is reviewed by the content moderation model, based on the predicted difficulty score of each piece of content to be reviewed by the content moderation model, target content that meets the difficulty score condition is selected from at least one piece of content to be reviewed. Thus, 1) in terms of content moderation, if the target content to be reviewed is manually reviewed to obtain the actual review results (i.e., manual review results), the amount of manual review can be reduced, and the cost of manual review can be lowered. That is, it can realize both machine review of the content to be reviewed and manual review of only the target content that meets the difficulty score condition. While reducing the amount of manual review, the cost of manual review is reduced and the efficiency of content moderation is improved through human-machine collaborative review; 2) in terms of model training, there is no need to label a large number of training samples. Only the target content that meets the difficulty score condition needs to be labeled, which improves the model training speed and timeliness.

[0225] Third, as the model training process progresses, the predictive ability of the obtained content review model becomes stronger and stronger, the prediction difficulty score for the content to be reviewed will decrease, and the target content to be reviewed based on the prediction difficulty score will become less and less, thereby further reducing the annotation cost (i.e., the target content to be reviewed that needs to obtain actual review results is reduced) and improving the model training speed.

[0226] Fourth, it can optimize and update the model based on the real-time content to be reviewed, making the trained content review model more flexible, adaptable to complex and ever-changing content to be reviewed, and improving the accuracy of model-based content review.

[0227] This application also provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the training method for the content moderation model provided in this application.

[0228] This application also provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are executed by a processor, the processor will execute the training method for the content moderation model provided in this application.

[0229] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0230] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0231] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0232] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0233] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A training method for a content moderation model, characterized in that, The method includes: The content review model is used to predict the compliance of at least one piece of content to be reviewed, and the predicted review result of each piece of content to be reviewed is obtained; wherein, the content review model includes a first number of sub-content review models, the predicted review result includes the intermediate predicted review result of each sub-content review model, and the sub-content review model is used to perform classification prediction of at least two categories; For each of the contents to be reviewed, in the first number of sub-content review models, the intermediate predicted review result indicates the second number of sub-content review models for each category; Based on the ratio of the second quantity to the first quantity corresponding to each category, the information entropy corresponding to the content to be reviewed is calculated, and the value of the information entropy is used as the prediction difficulty score of the content to be reviewed; wherein, the prediction difficulty score is used to indicate the degree of difficulty in predicting the content to be reviewed through the content review model to obtain an accurate prediction result; From the at least one piece of content to be reviewed, target content that meets the predicted difficulty score condition is selected for review, and the actual review result of the target content is obtained. From the target audit content, determine the first audit content whose predicted audit result is inconsistent with the actual audit result; perform data augmentation processing on the first audit content to obtain the second audit content, and use the actual audit result of the first audit content as the actual audit result of the second audit content; Using the target content to be reviewed and the second content to be reviewed as samples, and using the actual review results of the target content to be reviewed and the actual review results of the second content to be reviewed as sample labels, the content review model is trained and updated.

2. The method of claim 1, wherein, The step of using a content moderation model to predict the compliance of at least one piece of content to be reviewed, and obtaining the predicted review results for each piece of content to be reviewed, includes: Obtain the time point at which each of the items to be reviewed was generated; According to the order of their generation time, the at least one piece of content to be reviewed is input into the content review model in batches; The content review model is used to sequentially predict the compliance of each batch of content to be reviewed, thereby obtaining the predicted review results for each batch of content to be reviewed, wherein the batch includes a target number of content to be reviewed.

3. The method of claim 1, wherein, The step of using a content moderation model to predict the compliance of at least one piece of content to be reviewed, and obtaining the predicted review results for each piece of content to be reviewed, includes: For each of the items pending review, the following processing shall be performed: By using each of the sub-content review models, compliance prediction is performed on the content to be reviewed, and intermediate prediction review results of each of the sub-content review models are obtained. For each of the sub-content review models, the prediction accuracy score of the sub-content review model is obtained, and the prediction accuracy score is used as the weight value of the sub-content review model. The prediction accuracy score is used to indicate the accuracy of the intermediate prediction review results of the corresponding sub-content review model. Based on the weight values ​​of each of the sub-content review models, the intermediate predicted review results of the multiple sub-content review models are weighted to obtain the predicted review result of the content to be reviewed.

4. The method of claim 1, wherein, The step of selecting target content for review from the at least one piece of content to be reviewed, which meets the predicted difficulty score criteria, includes: From the at least one piece of content to be reviewed, select the content whose predicted difficulty score reaches the first difficulty score threshold as the target content to be reviewed.

5. The method as described in claim 1, characterized in that, The step of selecting target content for review from the at least one piece of content to be reviewed, which meets the predicted difficulty score criteria, includes: Based on the predicted difficulty score, the at least one piece of content to be reviewed is sorted in descending order; From the sorted list of at least one item to be reviewed, select the target number of items to be reviewed that are ranked highest as the target review items.

6. The method of claim 1, wherein, When the number of target review items is multiple, before obtaining the actual review result of the target review items, the method further includes: For each of the target review contents, based on the predicted difficulty score of the target review contents, an object review score is determined for the target review contents, and the object review score is used to indicate the degree of likelihood of object review of the target review contents; The target review content that reaches the review score threshold for the object is defined as the object review content. The step of obtaining the actual review result of the target review content includes: obtaining the actual review result of the object review content; The step of training the content review model using the target review content and the second review content as samples, and using the actual review results of the target review content and the actual review results of the second review content as sample labels for the samples, includes: Using the object review content and the second review content as samples, and using the actual review results of the object review content and the actual review results of the second review content as sample labels, the content review model is trained.

7. The method of claim 6, wherein, The process of determining the object review score of the target review content based on the predicted difficulty score of the target review content includes: When the predicted difficulty score is lower than the second difficulty score threshold, the product of the predicted difficulty score and the preset object review ratio is used as the object review score of the target review content. When the predicted difficulty score is not lower than the second difficulty score threshold, the object review score of the target review content is determined as the target score. The target score is used to indicate the operation of determining the target review content as the object review content.

8. The method of claim 7, wherein, The method further includes: Obtain the planned content review volume and the remaining content review volume for the target time period, and use the ratio of the remaining content review volume to the planned content review volume as the time period review ratio for the target time period; The step of using the product of the predicted difficulty score and the preset object review ratio as the object review score of the target review content includes: The product of the predicted difficulty score, the object review ratio, and the time period review ratio is used as the object review score of the target review content. The determination of the target review score for the target review content as the target score includes: The target review score for the content to be reviewed is determined as the product of the target score and the review ratio for the specified time period.

9. The method as described in claim 1, characterized in that, The step of training the content review model using the target review content and the second review content as samples, and using the actual review results of the target review content and the actual review results of the second review content as sample labels for the samples, includes: Obtain a first review content sample and a second review content sample. The first review content sample carries a tag, while the second review content sample does not carry a tag. Using the target review content and the second review content as samples, and the actual review results of the target review content and the actual review results of the second review content as sample labels for the samples, the content review model is trained in conjunction with the first review content sample to obtain an intermediate content review model; Using the second sample of content to be reviewed, the intermediate content review model is trained for the target number of rounds.

10. The method of claim 9, wherein, The target number of rounds is k, where k is an integer greater than 0. The step of training the intermediate content review model to the target number of rounds using the second review content sample includes: For the first round of training in the k rounds of training, the second content review sample is predicted by the intermediate content review model to obtain a first prediction result, and the first prediction result is used as the first sample label of the second content review sample; Based on the second content sample carrying the first sample label, the intermediate content review model is trained to obtain the first content review model trained in the first round of k rounds of training. For the nth round of training in the k rounds, the second content review sample is predicted by the first content review model trained in the (n-1)th round to obtain the second prediction result, and the second prediction result is used as the second sample label of the second content review sample; Based on the second content sample carrying the second sample label, the first content review model in the (n-1)th round of training is trained to obtain the first content review model in the (n-1)th round of training in the k rounds of training, where n is an integer greater than 1 and less than or equal to k; By iterating through n, we obtain the first content moderation model trained in the kth round.

11. The method of claim 1, wherein, After training the content review model using the target review content and the second review content as samples, and using the actual review results of the target review content and the actual review results of the second review content as sample labels for the samples, the method further includes: Obtain a sample of test audit content, wherein the test audit content sample carries a test sample tag; The target content moderation model obtained by training the content moderation model is used to predict the test content moderation sample to obtain the test prediction result; Based on the test prediction results and the test sample labels, the prediction accuracy score of the target content review model is determined, and the prediction accuracy score is used to indicate the accuracy of the test prediction results. When the predicted accuracy score exceeds the accuracy score threshold, the content moderation model is updated using the target content moderation model.

12. A training device for a content moderation model, characterized in that, The device includes: The prediction module is used to predict the compliance of at least one piece of content to be reviewed through a content review model, and obtain the predicted review result of each piece of content to be reviewed; wherein, the content review model includes a first number of sub-content review models, the predicted review result includes the intermediate predicted review result of each sub-content review model, and the sub-content review model is used to perform classification prediction of at least two categories; The determining module is used to, for each of the contents to be reviewed, obtain, in the first number of sub-content review models, the intermediate predicted review result indicating the second number of sub-content review models for each of the categories; Based on the ratio of the second quantity to the first quantity corresponding to each category, the information entropy corresponding to the content to be reviewed is calculated, and the value of the information entropy is used as the prediction difficulty score of the content to be reviewed; wherein, the prediction difficulty score is used to indicate the degree of difficulty in predicting the content to be reviewed through the content review model to obtain an accurate prediction result; The filtering module is used to filter out target review content that meets the predicted difficulty score condition from the at least one piece of content to be reviewed, and obtain the actual review result of the target review content; The training module is used to determine, from the target review content, a first review content whose predicted review result is inconsistent with the actual review result; perform data augmentation processing on the first review content to obtain a second review content; and use the actual review result of the first review content as the actual review result of the second review content. Using the target content to be reviewed and the second content to be reviewed as samples, and using the actual review results of the target content to be reviewed and the actual review results of the second content to be reviewed as sample labels, the content review model is trained and updated.

13. The apparatus according to claim 12, characterized in that, The prediction module is also used to obtain the generation time of each of the contents to be reviewed; According to the order of their generation time, the at least one piece of content to be reviewed is input into the content review model in batches; The content review model is used to sequentially predict the compliance of each batch of content to be reviewed, thereby obtaining the predicted review results for each batch of content to be reviewed, wherein the batch includes a target number of content to be reviewed.

14. The apparatus according to claim 12, characterized in that, The prediction module is also used to perform the following processing on each of the contents to be reviewed: By using each of the sub-content review models, compliance prediction is performed on the content to be reviewed, and intermediate prediction review results of each of the sub-content review models are obtained. For each of the sub-content review models, the prediction accuracy score of the sub-content review model is obtained, and the prediction accuracy score is used as the weight value of the sub-content review model. The prediction accuracy score is used to indicate the accuracy of the intermediate prediction review results of the corresponding sub-content review model. Based on the weight values ​​of each of the sub-content review models, the intermediate predicted review results of the multiple sub-content review models are weighted to obtain the predicted review result of the content to be reviewed.

15. The apparatus according to claim 12, characterized in that, The filtering module is further configured to select, from the at least one piece of content to be reviewed, the content whose predicted difficulty score reaches a first difficulty score threshold as the target content to be reviewed.

16. The apparatus according to claim 12, characterized in that, The filtering module is also used to sort the at least one piece of content to be reviewed in descending order based on the predicted difficulty score; From the sorted list of at least one item to be reviewed, select the target number of items to be reviewed that are ranked highest as the target review items.

17. The apparatus according to claim 12, characterized in that, The filtering module is further configured to determine the object review score of each target review content based on the predicted difficulty score of the target review content, wherein the object review score is used to indicate the degree of likelihood of object review of the target review content; The target review content that reaches the review score threshold for the object is defined as the object review content. The filtering module is also used to obtain the actual review results of the object's review content; The training module is further configured to train the content review model using the object review content and the second review content as samples, and using the actual review results of the object review content and the actual review results of the second review content as sample labels for the samples.

18. The apparatus according to claim 17, characterized in that, The filtering module is also used to, when the predicted difficulty score is lower than the second difficulty score threshold, use the product of the predicted difficulty score and the preset object review ratio as the object review score of the target review content. When the predicted difficulty score is not lower than the second difficulty score threshold, the object review score of the target review content is determined as the target score. The target score is used to indicate the operation of determining the target review content as the object review content.

19. The apparatus according to claim 18, characterized in that, The filtering module is also used to obtain the planned content review volume and the remaining content review volume for the target time period, and to use the ratio of the remaining content review volume to the planned content review volume as the time period review ratio for the target time period. The filtering module is also used to take the product of the predicted difficulty score, the object review ratio, and the time period review ratio as the object review score of the target review content; The filtering module is also used to determine the object review score of the target review content as the product of the target score and the review ratio of the time period.

20. An electronic device, comprising: The electronic device includes: Memory is used to store executable instructions for computation; A processor, when executing computer-executable instructions stored in the memory, implements the training method of the content moderation model according to any one of claims 1 to 11.

21. A computer-readable storage medium storing computer-executable instructions, wherein execution of the computer-executable instructions by one or more processors of a computing system causes the one or more processors to perform operations comprising: When the computer-executable instructions are executed by the processor, they implement the training method of the content moderation model according to any one of claims 1 to 11.

22. A computer program product comprising computer programs or computer executable instructions, characterized in that, When the computer program or computer-executable instructions are executed by the processor, the training method of the content moderation model according to any one of claims 1 to 11 is implemented.

Images