Training sample generation method, device, electronic device and storage medium

Abstract

The present disclosure provides a training sample generation method, device, electronic device and storage medium, which relate to the field of artificial intelligence technology, and in particular to the field of large model technology. The training sample generation method includes: obtaining audit rules and first sample data for a content audit model, wherein the first sample data is multimedia data; using the first large model to process a first prompt word obtained according to the audit rules and the first sample data to obtain second sample data; using the second large model to process a second prompt word obtained according to the audit rules, the first sample data and the second sample data to obtain a pre-audit category of the second sample data, wherein the pre-audit category indicates whether the second sample data complies with the audit rules; and auditing the pre-audit category through a processor. If the audit result meets a predetermined condition, the first sample data and the second sample data are used as training samples for training the content audit model.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to the field of large model technology. More specifically, the present disclosure provides a training sample generation method, a content review model training method, a content review method, an apparatus, an electronic device, and a storage medium. Background Art

[0002] With the rapid development of the internet and the surge in user content, content moderation has become an essential component of digital governance across social media platforms, online forums, news websites, and more. Effective content moderation not only helps maintain a healthy online ecosystem but also protects users from inappropriate or harmful content.

[0003] However, the diversity, complexity, and rapid evolution of inappropriate content pose significant challenges to traditional content moderation models, making them difficult to fully adapt and respond to. Improving the capabilities of content moderation models to ensure accurate results in this dynamic and ever-changing environment has become a pressing challenge. Summary of the Invention

[0004] The present disclosure provides a training sample generation method, a content review model training method, a content review method, an apparatus, an electronic device, and a storage medium.

[0005] According to one aspect of the present disclosure, a training sample generation method is provided, comprising: obtaining audit rules and first sample data for a content audit model, wherein the first sample data is multimedia data; using a first large model to process a first prompt word obtained according to the audit rules and the first sample data to obtain second sample data; using a second large model to process a second prompt word obtained according to the audit rules, the first sample data, and the second sample data to obtain a pre-audit category of the second sample data, wherein the pre-audit category characterizes whether the second sample data complies with the audit rules; auditing the pre-audit category through a processor, and when the audit result meets predetermined conditions, using the first sample data and the second sample data as training samples for training the content audit model.

[0006] According to another aspect of the present disclosure, a method for training a content review model is provided, comprising: training the content review model using training samples; wherein the training samples are generated using the training sample generation method as described above.

[0007] According to another aspect of the present disclosure, a content review method is provided, including: obtaining content to be reviewed, reviewing the content to be reviewed using a content review model, and obtaining a review category; wherein the content to be reviewed is multimedia data, and the content review model is trained using the content review model training method described above.

[0008] According to another aspect of the present disclosure, a training sample generating device is provided, including: a sample data acquisition module, used to obtain audit rules and first sample data for a content audit model, wherein the first sample data is multimedia data; a sample data enhancement module, used to use a first large model to process a first prompt word obtained according to the audit rules and the first sample data to obtain second sample data; an enhanced sample classification module, used to use a second large model to process a second prompt word obtained according to the audit rules, the first sample data and the second sample data to obtain a pre-audit category of the second sample data, wherein the pre-audit category represents whether the second sample data complies with the audit rules; a training sample determination module, used to audit the pre-audit category through a processor, and when the audit result meets predetermined conditions, use the first sample data and the second sample data as training samples for training the content audit model.

[0009] According to another aspect of the present disclosure, a training device for a content review model is provided, comprising: a model training module for training the content review model using training samples; wherein the training samples are generated using the training sample generation device as described above.

[0010] According to another aspect of the present disclosure, a content review device is provided, including: a content review module, used to obtain content to be reviewed, use a content review model to review the content to be reviewed, and obtain a review category; wherein the content to be reviewed is multimedia data, and the content review model is trained using the training device of the content review model as described above.

[0011] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described above.

[0012] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method described above.

[0013] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method described above when executed by a processor.

[0014] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0016] Figure 1 This is a schematic diagram of an exemplary application scenario in which the training sample generation method, content review model training method, content review method, and apparatus according to an embodiment of the present disclosure can be applied;

[0017] Figure 2 is a flowchart of a method for generating training samples according to an embodiment of the present disclosure;

[0018] Figure 3 is a schematic diagram of a training sample generation method according to an embodiment of the present disclosure;

[0019] Figure 4 is a flow chart for reviewing pre-review categories according to one embodiment of the present disclosure;

[0020] Figure 5 is a flowchart of a method for training a content moderation model according to one embodiment of the present disclosure;

[0021] Figure 6 is a flow chart of a content review method according to one embodiment of the present disclosure;

[0022] Figure 7 is a block diagram of a training sample generating apparatus according to an embodiment of the present disclosure;

[0023] Figure 8 is a block diagram of a training apparatus for a content moderation model according to one embodiment of the present disclosure;

[0024] Figure 9 is a block diagram of a content review apparatus according to an embodiment of the present disclosure;

[0025] Figure 10 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0026] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0027] In the technical solutions disclosed herein, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0028] In scenarios where personal information is used for automated decision-making, the methods, devices, and systems provided by the embodiments of the present disclosure all provide users with corresponding operation portals for them to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge, and skills, and have reached a certain level of professionalism.

[0029] In the development of artificial intelligence (AI) technology, the training effectiveness of content moderation models plays a crucial role in the accuracy of their results. The distribution of training data determines the effectiveness of content moderation models. Currently, inappropriate content takes many forms and is constantly evolving. Traditional data augmentation methods often struggle to fully capture these variations, resulting in insufficient generalization capabilities for content moderation models when dealing with new or evolving content.

[0030] In view of this, embodiments of the present disclosure provide a training sample generation method, a content moderation model training method, a content moderation method, an apparatus, an electronic device, and a storage medium, which can automatically generate diverse training samples of inappropriate content that are adapted to the development of the Internet, in order to train the content moderation model. In this way, the limitations of traditional data augmentation methods can be overcome, and the generalization ability and accuracy of the content moderation model can be improved.

[0031] Figure 1 This is a schematic diagram of an exemplary application scenario in which the training sample generation method, content audit model training method, content audit method and device according to an embodiment of the present disclosure can be applied. It should be noted that, Figure 1 The examples shown are merely examples of application scenarios in which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0032] like Figure 1As shown, the application scenario 100 according to this embodiment may include terminal devices 101 and 102 and a server 103 .

[0033] The terminal devices 101 and 102 may be communicatively connected to the server 102 via a network. The network may include various connection types, such as wired and / or wireless communication links, etc.

[0034] The terminal devices 101 and 102 may have a display screen, which may provide an interactive interface. Users may perform operations such as selection and editing on the interactive interface of the terminal devices 101 and 102 to define the review rules and first sample data for the content review model. The terminal devices 101 and 102 include but are not limited to smart phones, tablet computers, laptop computers, desktop computers, etc.

[0035] Server 103 may be a server that provides various services, such as a background management server (for example only) that performs enhancement of first sample data, category review, and verification of the enhanced sample data in response to user operations on the interactive interfaces of terminal devices 101 and 102. The background management server may obtain the review rules and first sample data defined by the user on terminal devices 101 and 102, and enhance the first sample data using prompt words and a large model to obtain second sample data. The server may also perform category pre-review and validity verification on the second sample data to expand the training samples of the content review model.

[0036] It should be noted that the training sample generation method, content audit model training method, and content audit method provided in the embodiments of the present disclosure can generally be executed by the server 103. Accordingly, the training sample generation device, content audit model training device, and content audit device provided in the embodiments of the present disclosure can generally be set in the server 103. The training sample generation method, content audit model training method, and content audit method provided in the embodiments of the present disclosure can also be executed by a server or server cluster that is different from the server 103 and can communicate with the terminal devices 101, 102 and / or the server 103. Accordingly, the training sample generation device, content audit model training device, and content audit device provided in the embodiments of the present disclosure can also be set in a server or server cluster that is different from the server 103 and can communicate with the terminal devices 101, 102 and / or the server 103.

[0037] It should be understood that Figure 1 The types and numbers of the terminal devices and servers are merely illustrative. Any types and numbers of terminal devices and servers may be used as required.

[0038] The following will be combined Figures 2 to 4 The training sample generation method provided in the present disclosure is described in detail.

[0039] Figure 2 is a flowchart of a training sample generation method according to an embodiment of the present disclosure.

[0040] like Figure 2 As shown, the training sample generating method 200 according to this embodiment may include operations S210 to S240.

[0041] In operation S210, an audit rule for a content audit model and first sample data are obtained, wherein the first sample data is multimedia data.

[0042] For example, a user may define audit rules and first sample data for a content audit model on a terminal device, and the server may receive the audit rules and first sample data.

[0043] For example, audit rules can include multiple audit categories defined by the user, as well as detailed discrimination criteria for each audit category. These audit categories and discrimination criteria serve as the basis for subsequent automatic data enhancement.

[0044] According to embodiments of the present disclosure, the multiple audit categories defined in the audit rules may differ according to different standards. For example, based on the degree of inappropriateness of the content to be audited, the audit categories may include normal, partially inappropriate, and completely inappropriate; based on the type of inappropriateness of the content to be audited, the audit categories may include inappropriate type A, inappropriate type B, inappropriate type C, and so on. Embodiments of the present disclosure do not limit the specific content of the audit categories.

[0045] For example, high-quality, representative, and typical data can be selected from a pre-set content review dataset as the first sample data. This first sample data will serve as the basis for generating new data, ensuring the category relevance of the data generated by the large model.

[0046] According to an embodiment of the present disclosure, the first sample data is multimedia data, which includes at least one of text, image, audio, and video. The content to be reviewed of the content review model corresponds to the type of the first sample data. For example, when the first sample data is text, the content review model can be the Wenxin model (ERNIE) or the Generative Pre-trained Transformer (GPT), etc. When the first sample data is an image, the content review model can be a Residual Network (ResNet) series model (such as ResNet-50), a Vision Transformer model (ViT), an End-to-End Object Detection with Transformers (DERT), etc. When the first sample data is a video, the content review model can be a Swin-Transformer model, etc.

[0047] According to an embodiment of the present disclosure, a content review model can be applied to social media platforms, online forums, news websites, etc., and the model can run in a central processing unit (CPU) or a graphics processing unit (GPU).

[0048] In operation S220, the first prompt word obtained according to the audit rule and the first sample data is processed using the first large model to obtain second sample data.

[0049] According to an embodiment of the present disclosure, the first prompt word can be a description text or instruction of the data enhancement task, so as to guide the first large model to accurately understand the task objective. For example, the first prompt word can include the audit rules and the first sample data, and can also include a guide word for guiding the first large model to enhance the first sample data according to the audit rules to generate new data, i.e., the second sample data.

[0050] According to embodiments of the present disclosure, a large model may be an artificial intelligence (AI) large model, which is a machine learning model with extremely large parameters and complex computational structures. AI large models can process massive amounts of data and complete various complex tasks, such as natural language processing, image recognition, and computer vision. AI large models may include language large models, vision large models, and multimodal large models.

[0051] According to an embodiment of the present disclosure, prompt words are used as input to the corresponding large model to guide the large model to generate output. Prompt words are an important means of interacting with the large model. They can be text or instructions that provide input to the large model to guide it to generate specific output. Prompt words can be text paragraphs provided by the user when interacting with the large model, used to describe the information, answers, text, etc. that the user wants to obtain from the large model. The purpose of prompt words is to guide the large model to produce the required response in order to better control the generated output. Prompt words can also be parameter descriptions in a certain format.

[0052] According to an embodiment of the present disclosure, a first prompt word is first obtained based on the review rules and the first sample data, and then the first prompt word is input into the first large model to obtain the second sample data. The second sample data is different from the first sample data. It is new data automatically generated under the guidance of the first prompt word using the powerful language understanding and generation capabilities of the first large model. It is new data that is amplified compared to the first sample data. Compared with the first sample data, the second sample data is richer in expression. It can also adjust the task description of the first prompt word so that the first large model can generate variants that simulate inappropriate content in the real world, such as synonyms, antonyms, sentence reorganization, random insertion, deletion, or exchange of word order, etc.

[0053] In operation S230 , the second prompt word obtained according to the audit rule, the first sample data, and the second sample data is processed using the second large model to obtain a pre-audit category of the second sample data, wherein the pre-audit category indicates whether the second sample data complies with the audit rule.

[0054] According to an embodiment of the present disclosure, both the first large model and the second large model may run in a CPU or a GPU, and the first large model and the second large model may be the same or different.

[0055] According to the embodiments of the present disclosure, a second prompt word is first derived based on the review rules, the first sample data, and the second sample data. The second prompt word is then input into the second large model to obtain a pre-review category for the second sample data. This pre-review category is automatically generated under the guidance of the second prompt word, leveraging the powerful language understanding and generation capabilities of the second large model.

[0056] According to an embodiment of the present disclosure, the second prompt word can be a description text or instruction of the category pre-audit task, so as to guide the second large model to accurately understand the task goal. For example, the second prompt word can include according to the audit rules, the first sample data and the second sample data, and can also include a guide word for guiding the second large model to perform category pre-audit on the second sample data according to the audit rules and the first sample data, and generate a pre-audit category for the second sample data.

[0057] In operation S240, the pre-review category is reviewed by the processor, and when the review result meets the predetermined conditions, the first sample data and the second sample data are used as training samples for training the content review model.

[0058] According to an embodiment of the present disclosure, the processor can be a CPU and a GPU. For example, for the pre-review category of the second sample data automatically generated by the second largest model, the pre-review category can be re-reviewed by the CPU to verify the validity of the second sample data. When the review result meets the predetermined conditions, the second sample data is determined to be valid, so that the first sample data and the second sample data can be used as training samples for training the content review model. In this way, the existing hardware resources can be fully utilized, and with the support of lower CPU resources, the speed and efficiency of re-reviewing the pre-review category can be improved, the burden of manual re-review can be reduced, and the consistency of the re-review can be ensured.

[0059] Through the embodiments of the present disclosure, the review rules and first sample data for the content review model are obtained, and the first sample data is enhanced by using prompt words and the powerful language understanding and generation capabilities of the large model to obtain second sample data, and the second sample data is pre-reviewed and verified for validity, providing more diverse data for the content review model to expand the training samples of the content review model. The embodiments of the present disclosure can automatically generate training samples of inappropriate content that are diverse and adaptable to the development of the Internet to train the content review model, which not only helps to improve the generalization ability of the content review model, but also significantly improves the review effect of the content review model when facing complex and changing content. It can be applied to scenarios such as social networks, instant messaging, and user-generated content platforms.

[0060] Figure 3 It is a principle diagram of a training sample generation method according to an embodiment of the present disclosure.

[0061] like Figure 3As shown, the training sample generation method 300 of this embodiment includes: obtaining the review rules 301 and the first sample data 302 for the content review model; using the first large model 304 to process the first prompt word 303 obtained according to the review rules 301 and the first sample data 302 to obtain the second sample data 305; using the second large model 307 to process the second prompt word 306 obtained according to the review rules 301, the first sample data 302 and the second sample data 305 to obtain the pre-review category 308 of the second sample data 305, wherein the pre-review category 308 represents whether the second sample data 305 complies with the review rules 301; reviewing the pre-review category 308 to obtain the review result 309; when the review result 309 meets the predetermined conditions, using the first sample data 302 and the second sample data 305 as the training samples 310 for training the content review model 311.

[0062] According to an embodiment of the present disclosure, at least one of the first large model and the second large model may include a plurality of processing layers connected in sequence. For example, at least one of the first large model and the second large model may include an embedding layer, an encoding layer, a decoding layer, a fully connected layer, and a logistic regression layer connected in sequence.

[0063] In some embodiments, the above operation S230 uses the second large model to process the second prompt word obtained according to the audit rules, the first sample data and the second sample data, including: generating the second prompt word according to the annotation example, the audit rules and the second sample data, wherein the annotation example includes at least part of the data in the first sample data that is marked with a real audit category.

[0064] For example, at least part of the first sample data may be labeled with a corresponding real audit category, which may be at least one of the multiple audit categories defined in the audit rules described above, such as inappropriate type A.

[0065] For example, at least part of the data in the first sample data can be multiple, and each at least part of the data is labeled with a corresponding real audit category, ensuring that the real audit categories of the multiple at least part of the data completely cover the multiple audit categories defined in the audit rules described above, so as to improve the comprehensiveness, completeness and typicality of the labeled examples.

[0066] Through the embodiments of the present disclosure, the second sample data is pre-reviewed by category using the second prompt word and the second large model, which can process a large amount of newly generated second sample data, reduce the burden of manual pre-review, and improve the pre-review efficiency and consistency of the second sample data.

[0067] Figure 4 The present invention is a flowchart of reviewing pre-review categories according to one embodiment of the present disclosure.

[0068] like Figure 4 As shown, in some embodiments, the above operation S240 may include operations S441 to S442 for reviewing the pre-review category through a processor.

[0069] In operation S441 , a first similarity between the pre-review category and the real review category of the annotation example is determined.

[0070] In operation S442 , when the first similarity is greater than a first preset similarity threshold, it is determined that the review result of the pre-review category meets a predetermined condition.

[0071] For example, the first preset similarity threshold is 0.8. When the first similarity between the pre-review category and the actual review category of the marked example is greater than 0.8, it can be determined that the review result of the pre-review category meets the predetermined conditions.

[0072] For example, the processor may re-audit the pre-audit category output by the second largest model to verify the validity of the newly generated second sample data. If the pre-audit category is highly similar to the real audit category of the labeled example, the second sample data may be determined to be valid.

[0073] Through the embodiments of the present disclosure, the second sample data is strictly screened and verified for validity to ensure that the pre-audit category of the newly generated second sample data is consistent with the actual audit category of the labeled example, thereby preventing the introduction of noise into the training samples.

[0074] In some embodiments, there are multiple second sample data; the above operation S240, through the processor, reviews the pre-review category and further includes: randomly extracting part of the second sample data from the multiple second sample data according to a predetermined ratio; determining the second similarity between the pre-review category of the part of the second sample data and the actual review category of the marked example; when the second similarity is greater than the second preset similarity threshold, determining that the review result of the pre-review category meets the predetermined conditions.

[0075] For example, the second preset similarity threshold is 0.7. When the first similarity between the pre-review category and the actual review category of the marked example is greater than 0.7, it can be determined that the review result of the pre-review category meets the predetermined conditions.

[0076] For example, the second preset similarity threshold may be the same as or different from the first preset similarity threshold.

[0077] For example, the processor can randomly sample from a large amount of newly generated second sample data, and re-review the pre-review categories of some of the randomly sampled second sample data. When the pre-review categories of this part of the second sample data are highly similar to the actual review categories of the labeled examples, it can be determined that the large amount of newly generated second sample data are valid.

[0078] For example, the portion of the second sample data may be obtained through a random number generator. A predetermined ratio and the total number of second sample data may be set in the random number generator, and the plurality of second sample data may be sorted. The random number generator may then randomly extract sequence numbers of the portion of the second sample data to obtain second sample data that meets the predetermined ratio.

[0079] According to an embodiment of the present disclosure, when the review result meets predetermined conditions, it also includes: using the second largest model to label the second sample data with a pre-review category; and using the second sample data and the first sample data labeled with the pre-review category as training samples for training the content review model.

[0080] Through the embodiments of the present disclosure, the second large model can not only pre-audit the second sample data by category, but also annotate the second sample data to provide accurate labels. This allows processing of large amounts of newly generated second sample data, reduces the burden of manual annotation, and improves the efficiency, quality, and consistency of the annotation of the second sample data. Furthermore, providing accurately labeled second sample data and first sample data as training samples makes the training samples applicable to supervised learning content review models.

[0081] In some embodiments, the training sample generation method also includes: when it is determined that the audit result of the pre-audit category does not meet the predetermined conditions, adjusting the second prompt word to control the output standard of the second large model; for the adjusted second prompt word, returning to the above operation S230 and using the second large model to process the second prompt word obtained according to the audit rules, the first sample data and the second sample data.

[0082] For example, a text description that limits the output standard of the second largest model can be added to the introduction of the second prompt word to adjust the second prompt word; then the second largest model is used to process the adjusted second prompt word to ensure that the pre-review category currently output by the second largest model is different from the pre-review category output previously, so that the pre-review category currently output can be reviewed again later.

[0083] Through the training sample generation method provided by the present disclosure, the powerful language understanding and generation capabilities of prompt words and large models are utilized to generate sample data that is closely connected to real-world scenarios, has clear categories, and is rich in diversity, thereby providing more comprehensive data for the training samples of the content review model, and being able to better utilize the content review model to protect the overall security ecology of users and platforms. In addition, the training sample generation method provided by the present disclosure can be more compatible with hardware terminals with limited resources. On hardware terminals with limited resources, new training samples that are more suitable for content review models can also be generated within a few minutes, reducing resource overhead.

[0084] Based on the training sample generation method provided by the present disclosure, the present disclosure also provides a training method for a content review model. Figure 5 A detailed description of the training method for the content review model is provided.

[0085] Figure 5 It is a flowchart of a method for training a content moderation model according to an embodiment of the present disclosure.

[0086] like Figure 5 As shown, the content review model training method 500 according to this embodiment may include operation S510, which may be executed by a server.

[0087] In operation S510, the content audit model is trained using a training sample, wherein the training sample is generated using the training sample generation method of any of the above embodiments.

[0088] According to an embodiment of the present disclosure, a content review model is used to review multimedia data, which includes at least one of text, images, audio, and video. The content review model can be applied to social media platforms, online forums, news websites, etc., and the model can run on a CPU or GPU.

[0089] In some embodiments, the training method of the content review model also includes: obtaining a test data set for the content review model; using the test data set to test the content review model trained using the training samples; and obtaining a trained content review model when the test results meet predetermined test requirements.

[0090] For example, in an actual environment, a specific test data set can be used to test the content audit model trained using the training samples mentioned above to test the audit effect of the content audit model.

[0091] The content review model trained by the training method provided by the present invention can be more adaptable to hardware terminals with limited resources, which not only improves the training efficiency, but also improves the generalization ability and accuracy of the content review model.

[0092] Based on the training method of the content audit model provided by the present disclosure, the present disclosure also provides a content audit method. Figure 6 Provide a detailed description of the content review method.

[0093] Figure 6 is a flowchart of a content review method according to an embodiment of the present disclosure.

[0094] like Figure 6 As shown, the content review method 600 according to this embodiment may include operation S610, which may be executed by a server.

[0095] In operation S610, the content to be reviewed is obtained, and the content to be reviewed is reviewed using a content review model to obtain a review category; wherein, the content to be reviewed is multimedia data, and the content review model is trained using the training method of the content review model of any of the above embodiments.

[0096] According to an embodiment of the present disclosure, multimedia data includes at least one of text, image, audio, and video. The content review model can be applied to social media platforms, online forums, news websites, etc., and the model can run in a CPU or a GPU.

[0097] Through the content review method provided by the present disclosure, after obtaining the content to be reviewed, the content review model trained using the content review model training method described above is used to review the content to be reviewed and obtain a review category. The review category can reflect the review type to which the content to be reviewed belongs under the review rules. In addition, the content review model is used to automatically review any content to be reviewed, thereby improving the quality and efficiency of content review and the accuracy of identifying inappropriate content, better protecting the overall security ecology of users and platforms, and can be applied to scenarios such as social networks, instant messaging, and user-generated content platforms.

[0098] Based on the training sample generation method provided by the present disclosure, the present disclosure also provides a training sample generation device. Figure 7 The device is described in detail.

[0099] Figure 7 is a block diagram of a training sample generating apparatus according to an embodiment of the present disclosure.

[0100] like Figure 7 As shown, the training sample generating apparatus 700 may include a sample data acquiring module 710 , a sample data enhancing module 720 , an enhanced sample classification module 730 and a training sample determining module 740 .

[0101] The sample data acquisition module 710 is used to obtain the review rules for the content review model and the first sample data, wherein the first sample data is multimedia data.

[0102] The sample data enhancement module 720 is configured to use the first large model to process the first prompt word obtained according to the audit rules and the first sample data to obtain second sample data.

[0103] The enhanced sample classification module 730 is used to use the second large model to process the second prompt word obtained according to the audit rules, the first sample data and the second sample data to obtain the pre-audit category of the second sample data, wherein the pre-audit category indicates whether the second sample data complies with the audit rules.

[0104] The training sample determination module 740 is used to review the pre-review category through a processor, and when the review result meets the predetermined conditions, the first sample data and the second sample data are used as training samples for training the content review model.

[0105] According to an embodiment of the present disclosure, the enhanced sample classification module 730 is further used to generate a second prompt word based on the annotation example, the review rule and the second sample data, wherein the annotation example includes at least part of the data in the first sample data that is marked with a real review category.

[0106] According to an embodiment of the present disclosure, the training sample determination module 740 is also used to: determine a first similarity between the pre-review category and the actual review category of the labeled example; and when the first similarity is greater than a first preset similarity threshold, determine that the review result of the pre-review category meets a predetermined condition.

[0107] According to an embodiment of the present disclosure, there are multiple second sample data; the training sample determination module 740 is also used to: randomly extract part of the second sample data from the multiple second sample data according to a predetermined ratio; determine the second similarity between the pre-review category of the part of the second sample data and the real review category of the labeled example; when the second similarity is greater than the second preset similarity threshold, determine that the review result of the pre-review category meets the predetermined conditions.

[0108] According to an embodiment of the present disclosure, the training sample generation device 700 also includes: a prompt word adjustment module, which is used to adjust the second prompt word when it is determined that the audit result of the pre-audit category does not meet the predetermined conditions, so as to control the output standard of the second large model; a prompt word input module, which is used to return to the step of using the second large model to process the second prompt word obtained according to the audit rules, the first sample data and the second sample data for the adjusted second prompt word.

[0109] Based on the training method of the content audit model provided by the present disclosure, the present disclosure also provides a training device for the content audit model. Figure 8 The device is described in detail.

[0110] Figure 8 It is a block diagram of a training device for a content moderation model according to an embodiment of the present disclosure.

[0111] like Figure 8 As shown, the training device 800 for the content review model may include a model training module 810.

[0112] The model training module 810 is used to train the content review model using training samples; wherein the training samples are generated using the training sample generation device of any of the above embodiments.

[0113] Based on the content audit method provided by the present disclosure, the present disclosure also provides a content audit device. Figure 9 The device is described in detail.

[0114] Figure 9 It is a block diagram of a content review apparatus according to an embodiment of the present disclosure.

[0115] like Figure 9 As shown, the content review device 900 may include a content review module 910.

[0116] The content review module 910 is used to obtain the content to be reviewed, use the content review model to review the content to be reviewed, and obtain the review category; wherein, the content to be reviewed is multimedia data, and the content review model is trained using the content review model training device of any of the above embodiments.

[0117] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0118] According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described above.

[0119] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the method described above.

[0120] According to an embodiment of the present disclosure, a computer program product includes a computer program, and when the computer program is executed by a processor, the computer program implements the method described above.

[0121] Figure 10 A schematic block diagram of an example electronic device that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0122] like Figure 10As shown, electronic device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. RAM 1003 may also store various programs and data required for the operation of electronic device 1000. Computing unit 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to bus 1004.

[0123] Multiple components in the electronic device 1000 are connected to the I / O interface 1005, including an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a magnetic disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the electronic device 1000 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0124] The computing unit 1001 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as the training sample generation method, the content review model training method, and the content review method. For example, in some embodiments, the training sample generation method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into RAM 1003 and executed by computing unit 1001, one or more steps of the training sample generation method, content audit model training method, and content audit method described above may be performed. Alternatively, in other embodiments, computing unit 1001 may be configured to perform the training sample generation method, content audit model training method, and content audit method in any other appropriate manner (e.g., by means of firmware).

[0125] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0126] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0127] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM) or flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0128] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) display or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0129] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0130] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0131] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0132] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A training sample generation method, comprising: Acquire audit rules for a content audit model and first sample data, wherein the first sample data is multimedia data; Obtaining a first prompt word according to the audit rule and the first sample data; Inputting the first prompt word into a first large model, and using the first large model to generate second sample data under the guidance of the first prompt word, wherein the second sample data is different from the first sample data and is new data expanded compared to the first sample data; obtaining a second prompt word according to the audit rule, the first sample data, and the second sample data; inputting the second prompt word into a second large model, and using the second large model under the guidance of the second prompt word to generate a pre-audit category for the second sample data, wherein the pre-audit category indicates whether the second sample data complies with the audit rule; The pre-review category is reviewed by a processor, and when the review result meets predetermined conditions, the first sample data and the second sample data are used as training samples for training the content review model.

2. The method according to claim 1, wherein The obtaining of the second prompt word according to the audit rule, the first sample data, and the second sample data includes: The second prompt word is generated according to the marking example, the audit rule and the second sample data, wherein the marking example includes at least part of the data in the first sample data marked with the real audit category.

3. The method according to claim 2, wherein: The reviewing of the pre-review category by the processor includes: determining a first similarity between the pre-audit category and the real-audit category of the annotated example; In a case where the first similarity is greater than a first preset similarity threshold, it is determined that the audit result of the pre-audit category meets a predetermined condition.

4. The method according to claim 2 or 3, wherein: The second sample data is a plurality of items; and the reviewing the pre-review category by the processor further includes: randomly extracting a portion of the second sample data from the plurality of the second sample data according to a predetermined ratio; determining a second similarity between the pre-audit category of the portion of the second sample data and the true audit category of the labeled example; In a case where the second similarity is greater than a second preset similarity threshold, it is determined that the audit result of the pre-audit category meets a predetermined condition.

5. The method according to claim 1, further comprising: When it is determined that the audit result of the pre-audit category does not meet the predetermined condition, adjusting the second prompt word to control the output standard of the second large model; For the adjusted second prompt word, return to the step of using the second large model to process the second prompt word obtained according to the audit rule, the first sample data, and the second sample data.

6. A method for training a content review model, comprising: Use training samples to train the content review model; The training samples are generated by the method according to any one of claims 1 to 5.

7. A content review method, comprising: Obtaining content to be reviewed, reviewing the content to be reviewed using a content review model, and obtaining a review category; Wherein, the content to be reviewed is multimedia data, and the content review model is trained using the method described in claim 6.

8. A training sample generating apparatus, comprising: A sample data acquisition module, configured to acquire audit rules for a content audit model and first sample data, wherein the first sample data is multimedia data; A first prompt word generating module, configured to obtain a first prompt word according to the audit rule and the first sample data; a sample data enhancement module, configured to input the first prompt word into a first large model, and generate second sample data using the first large model under the guidance of the first prompt word, wherein the second sample data is different from the first sample data and is new data augmented compared to the first sample data; A second prompt word generating module, configured to obtain a second prompt word according to the audit rule, the first sample data, and the second sample data; an enhanced sample classification module, configured to input the second prompt word into a second large model, and use the second large model to generate a pre-audit category for the second sample data under the guidance of the second prompt word, wherein the pre-audit category indicates whether the second sample data complies with the audit rules; The training sample determination module is used to review the pre-review category through a processor, and when the review result meets the predetermined conditions, use the first sample data and the second sample data as training samples for training the content review model.

9. The device according to claim 8, wherein The second prompt word generation module is further used for: The second prompt word is generated according to the marking example, the audit rule and the second sample data, wherein the marking example includes at least part of the data in the first sample data marked with the real audit category.

10. The device according to claim 9, wherein The training sample determination module is further configured to: determining a first similarity between the pre-audit category and the real-audit category of the annotated example; In a case where the first similarity is greater than a first preset similarity threshold, it is determined that the audit result of the pre-audit category meets a predetermined condition.

11. The device according to claim 9 or 10, wherein: There are multiple second sample data; and the training sample determination module is further used to: randomly extracting a portion of the second sample data from the plurality of the second sample data according to a predetermined ratio; determining a second similarity between the pre-audit category of the portion of the second sample data and the true audit category of the labeled example; In a case where the second similarity is greater than a second preset similarity threshold, it is determined that the audit result of the pre-audit category meets a predetermined condition.

12. The apparatus according to claim 8, further comprising: a prompt word adjustment module, configured to adjust the second prompt word, if it is determined that the audit result of the pre-audit category does not meet a predetermined condition, so as to control the output standard of the second large model; The prompt word input module is used to return to the step of using the second large model to process the second prompt word obtained according to the audit rules, the first sample data and the second sample data for the adjusted second prompt word.

13. A training device for a content review model, comprising: Model training module, used to train the content review model using training samples; Wherein, the training samples are generated using the device according to any one of claims 8 to 12.

14. A content review device, comprising: A content review module is used to obtain content to be reviewed, review the content to be reviewed using a content review model, and obtain a review category; Wherein, the content to be reviewed is multimedia data, and the content review model is trained using the device described in claim 13.

15. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images