Automated censor of platform content for harmful content using generative artificial intelligence

Abstract

A device or method receives user input defining what content on a particular platform is to be blocked as objectionable content; generates a custom prompt template to submit to a generative artificial intelligence (GAI) annotation service, the custom prompt template instructing the GAI annotation service to label objectionable content within an annotation request according to the user input defining what content on a particular platform is to be blocked as objectionable content; receives an annotation request, the annotation request including platform content to be checked for any objectionable content based on the user input; generates a prompt for the GAI annotation service by combining the platform content with the custom prompt template; and receives, from the GAI annotation service, an identification of objectionable content in the platform content to be blocked based specifically on the user input defining what content is objectionable content.

Description

Background Technology

[0001] With the continuous development of online platforms and digital communities, regulating harmful content has become a significant challenge. Manually reviewing all the vast amounts of digital content generated on most platforms is simply impractical. Therefore, various industries have implemented content security technologies, deploying automated, lightweight classifiers to flag potentially harmful content. However, a major obstacle lies in the differing interpretations of what constitutes "harmful content" across different use cases. This variability still necessitates additional manual review after content is flagged to attempt to align the results with each platform's specific community guidelines or terms of use. Unfortunately, this manual review incurs substantial costs and time delays, limiting the efficiency of content moderation efforts.

[0002] Therefore, the following technical challenge arises: how to detect harmful content in a way that is tailored to a specific platform or digital community, while minimizing subsequent manual review and the associated communication delays and other added costs. In other words, a more efficient, streamlined, and cost-effective method is needed for content moderation in a specific environment or platform. Summary of the Invention

[0003] In one general aspect, the following description proposes an apparatus including a processor and a memory storing executable instructions, which, when executed by the processor, cause the processor, alone or in combination with other processors, to perform the following functions: receiving user input defining what content on a particular platform will be blocked as harmful content; generating a custom prompt word template based on the user input to submit to a generative artificial intelligence (GAI) annotation service, the custom prompt word template instructing the GAI annotation service to flag harmful content within an annotation request based on the user input defining what content on a particular platform will be blocked as harmful content; receiving an annotation request including platform content for which any harmful content should be checked based on the user input; generating a prompt word for the GAI annotation service by combining the platform content with the custom prompt word template; and specifically receiving, from the GAI annotation service, an identifier of harmful content to be blocked in the platform content based on the user input defining what content is harmful content.

[0004] In another general aspect, the following description proposes a method for blocking harmful content from platform content of a specific electronic community. The method includes: receiving user input defining what content on a specific platform will be blocked as harmful content; generating a custom prompt word template based on the user input to submit to a generative artificial intelligence (GAI) annotation service, the custom prompt word template instructing the GAI annotation service to flag harmful content within an annotation request based on the user input defining what content on a specific platform will be blocked as harmful content; receiving an annotation request, the annotation request including platform content to be checked for any harmful content based on the user input; generating prompt words for the GAI annotation service by combining the platform content with the custom prompt word template; and specifically receiving, from the GAI annotation service, identifiers of harmful content to be blocked in the platform content based on the user input defining what content is harmful content.

[0005] In another general aspect, the following description proposes a system for blocking harmful content from a platform serving an electronic community, the community having guidelines on what constitutes harmful content. The system includes: a processor communicating with the platform to receive platform content to be inspected for harmful content; and a memory storing executable instructions that, when executed by the processor, cause the processor, alone or in combination with other processors, to perform the following functions: generating a customized prompt word template based on the guidelines for submission to a generative artificial intelligence (GAI) annotation service, the customized prompt word template instructing the GAI annotation service to flag harmful content within an annotation request according to the guidelines; receiving an annotation request, the annotation request including platform content to be inspected for any harmful content; generating prompt words for the GAI annotation service by combining the platform content with the customized prompt word template; and using the prompt words, specifically based on the guidelines defining what content is harmful content, generating from the GAI annotation service an identifier of harmful content in the platform content to be blocked.

[0006] This summary is provided to introduce, in a simplified form, a set of concepts further described below in the detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementing any or all of the shortcomings pointed out in any part of this disclosure. Attached Figure Description

[0007] The accompanying drawings illustrate one or more embodiments according to this teaching by way of example only and not by way of limitation. In the drawings, similar reference numerals refer to the same or similar elements. Furthermore, it should be understood that the drawings are not necessarily drawn to scale.

[0008] Figure 1 This is a flowchart illustrating an example of automated content moderation based on the principles described herein.

[0009] Figure 2A This is a system diagram with additional details of an example of automated content moderation based on the principles described in this article.

[0010] Figure 2B This is another flowchart providing additional details on an example of automated content moderation based on the principles described in this article.

[0011] Figure 3A An example of a user guide that defines harmful content for a specific platform is shown.

[0012] Figure 3B It shows the use of Figure 3A Examples of custom prompt word templates for generative artificial intelligence in the guidelines.

[0013] Figure 4 This is a flowchart with additional details of an example of automated content moderation based on the principles described in this article.

[0014] Figure 5A The first-level cue word compression is described based on the principles described in this paper.

[0015] Figure 5B The second-level cue word compression is described based on the principles described in this paper.

[0016] Figure 6 This paper describes the method for configuring reliability levels based on the principles described in this paper, as well as the method for labeling examples of harmful content.

[0017] Figure 7 This is a block diagram illustrating an exemplary software architecture, the various parts of which can be used in conjunction with the various hardware architectures described herein.

[0018] Figure 8 This is a block diagram illustrating the components of an exemplary machine configured to read instructions from a machine-readable medium and execute any features described herein. Detailed Implementation

[0019] Digital or electronic communities may have written guidelines for proper communication on their platforms. For example, these guidelines might specify one or more objectives, such as:

[0020] Civility is at the heart of our community. To foster a civilized environment, our community prohibits certain types of content, including:

[0021] Sensitive real-world events: Our community prohibits the reproduction of specific real-world events of a sensitive nature, ridicule of victims of such events, support or promotion of the perpetrators or consequences of such events, or use of such events for commercial purposes, including content concerning (1) acts of mass violence, (2) specific real-world natural disasters, or (3) violations of human or civil rights.

[0022] Violent Content and Gore: Our community prohibits content containing extreme violence or severe physical or psychological abuse, including: (1) animal abuse and torture, (2) realistic or real-world depictions of extreme violence, gore, or death…

[0023] Figure 3A A more detailed version of this example is shown in the image.

[0024] To implement these guidelines, platform operators can guide user content through lightweight automated classifiers. This could be a keyword system that flags keywords associated with prohibited content. If the automated classifier doesn't flag content as harmful, it will be approved. However, such automated classifiers may inaccurately flag acceptable content as potentially harmful. If content is flagged by an automated classifier, it will be reviewed by a team of human content moderators who will determine whether the content is indeed inappropriate and should therefore be blocked, or whether it does not violate the guidelines and can be approved. While this approach can be effective, it is time-consuming and expensive, significantly increasing latency in content communication on the platform.

[0025] Therefore, as mentioned above, the following technical problem arises: the lack of a system that detects harmful content in a way tailored to specific platforms or digital communities, while limiting any need for manual review and the associated increase in harmfulness due to communication delays and other factors. In other words, there is a need for an effective, streamlined, and cost-effective content moderation method based on specific guidelines for a particular community. The technical solution to these problems, as described below, is called adaptive annotation.

[0026] Adaptive annotation is a service that enhances human review of potentially harmful content. More specifically, it aims to empower users to improve the human review process for potentially harmful content while adhering to community-specific guidelines. The primary goal is to strike a balance between high accuracy and increased efficiency in content moderation. By leveraging the adaptive annotation service described herein, users can create custom categories and generate plaintext annotation prompts that align with the specific nuances of community guidelines or terms of use. This proactive approach enables the annotation of content, resulting in adaptive detection results fine-tuned to the specific requirements of each digital community. Users can create custom categories and generate plaintext annotation prompts based on specific community guidelines / terms of use. They can then receive content annotations tagged by the adaptive annotation service for further review. In this way, the adaptive annotation service provides adaptive detection results for harmful content using generated prompts specific to user guidelines or terms of service.

[0027] Figure 1 This is a flowchart illustrating an exemplary method for detecting harmful content based on user-specific guidelines, according to the principles described herein. Figure 1 As shown, user 100 or user group has an application or service that generates content for the digital community. The application or service can be, for example, a social media platform, a discussion forum, generative artificial intelligence, or other applications / services that users or generative artificial intelligence use to create content for the digital community.

[0028] In this scenario, the user groups operating the platform may want to restrict harmful content from appearing on the platform. In different contexts, harmful content can refer to any content that diminishes the platform's value or the target audience's enjoyment of the platform. For example, harmful content is content deemed offensive, threatening, privacy-invading, or unsuitable for the digital community it serves.

[0029] What constitutes "harmful" content will vary depending on the user group, the digital community served, the type of content generated, and other factors. Therefore, content considered harmful on one platform may not be considered harmful on another. For this reason, a set of guidelines for defining harmful content for one platform may not apply to a second platform. Thus, creating a single system that monitors harmful content in the same way across all platforms is not feasible.

[0030] As used herein, the term "user" refers to a platform administrator or management group responsible for reviewing content on the platform to minimize harmful content. The term "platform" generally refers to any application or service on which content is generated for the community and requires content review.

[0031] like Figure 1As shown, the method described in this article allows users to implement their own desired guidelines within the system, which will help in the moderation of harmful content on their platform. References Figure 1 User 100 determines certain user guidelines 101, which indicate or define what content is considered harmful on the user's platform. The adaptive annotation service shown will utilize user guidelines 101 to create a custom cue word template 102. This cue word template 102 is used to submit content from the user's platform to a generative artificial intelligence (GAI) model, such as a large language model (LLM).

[0032] The prompt word template 102 uses user guidelines to instruct the GAI model how to review harmful content from the user's platform, based on the user guidelines. Using the guidance from the prompt word template, the GAI can then identify content that user 100 specifically considers harmful, without flagging content that is not considered harmful in the context of user guidelines 101 already provided by user 100.

[0033] After preparing the prompt word template 102, the system is ready to review content from the platform for the user. Therefore, user 100 submits content from the platform as a comment request 103 to allow the system to identify any content that should be further reviewed as potentially harmful. Comment requests 103 can be a series of requests that continuously stream content from the platform to the review system. Alternatively, comment requests 103 can be submitted periodically only when user 100 or some other lightweight system has already identified content from the platform that needs review.

[0034] The content of annotation request 103 is combined with a customized prompt template 102 to generate a GAI prompt 104. This prompt is submitted to the GAI annotation service 105. In the GAI annotation service 105, a GAI model (e.g., LLM) reviews the content of annotation request 103 based on user guidelines 101 incorporated into the prompt template 102. As a result, the GAI annotation service 105 returns annotation 106 from the requested content to user 100. In this annotation 106, potentially harmful content is flagged or annotated to help user 100 review the content effectively and promptly.

[0035] In a more detailed description below, several different GAI models can be used for different purposes in the adaptive annotation service described. The primary GAI is GAI Annotation Service 105, which reviews and annotates any harmful content in the platform's content based on prompt words. Other GAI models, which may have lower training and operational costs, will be mentioned below for performing additional functions that support the adaptive annotation service ultimately provided by GAI Annotation Service 105.

[0036] Figure 2AThis is a system diagram that further illustrates the details of the system. In the example given above, the user begins with a set of written guidelines 101 defining what content will be considered harmful. In such an example, a custom prompt word template 102 can be generated as described below.

[0037] like Figure 2A As shown, to generate a custom prompt word template, the system can submit a written user guide 101 to a custom prompt word template generator 122 that invokes a GAI model 120. This GAI model can be a large language model or a small language model. In some examples, GAI 120 is not as advanced as other GAIs described herein. Prompting the GAI 120 will require it to extract annotations and annotation definitions from the written user guide 101 and place this information within the structure of the custom prompt word template. GAI 120 then returns the resulting custom prompt word template 102, which is specifically derived from the user's written guide 101. Therefore, the user guide 101 does not need to be prepared in the form most easily ingested by the system. Instead, any user, regardless of technical expertise, can prepare and provide a guide 101 in natural language, which the system then implements in content review. This system solves the technical problem for users of implementing guides expressed solely in natural language.

[0038] Based on the example in User Guide 101 given above, an example of customizing prompt word template 102 can be as follows:

[0039] #Community Standards and Civility Standards Guidelines

[0040] ##How to apply

[0041] ### Annotation Set

[0042] The tag set includes “harmless” (tag 0), “real-world sensitive events” (tag 1), “violent content and gore” (tag 2), “romantic and pornographic content” (tag 3), “illegal and regulated goods or activities” (tag 4), “profanity” (tag 5), and “political content” (tag 6).

[0043] ### Apply all applicable annotations

[0044] Given an additional text fragment, apply all tags that match the content.

[0045] For example, if a portion of the content is suitable for two or more annotations, then all applicable annotations are applied to the annotations for that portion of the content.

[0046] ###illustrate

[0047] Please focus only on the scenarios listed in the annotation guidelines. Other scenarios not covered by the provided guidelines are beyond the scope of this task.

[0048] ## Definition of annotation

[0049] ### Mark 0: Harmless

[0050] ###Note 1: Sensitive events in the real world…

[0051] Figure 3B A more detailed example of this custom prompt template is shown below. The content to be checked is then appended to the custom prompt template 102, as described below.

[0052] Continue in Figure 2A In the middle, user 100 then submits content from the platform for review in the form of a comment request 103, as mentioned above. Figure 1 As described in [the document]. The content of annotation request 103 is appended to a custom prompt template 102 by prompt generator 124 to generate prompts 104 for GAI annotation service 105. The resulting prompts enable GAI annotation service 105 to review the content and label potentially harmful content as potentially harmful based on the types of content defined from user guide 101.

[0053] The prompt 104 can also instruct the GAI annotation service 105 to discard approved and deemed safe content. In this way, the amount of content from the platform or annotation requests 103 is filtered into a more manageable quantity, with all content identified as potentially harmful and labeled with the type of potentially harmful content. The adaptive annotation service can also indicate the severity of each labeled section. In this way, manual content review can be eliminated, or at least the amount of content presented for further user review can be appropriately reduced to maximize the efficiency of any manual review implemented.

[0054] Therefore, in this system, the prompts used for GAI annotation service 105 are automatically generated based on reliable guidelines, such as text or multimodal content, according to categories provided by the user in a simplified manner. Users do not need to manually create prompts. Instead, as described above, the prompts are generated based on reliable guidelines, for example, automatically from plain text to an LLM-understandable format according to the categories provided by the user using prompts described for GAI 120. A list of annotation (classes) and a detailed definition of each annotation (class) are automatically extracted, and the output format is automatically declared in the prompts. In addition to harmful annotation results (annotations), inference can be returned to achieve interpretability. As further described below, confidence levels can be returned for filtering to higher precision.

[0055] As mentioned above, Figure 3A and Figure 3B The user guide is shown separately. Figure 3A ) and the resulting customized prompt template ( Figure 3B Example 102. In this example, the prompt sent to GAI 120 to generate a prompt template based on the guidelines will instruct GAI 120 to extract a list of tags (classes) of harmful content and a detailed definition of each tag (class). If possible, the prompt will also instruct GAI 120 to include examples of tags (few-shots) extracted from the guidelines in prompt template 102.

[0056] The prompts for GAI 120 may also guide the inclusion of the following instructions in the resulting custom prompt template in order to (1) declare the input format and target output format, such as annotations, reasoning or interpretations for placing annotations, and confidence scores that can be used for further filtering, and (2) include principles on how to handle out-of-scope cases and / or how to handle instances of harmful content that are subject to multiple annotations, meaning that the content is harmful in more than one category with definitions and annotations.

[0057] If the user guide does not provide annotation examples (few samples), additional prompts for GAI 120 can instruct GAI 120 to summarize the description for each annotation, generate different few samples for each summarized description, and convert the generated few samples into examples to be included in the target output of the custom prompt template 102. As used herein, the term “few samples” is used synonymously with “example” to refer to an example of harmful content, which may be accompanied by annotations identifying the category or type of harmful content.

[0058] However, in some cases, users may only have a vague idea of ​​what content might be generated on the platform and when or whether any such content should be considered harmful. For example, user 100 might only think of a few keywords that indicate harmful content in their platform context. This is called a cold start. In this case, the adaptive annotation service will accept those keywords as user guidelines. As described above, the system will submit the keyword set to GAI model 120. The keywords are submitted along with prompts, requesting GAI model 120 to create a set of annotations and corresponding definitions of potentially harmful content based on the keywords. Therefore, GAI model 120 then returns a customized prompt template, as in the example above, which is constructed to detect harmful content based on those keywords that serve as indicators of harmful content.

[0059] During a cold start, the system can also return a natural language version of the guide that users can adopt and publish to the community based on keywords. This version of the guide can then be edited to ensure accuracy or expanded as needed. In this case, as mentioned above, the revised version can be used as the basis for a new custom prompt word template 102.

[0060] Reference Figure 2B For keyword-based cold starts, the system includes a cold start handler 125. The cold start handler 125 uses a taxonomy specifically built for the handler, based on Responsible AI. In this taxonomy, each node is a high-level category, such as violence, while each leaf node is a detailed subcategory of its parent node, such as animal cruelty, school bullying, etc. For each node (leaf or non-leaf), there is a category name and a list of synonyms for the category name, as well as guidelines and prompts.

[0061] The cold start handler maps input keywords to any category within the taxonomy by calculating the similarity between the input and category name synonyms. When a keyword is mapped to a taxonomy, details from the taxonomy (e.g., guides and prompts) are used to describe the corresponding content to be flagged as objectionable in subsequent prompts for GAI 120. If mapping fails, the cold start handler uses LLM prompts to create a first version of the user guide, summarizing the detailed scenario within the input category.

[0062] Figure 4 Another example of a system for filtering potentially harmful content from a platform, based on the principles described herein, is shown. Figure 4 As shown, a user-specific platform guideline 101 is provided to identify or define offensive or inappropriate content, collectively referred to as harmful content. The guideline may be provided in JSON format, in plain text (as described above), or in some other format. The guideline 101 may include a definition of each tag that the user wishes to apply to potentially harmful content. Optionally, the guideline 101 may also include a list of detailed scenarios for each tag, as well as examples of harmful content to which the tag will be applied.

[0063] The text converter in the service will utilize LLM prompts for GAI 120, such as... Figure 2A As shown, key information is extracted from text guide 101, including the definition of each label (class), the general principles for the current category annotation, emphasis on the input format and the final output format, etc. The general principles of the output added to GAI 120 can include how to handle cases where multiple labels (classes) are hit or how to handle cases where the scope is out of bounds.

[0064] If user 100 does not have or cannot provide examples of harmful content for a given label (few samples), the system will use, for example... Figure 2A The GAI 120 is used to generate such examples 131. In a cold start scenario, GAI 120 can be prompted to suggest examples of harmful content based on user-provided keywords. Several examples can be created for each tag (class) to improve the final annotation quality. Thus, the system can utilize LLM prompts to generate detailed examples that match the definition and severity rating of each tag, rather than simply copying descriptions with different literary forms and writing styles.

[0065] like Figure 4 As shown, after generating the GAI prompt words, jailbreak detection 132 is performed. Specifically, jailbreak detection is used to detect whether jailbreak intent exists in the provided guide 101. In this context, the term "jailbreak" describes a user attempting to use text input to break the model's own rules, thereby enabling the user to bypass system safeguards and generate harmful content. The system implements jailbreak detection because users might inject harmful content into the guide provided to the adaptive annotation service to request the LLM used by the adaptive annotation service backend to do harmful things. Using jailbreak detection 132, the adaptive annotation service will reject user requests with jailbreak intent. Specifically, the adaptive annotation service will use a third additional prompt word (in addition to the first prompt word extracted from the annotation definition and the second prompt word generated from a small sample) to instruct GAI 120 to detect whether jailbreak intent exists in the user guide 101. If jailbreak is detected, the user request will return a failure status code, meaning that the prompt word template 102 was not successfully generated.

[0066] As in the previous example, the result of processing guideline 101 is a customized prompt template 102. For example... Figure 4 As shown, the user can then enter a comment request 103 for the content to be checked. The content is then combined with a prompt word template 102 to complete the prompt word for the main GAI comment service 105.

[0067] exist Figure 4 In the example, the prompt words can also be compressed 133. Specifically, the generated prompt words are compressed to reduce tokens, thereby saving annotation costs and reducing latency, while avoiding significant quality degradation. Prompt word compression will be described in more detail below. In addition to identifying potentially harmful content in the annotation results, the prompt word template can also request the GAI annotation service 105 to return inferences supporting why the content was marked as potentially harmful to improve the interpretability of the GAI output, i.e., annotations 106 from the content of the annotation request.

[0068] The adaptive annotation service also supports dynamic few-shot selection 134 to improve annotation quality. Specifically, as described above, a pre-annotated dataset containing multiple few-shot candidates provided by user 100 can exist, which can be gradually collected and accumulated in the user's account. For each batch of sampled content in annotation request 103, the system dynamically selects several examples from the few-shot candidate set 134 that are semantically relevant to the input content to be included in the prompt words of the GAI annotation service 105 to guide the annotation 106 from the requested content. Semantic similarity and relevance can be calculated based on any pairwise similarity algorithm, including but not limited to cosine similarity between the embedding vectors of two texts, pyramid matching, etc. Diversity (of annotations) is also considered when selecting relevant examples. At function 134, the adaptive annotation service will randomly select some from those whose similarity to the current input text is higher than a threshold. After each round of evaluation and iteration, more few-shot candidates can be added to the few-shot corpus. Multimodal input will be supported. Chat and non-chat input are supported.

[0069] For example, a user might recognize phrases or ideas that are regularly expressed on the platform being reviewed and often have similar structures. Users can record these examples or small samples as a corpus of pre-labeled data. For instance, the phrase "the game is amazing…" might be labeled with a 0, indicating harmless content. Similarly, a sentence such as "The gameplay mechanics are smooth and intuitive, creating an enjoyable gaming experience for both beginners and experienced players" might also be labeled with a 0. As described above, the system can analyze the content of requests to be reviewed for these or semantically similar phrases.

[0070] When such similar phrases are found, the prompt words for GAI annotation service 105 can be dynamically updated to include matching examples with label 0. Therefore, GAI annotation service 105 can easily and correctly label those matching parts of the content to be checked with label 0. For example, suppose the content to be checked includes the sentence: “I love this game.” This phrase will be identified as semantically similar to the example: “The game is amazing.” Therefore, the example “The game is amazing (label 0)” will be added to the prompt words for GAI annotation service 105. Thus, the phrase “I love this game” will be labeled with label 0 in the output from GAI annotation service 105 to the user. Specifically, if similar examples of “I love this game” from a few-sample candidate corpus have label 0, the adaptive annotation service will be more likely to label “I love this game” with label 0. However, the adaptive annotation service can provide output based on both guidelines and few samples.

[0071] User-provided few-sample examples can also be phrases that include harmful content with corresponding annotations, so that any such similar phrases in the content to be checked will cause those examples and their annotations to be included in the prompt words for the GAI annotation service 105. As mentioned above, if no user-provided few samples are available, the initial GAI prompt 120 can generate some few samples based on the user's keywords for harmful content. In either case, the user can add to the few-sample corpus 141 over time.

[0072] For example Figure 4 As shown, annotations 106 from the requested content can be evaluated offline 135. If the system does not optimally identify content that user 100 wants to mark as harmful, the user guide 101 can be updated accordingly and iteratively implemented. Figure 4 The process improves the output of the customized prompt word template 102 and GAI annotation service 105.

[0073] refer to Figure 5A and Figure 5BExplain the compression of prompts for GAI annotation service 105. As will be understood, the larger the amount of data in the prompts for GAI annotation service 105, the more bandwidth will be required to send the prompts, and the more time and processing resources will be needed at GAI annotation service 105 to respond to the prompts. However, some parts of the prompts may be redundant in a given instance, thus unnecessarily increasing the demand for bandwidth, processing power, and time. For example, based on the content of the request to be reviewed 103, some instructions in the customized prompt template 102 may not be applicable at all and can be omitted. In addition, some content in the request to be reviewed 103 can be preemptively eliminated during the review process without a significant risk of losing harmful content. Therefore, two steps can be performed in prompt compression 133. Compression 133 is performed based on information retrieval or probability theory.

[0074] refer to Figure 5A As a first step, the cue word compressor 140 segments the cue words into several fragments and uses another LLM cue word, fed into another cheaper LLM, to mark which cue word fragments are relevant to the current input. Only the relevant parts are retained, and the others are removed from the current input.

[0075] For example, different annotation definitions, along with the requested content, can be fed into a smaller, cheaper GAI 170 (e.g., LLM) using instructions that identify which annotation definitions might apply to the content being inspected. Figure 5A As shown, the prompt word template 102 can define labels 0-N. However, considering the specific content of the current request for harmful content to be reviewed, only labels 1, 4, and 7 seem applicable. Accordingly, refer again... Figure 4 The content to be reviewed 103 is combined with the customized prompt template 102. Then, to compress the prompt, the lighter GAI 170 is invoked. Based on the results, before submitting the prompt to the GAI annotation service 105, instructions regarding all annotations except 1, 4, and 7 are removed to compress the prompt.

[0076] As a second step, further compression is achieved at a finer granularity based on the compressed cue words from the first step. Specifically, the cue word compressor 140 removes words that have a high conditional probability given their context. More specifically, the system uses a pre-computed table of n-gram probabilities (i.e., the probability of the nth word given the previous (n-1) words in sequence).

[0077] Generally, LLM operates by using a large training set and input cue words as a starting point to determine the word most likely to follow the previous word. Similarly, this stage of the cue word compressor 140 will calculate the conditional probability of each word or phrase in the content to be inspected, given its context, based on a pre-computed table of n-gram probabilities or using a small-scale GAI or LLM.

[0078] exist Figure 5B In the example, the first phrase is "I usually". In the given context, the prompt word compressor 140 determines that the probability of the word "usually" following the word "I" is 90% (0.9). Accordingly, the word "usually" can be omitted from the prompt word to further compress the prompt word without significantly weakening the service's ability to detect harmful content. Figure 5B In the second part of the example, the input phrase is "create a bomb". As shown, the article "a" has a probability of 0.87 following "create". Accordingly, the word "a" can be eliminated to further compress the prompt word. In the given context, the probabilities of "create" and "bomb" are much lower (0.3 and 0.01, respectively). Accordingly, the prompt word compressor 140 does not omit any words. A threshold can be set for the probability score of the prompt word compressor 140 omitting words from the prompt word.

[0079] Figure 6 This paper describes the process of providing annotations for few samples based on the principles described herein. As mentioned above, users can provide labeled few samples to help the system annotate content to be reviewed. Alternatively, if users do not provide those few samples but provide written community guidelines, the system can generate multiple few samples based on the written guidelines. Or, as mentioned above, if users only provide keywords without written guidelines, the system can use the user-provided keywords to generate multiple few samples. In these cases, the generated few samples need to be annotated. In other cases, users can provide few samples (examples) of harmful content without providing annotations about the type of harmful content, or the system can attempt to verify the annotations provided by the user for the examples.

[0080] Therefore, as Figure 6 As shown, the system has multiple minority samples to enhance the prompts for the GAI annotation service 105. Each of these minority samples can be associated with a tag identifying the type of harmful content it represents, or such a tag may be missing. If the minority sample is automatically generated, it may not necessarily have a tag. Alternatively, the user may omit the tagging of the example.

[0081] Accordingly, for each few samples, the system will use multiple models 161 to classify or label them. These models 161 may be LLMs, or smaller, cheaper models, which use a list of possible labels and instructions for assigning one of the labels to each example in the set 141 for prompting, each possible label being defined by a different type of harmful content.

[0082] The confidence level for each of these levels is determined by a vote among the different models. In this context, a vote refers to how many models make the same prediction. Figure 6 In one of the minority samples, the phrase "You're a dead man, Mr. Bond" was submitted to the voting model 161. One-third (1 / 3) predicted the content was offensive and assigned a label of 1. However, two-thirds (2 / 3) of the models predicted the content was harmless and assigned a label of 0. The system could adopt the majority result and assign a label of 0. Alternatively, the system could attempt to avoid missing any potentially harmful content and could assign a label of 1, as one of the models indicated.

[0083] In either case, the label can be associated with a confidence level. Figure 6 In the example, the example is assigned a label of 1, but it's important to note that this assigned confidence level is low. The example, label, and confidence level can then be combined into the prompt word for the GAI annotation service 105 to fine-tune the prompt word.

[0084] exist Figure 6 In another example, the example in set 141 is “In GTA5, killing animals is a great way to make money quickly.” This example is unlabeled, likely because the user omitted applying labeling when providing the example. When examples are automatically generated or there are few samples, they are generated by labeling and therefore have labels. In the illustrated example, when submitted to voting model 161, the result is that all three (3 / 3) predict that the statement is harmful content associated with label 1 of this type of content. Accordingly, this example is assigned label 1 with full confidence. Again, the example, label, and confidence level can then be combined into the prompt word for GAI annotation service 105 to fine-tune the prompt word.

[0085] Different voting results can indicate the difficulty of the inference. The system can map confidence levels to LLM natural language inference statements, such as... Figure 6 As shown in the example, this reasoning statement will be easier for the GAI annotation service 105 to process compared to a numerical confidence rating.

[0086] As described above, when checking for harmful content in new input text, the system dynamically utilizes the few-shot technique to select a subset of relevant few samples. The GAI annotation service 105 then learns confidence (difficulty) about the annotations and predictions from these relevant few samples and outputs the annotations and confidence scores when annotating the current input text. With the output confidence scores, the user can filter the output based on the confidence level. For example, if a user wants to improve accuracy (whether the predicted harmful input is indeed harmful traffic), he or she can focus on indicators of harmful content with low or no confidence, and can provide additional examples or annotations to supplement where the system lacks confidence in the annotation determination.

[0087] Furthermore, the voting process can take different approaches. If a majority of models agree on the prediction for a few samples of text, the system can assign the majority prediction in the case of unanimous voting by conveying confidence through natural language suffixes such as "the prediction has confidence" or "has full confidence." If different models are evenly distributed across their predictions for a few samples, indicating that the text is difficult to infer, the system can assign one of the labels with moderate or low confidence. If the user provides labels for a few samples, the system can compare the predictions made by different models with the user's true values. If a majority of models obtain the same prediction as the user's true values, the confidence statement could be "the prediction has full confidence." If a majority of models obtain incorrect predictions compared to the user's labels, the system can include phrases such as "the prediction has low confidence." If no model obtains the same prediction as the user's labels, then "the prediction has no confidence."

[0088] The model participating in the voting can be any lightweight model trained previously, or any previous version of the cue word in each iteration. If a corpus of few-sample candidates exists, the LLM service / application owner can also set them as release gates for the evaluation set and cue word iterations. They can try updating guidelines for the same category and decide which version (not necessarily the final version) to retain based on consistency between annotations and cue word annotation results on the evaluation set.

[0089] Figure 7 This is a block diagram 700 illustrating an exemplary software architecture 702, the various parts of which can be used in conjunction with various hardware architectures described herein that can implement any of the features described above. Figure 7 This is a non-limiting example of a software architecture, and it will be understood that many other architectures can be implemented to facilitate the functionality described herein. Software architecture 702 can be implemented in, for example... Figure 8It executes on hardware such as machine 800, which includes processor 810, memory 830, and input / output (I / O) components 850, etc. A representative hardware layer 704 is shown, and it can represent, for example... Figure 8 The machine 800. A representative hardware layer 704 includes a processing unit 706 and associated executable instructions 708. The executable instructions 708 represent executable instructions of the software architecture 702, including implementations of the methods, modules, etc., described herein. Hardware layer 704 also includes a memory / storage device 710, which also includes the executable instructions 708 and accompanying data. Hardware layer 704 may also include other hardware modules 712. The instructions 708 held by the processing unit 706 may be portions of the instructions 708 held by the memory / storage device 710.

[0090] The exemplary software architecture 702 can be conceptualized as layers, each providing various functionalities. For example, software architecture 702 may include layers and components such as an operating system (OS) 714, libraries 716, frameworks 718, applications 720, and a presentation layer 744. Operationally, applications 720 and / or other components within a layer can invoke API calls 724 to other layers and receive corresponding results 726. The layers shown are representative in nature, and other software architectures may include additional or different layers. For example, some mobile or dedicated operating systems may not provide a framework / middleware 718.

[0091] OS 714 can manage hardware resources and provide public services. OS 714 may include, for example, a kernel 728, services 730, and drivers 732. The kernel 728 can act as an abstraction layer between the hardware layer 704 and other software layers. For example, the kernel 728 may be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, etc. Services 730 can provide other public services to other software layers. Drivers 732 can be responsible for controlling or interfacing with the underlying hardware layer 704. For example, depending on the hardware and / or software configuration, drivers 732 may include display drivers, camera drivers, memory / storage device drivers, peripheral device drivers (e.g., via Universal Serial Bus (USB)), network and / or wireless communication drivers, audio drivers, etc.

[0092] Library 716 provides common infrastructure that can be used by application 720 and / or other components and / or layers. Library 716 typically provides functionality used by other software modules to perform tasks, rather than interacting directly with OS 714. Library 716 may include system libraries 734 (e.g., the C standard library), which provide functions such as memory allocation, string manipulation, and file operations. Additionally, library 716 may include API libraries 736, such as media libraries (e.g., supporting the rendering and manipulation of image, sound, and / or video data formats), graphics libraries (e.g., OpenGL libraries for rendering 2D and 3D graphics on a display), database libraries (e.g., SQLite or other relational database functions), and web libraries (e.g., WebKit, which provides web browsing functionality). Library 716 may also include a wide variety of other libraries 738 to provide numerous functionalities for application 720 and other software modules.

[0093] Framework 718 (sometimes referred to as middleware) provides a higher level of common infrastructure that can be used by Application 720 and / or other software modules. For example, Framework 718 can provide various graphical user interface (GUI) functionalities, advanced resource management, or advanced location services. Framework 718 can provide a wide range of other APIs for Application 720 and / or other software modules.

[0094] Application 720 includes built-in application 740 and / or third-party application 742. Examples of built-in application 740 may include, but are not limited to, contact applications, browser applications, location applications, media applications, messaging applications, and / or game applications. Third-party application 742 may include any application developed by an entity other than a platform-specific vendor. Application 720 may use the functionality available via OS 714, library 716, framework 718, and presentation layer 744 to create a user interface for user interaction.

[0095] Some software architectures use virtual machines, such as virtual machine 748. Virtual machine 748 provides an execution environment in which applications / modules can run as if they were on a hardware machine (e.g., a physical machine). Figure 8 The virtual machine 748 can be run on the same machine 800. It can be hosted by a host OS (e.g., OS 714) or a hypervisor, and can have a virtual machine monitor 746 that manages the operation of the virtual machine 748 and its interoperability with the host operating system. A software architecture different from the external software architecture 702 can run within the virtual machine 748, such as an OS 750, libraries 752, frameworks 754, applications 756, and / or a presentation layer 758.

[0096] Figure 8This is a block diagram illustrating the components of an exemplary machine 800, which is configured to read instructions from a machine-readable medium (e.g., a machine-readable storage medium) and perform any features described herein. The exemplary machine 800 is in the form of a computer system in which instructions 816 (e.g., in the form of software components) for causing the machine 800 to perform any features described herein can be executed.

[0097] Therefore, instruction 816 can be used to implement the modules or components described herein. Instruction 816 enables an unprogrammed and / or unconfigured machine 800 to act as a specific machine configured to perform the described features. Machine 800 can be configured to act as a standalone device or can be coupled (e.g., networked) to other machines. In a networked deployment, machine 800 can operate as a server machine or client machine in a server-client network environment, or as a node in a peer-to-peer or distributed network environment. Machine 800 can be embodied as, for example, a server computer, client computer, personal computer (PC), tablet computer, laptop computer, netbook, set-top box (STB), gaming and / or entertainment system, smartphone, mobile device, wearable device (e.g., smartwatch), and Internet of Things (IoT) device. Furthermore, although only a single machine 800 is shown, the term "machine" includes a collection of machines that execute instruction 816 individually or jointly.

[0098] Machine 800 may include processor 810, memory 830, and I / O components 850, which may be communicatively coupled via, for example, bus 802. Bus 802 may include multiple buses that couple various components of machine 800 via various bus technologies and protocols. In examples, processor 810 (including, for example, a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), ASIC, or suitable combinations thereof) may include one or more processors 812a to 812n capable of executing instructions 816 and processing data. In some examples, one or more processors 810 may execute instructions provided or recognized by one or more other processors 810. The term "processor" includes multi-core processors, which include cores capable of executing instructions simultaneously. Although Figure 8 Multiple processors are shown, but machine 800 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors each with a single core, multiple processors each with multiple cores, or any combination thereof. In some examples, machine 800 may include multiple processors distributed across multiple machines.

[0099] Memory / storage device 830 may include main memory 832, static memory 834 or other memory, and memory cell 836, both of which may be accessed by processor 810, for example, via bus 802. Memory cell 836 and memories 832, 834 store instructions 816 embodying any one or more of the functions described herein. Memory / storage device 830 may also store temporary, intermediate, and / or long-term data for processor 810. Instructions 816 may also reside wholly or partially in memories 832, 834, memory cell 836, at least one of processor 810 (e.g., in a command buffer or cache memory), memory at at least one of I / O components 850, or any suitable combination thereof, during their execution. Accordingly, memories 832, 834, memory cell 836, memory in processor 810, and memory in I / O components 850 are examples of machine-readable media.

[0100] As used herein, "machine-readable medium" refers to a device capable of temporarily or permanently storing instructions and data that cause machine 800 to operate in a particular manner, and may include, but is not limited to: random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical storage media, magnetic storage media and devices, cache memory, network-accessible or cloud storage devices, other types of storage devices, and / or any suitable combination thereof. The term "machine-readable medium" applies to a single medium or combination of media used to store instructions (e.g., instruction 816) that are executed by machine 800, such that when the instructions are executed by one or more processors 810 of machine 800, they cause machine 800 to perform one or more features described herein. Accordingly, "machine-readable medium" can refer to a single storage device, as well as a "cloud-based" storage system or storage network comprising multiple storage devices or equipment. The term "machine-readable medium" excludes signals themselves.

[0101] I / O component 850 may include a wide variety of hardware components suitable for receiving input, providing output, generating output, sending information, exchanging information, capturing measurements, etc. The specific I / O component 850 included in a particular machine will depend on the type and / or function of the machine. For example, mobile devices such as mobile phones may include touch input devices, while terminalless servers or IoT devices may not include such touch input devices. Figure 8The specific examples of I / O components shown are by no means limiting, and other types of components may be included in machine 800. The grouping of I / O components 850 is merely for the sake of simplicity in this discussion, and the grouping is by no means limiting. In various examples, I / O components 850 may include user output components 852 and user input components 854. User output components 852 may include, for example, display components for displaying information (e.g., liquid crystal display (LCD) or projector), acoustic components (e.g., speakers), haptic components (e.g., vibration motors or force feedback devices), and / or other signal generators. User input components 854 may include, for example, alphanumeric input components (e.g., keyboards or touchscreens), pointing components (e.g., mouse devices, touchpads, or other pointing tools) configured to receive various user inputs, such as user commands and / or selections.

[0102] In some examples, I / O component 850 may include biometric component 856, motion component 858, environmental component 860, and / or positioning component 862, as well as a variety of other physical sensor components. Biometric component 856 may include components for detecting body expressions (e.g., facial expressions, vocal expressions, hand or body gestures, or eye tracking), measuring biosignals (e.g., heart rate or brainwaves), and identifying people (e.g., via voice-based, retinal, fingerprint, and / or facial recognition). Motion component 858 may include, for example, accelerometers (e.g., accelerometers) and rotation sensors (e.g., gyroscopes). Environmental component 860 may include, for example, light sensors, temperature sensors, humidity sensors, pressure sensors (e.g., barometers), acoustic sensors (e.g., microphones for detecting ambient noise), proximity sensors (e.g., infrared sensing of nearby objects), and / or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. The positioning component 862 may include, for example, a position sensor (e.g., a Global Positioning System (GPS) receiver), an altitude sensor (e.g., a barometric pressure sensor from which altitude can be derived), and / or a direction sensor (e.g., a magnetometer).

[0103] I / O component 850 may include communication component 864, which implements a variety of technologies operable to couple machine 800 to network 870 and / or device 880 via corresponding communication couplers 872 and 882. Communication component 864 may include one or more network interface components or other suitable devices for interfacing with network 870. Communication component 864 may include components, for example, adapted to provide wired communication, wireless communication, cellular communication, near field communication (NFC), Bluetooth communication, Wi-Fi, and / or communication via other modalities. Device 880 may include other machines or various peripheral devices (e.g., via USB coupling).

[0104] In some examples, communication component 864 may detect identifiers or include components suitable for detecting identifiers. For example, communication component 864 may include a radio frequency identification (RFID) tag reader, an NFC detector, an optical sensor (e.g., a one-dimensional or multi-dimensional barcode, or other optical code), and / or an acoustic detector (e.g., a microphone for identifying audio signals of the tag). In some examples, location information may be determined based on information from communication component 864, such as, but not limited to, geographic location via Internet Protocol (IP) address, location via Wi-Fi, cellular, NFC, Bluetooth, or other wireless station identification and / or signal triangulation.

[0105] While various embodiments have been described, this description is intended to be exemplary and not restrictive, and it should be understood that many more embodiments and implementations within the scope of these embodiments are possible. Although many possible combinations of features are shown in the drawings and discussed in the detailed embodiments, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature of any embodiment may be used in combination with or in lieu of any other feature or element in any other embodiment. Therefore, it will be understood that any feature shown and / or discussed in this disclosure may be implemented together in any suitable combination. Thus, these embodiments are not limited unless considered in connection with the appended claims and their equivalents. Moreover, various modifications and changes may be made within the scope of the appended claims.

[0106] Typically, the functions described in this article (e.g., Figure 1-) Figure 6The features shown herein can be implemented using software, firmware, hardware (e.g., fixed logic, finite state machines, and / or other circuitry), or a combination of these implementations. In the case of a software implementation, the program code performs a specific task when executed on a processor (e.g., a CPU or several CPUs). The program code can be stored in one or more machine-readable storage devices. The features of the techniques described herein are system-independent, meaning that these techniques can be implemented on various computing systems having various processors. For example, implementations may include entities (e.g., software) that cause hardware to perform operations (e.g., processor function blocks). Hardware devices may include machine-readable media that can be configured to maintain instructions that cause the hardware device (including an operating system running thereon and associated hardware) to perform operations. Thus, the instructions can be used to configure the operating system and associated hardware to perform operations, thereby configuring or otherwise adapting the hardware device to perform the functions described above. The instructions can be provided by the machine-readable medium to the hardware elements executing the instructions in various different configurations.

[0107] In the foregoing detailed embodiments, numerous specific details have been illustrated by way of example to provide a thorough understanding of the relevant teachings. It will be apparent to those skilled in the art, upon reading this specification, that various aspects can be implemented without these details. In other instances, well-known methods, procedures, components, and / or circuits have been described at a relatively high level, but without detailed explanation to avoid unnecessarily obscuring aspects of the teachings herein.

[0108] While the content and / or other examples considered to be best practices have been described above, it should be understood that various modifications may be made therein, and the subject matter disclosed herein can be implemented in a variety of forms and examples. These teachings are applicable to a wide range of applications, only a few of which are described herein. The appended claims are intended to protect any and all applications, modifications, and variations that fall within the true scope of these teachings.

[0109] Unless otherwise stated, all measurements, values, ratings, locations, sizes, dimensions, and other specifications set forth in this specification, including in the appended claims, are approximate and not precise. They are intended to have a reasonable range consistent with the functions they address and with the custom in the fields to which they relate.

[0110] The scope of protection is limited only by the appended claims. When interpreted in accordance with this specification and subsequent application history, this scope is intended and should be interpreted as consistent with the ordinary meaning of the language used in the claims and covers all structural and functional equivalents. Nevertheless, none of the claims are intended to include subject matter that does not meet the requirements of Sections 101, 102, or 103 of the Patent Act, nor should they be interpreted in this manner. Therefore, no unintended inclusion of such subject matter is claimed.

[0111] Apart from the foregoing, nothing set forth or described is intended or should be construed as resulting in any component, step, feature, object, benefit, advantage, or equivalent content being exclusive to the public, whether or not it is described in the claims.

[0112] It will be understood that, unless otherwise specified herein, the terms and expressions used herein have the general meaning consistent with those of such terms and expressions in relation to their respective areas of exploration and research.

[0113] Relational terms such as "first" and "second" may be used only to distinguish one entity or action from another, without necessarily requiring or implying any actual such relationship or order between these entities or actions. The terms "comprises," "comprising," and any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such a process, method, article, or apparatus. Unless otherwise limited, an element preceded by "a" or "an" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes that element.

[0114] An abstract of this disclosure is provided to enable the reader to quickly identify the nature of the technical disclosure. The abstract is submitted based on the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Furthermore, as can be seen in the foregoing detailed description, various features are combined in different examples for the purpose of simplification. The method of this disclosure should not be construed as reflecting an intention that any claim requires more features than those explicitly stated in the claim. Rather, as reflected in the following claims, the inventive subject matter lies in fewer than all features of a single disclosed example. Therefore, with each claim serving as an example of a separately claimed subject matter, the following claims are incorporated herein by reference.

Claims

1. A data processing system, comprising: Processor (810), and A memory (830) stores executable instructions that, when executed by the processor, cause the processor, alone or in combination with other processors (810), to perform the following functions: Receive user input defining what content on a specific platform will be blocked as harmful content (101). Based on the user input (101), a custom prompt word template (102) is generated and submitted to the generative artificial intelligence (GAI) annotation service (105). The custom prompt word template (102) instructs the GAI annotation service (105) to annotate the harmful content in the annotation request (103) according to the user input that defines what content on a specific platform will be blocked as harmful content. Receive annotation request (103), the annotation request (103) including platform content to be checked for harmful content based on the user input; By combining the platform content with the customized prompt word template (102), prompt words (104) for the GAI annotation service (105) are generated. Compress the prompt word (104) before submitting it to the GAI annotation service (105); and Using the prompt word (104), specifically based on the user input that defines what content is harmful content, the platform annotation service (105) generates an identifier of harmful content to be blocked in the platform content.

2. The data processing system according to claim 1, wherein, The custom prompt word template (102) includes several examples of harmful content.

3. The data processing system according to claim 2, wherein, The example of the harmful content was generated by GAI (120) based on the user input (101).

4. The data processing system according to claim 2, wherein, Each of the examples of harmful content is labeled based on a set of labels, each label being applied to a different type of harmful content and having an associated definition of that type of harmful content based on the user input (101).

5. The data processing system according to claim 2, wherein, GAI (120) is invoked to associate a label in the label set with each of the examples, each label being applied to a different type of harmful content and having an association definition for that type of harmful content based on the user input (101).

6. The data processing system according to claim 5, wherein: The GAI (120) includes multiple models (161); and Specific examples in the custom prompt word template (102) are associated with annotations in the annotation set and confidence statements based on the outputs of the multiple models (161).

7. The data processing system according to claim 6, wherein, When the multiple models (161) disagree on which label to associate with the particular example, the confidence statement associated with the particular example indicates a reduced confidence level for the associated label.

8. The data processing system according to claim 6, wherein, The annotation associated with the specific example is selected based on the output consistent with the majority of the multiple models (161).

9. The data processing system according to any one of claims 1-8, wherein, The customized prompt word template (102) is generated by calling GAI (120) based on the user input (101) using a request, the request being for GAI (120) to generate the customized prompt word template (102) based on the user input.

10. The data processing system according to claim 9, wherein, The user input includes a description of different types of harmful content to be blocked in the platform content, and the call to the GAI (120) is used to generate the custom prompt word template (102) that instructs the GAI (120) to define a set of annotations, each annotation being applied to different types of harmful content and having an associated definition of that type of harmful content based on the user input.

11. The data processing system according to claim 10, wherein, The user input (101) includes only keywords that identify the different types of harmful content.

12. The data processing system according to any one of claims 1-11, wherein, The customized prompt word template (102) includes a set of annotations, each annotation being applied to different types of harmful content and having an association definition of that type of harmful content based on the user input (101).

13. The data processing system according to claim 12, wherein, The identifier of the harmful content includes annotations (106) of the platform content, the annotations including the application of the annotation set to the corresponding harmful content in the platform content.

14. The data processing system according to any one of claims 1-13, further comprising: The processor (810) is enabled to block executable instructions for identified harmful content on the platform based on the identifier of the harmful content.

15. The data processing system according to any one of claims 1-14, wherein: The user input (101) provides or is used to generate a set of labels for different types of harmful content to be identified in the content of the platform; and Compressing the prompt word (104) includes: using a request (106) to invoke GAI (120) to identify a subset of the set of annotations applied to the platform content, and removing information from the prompt word (104) about annotations that are not identified by GAI (120) as applied to the platform content.

16. The data processing system according to any one of claims 1-15, wherein, Compressing the prompt word (104) includes: determining the probability of each consecutive word in the platform content, and deleting words from the prompt word (104) whose probability is higher than a threshold.

17. A method for blocking harmful content from platform content targeting a specific online community, the method comprising: Receive user input defining what content on a specific platform will be blocked as harmful content (101). Based on the user input (101), a custom prompt word template (102) is generated and submitted to the Generative Artificial Intelligence (GAI) (120) annotation service. The custom prompt word template (102) instructs the GAI annotation service (105) to annotate the harmful content in the annotation request (103) according to the user input that defines what content on a specific platform will be blocked as harmful content. Receive annotation request (103), the annotation request (103) including platform content to check for any harmful content based on the user input; By combining the platform content with the customized prompt word template (102), prompt words (104) for the GAI annotation service (105) are generated. as well as Specifically, based on the user input defining what content is harmful, the platform receives identifiers of harmful content to be blocked from the GAI annotation service (105).

18. The method of claim 17, further comprising: Based on the user input and using the request (106), the custom prompt word template (102) is generated by calling GAI (120), the request (106) being a request from GAI (120) to generate the custom prompt word template (102) based on the user input.

19. A system for blocking harmful content from a platform serving an online community, the community having guidelines on what constitutes harmful content, the system comprising: A processor (810) that communicates with the platform to receive platform content to be inspected for harmful content; A memory (830) stores executable instructions that, when executed by the processor, cause the processor, alone or in combination with other processors (810), to perform the following functions: Based on the guidelines, a custom prompt word template (102) is generated and submitted to the generative artificial intelligence (GAI) annotation service (105), the custom prompt word template (102) instructing the GAI annotation service (105) to annotate harmful content in the annotation request (103) according to the guidelines; Receive annotation request (103), the annotation request (103) including platform content to be checked for any harmful content; By combining the platform content with the customized prompt template (102), prompts (104) are generated for the GAI annotation service (105); and Using the aforementioned prompt (104), specifically based on the guidelines defining what content is harmful, the platform annotation service (105) generates an identifier for harmful content to be blocked in the platform content.

20. The system of claim 19, further comprising: The processor compresses the executable instructions of the prompt word (104) before submitting them to the GAI annotation service (105).

Images