A large model-based risk content identification method and device

Abstract

Embodiments of the present specification provide a method and device for identifying risk content based on a large model. In the method, a first text reflecting the reasoning of experts is generated according to the analysis report of experts on historical risk content, a first large model is trained according to the historical risk content and the first text, and a trained second large model is obtained. A first prompt word is constructed based on the risk supervision requirement text issued by the regulatory department and the corresponding risk content sample, and the first prompt word contains a first task instruction. The first task instruction is used to instruct to generate a risk qualitative rule for identifying risk content based on the risk supervision requirement text and the risk content sample. The first prompt word is input into the second large model to obtain the risk qualitative rule, and the first content to be identified in the content platform is identified according to the risk qualitative rule. Privacy protection is required for privacy data in the data processing process.

Description

Technical Field

[0001] This specification relates to the field of natural language processing technology in one or more embodiments, and in particular to a method and apparatus for identifying risky content based on a large model. Background Technology

[0002] With the rapid development of the internet content ecosystem, regulatory authorities have imposed strict risk control requirements on online content platforms, requiring these platforms to assume primary responsibility for content guidance management and risk assessment. Regulatory authorities periodically issue risk supervision requirements and risk samples targeting illegal or harmful information, demanding that content platforms respond quickly and implement measures such as deletion and blocking. Simultaneously, they also require the anonymization of privacy data on content platforms.

[0003] Currently, there is a desire for improved solutions that can enhance the timeliness of content platforms' risk control responses. Summary of the Invention

[0004] This specification describes one or more embodiments of a risky content identification method and apparatus based on a large model, to more promptly identify risky content in content platforms. The specific technical solution is as follows.

[0005] Firstly, the embodiments provide a risk content identification method based on a large model, including:

[0006] Based on the expert analysis report on historical risk content, a first text reflecting the expert's reasoning is generated; the historical risk content is input into the first large model to obtain a second text representing the risk reasoning for prediction; based on the difference between the second text and the first text, the parameters of the first large model are updated to obtain the second large model.

[0007] A first prompt word is constructed based on the risk supervision requirements text issued by the regulatory authorities and the corresponding risk content sample. The first prompt word includes a first task instruction, which is used to instruct the generation of risk qualitative rules for identifying risk content based on the risk supervision requirements text and the risk content sample.

[0008] Input the first prompt word into the second large model to obtain the risk qualitative rule;

[0009] Based on the aforementioned risk characterization rules, risk identification is performed on the first piece of content in the content platform that is subject to risk identification.

[0010] In one implementation, the first text and the second text are thought chain texts; the step of generating the first text reflecting the expert's reasoning process based on the expert's analysis report on historical risk content includes:

[0011] The analysis report of experts on historical risk content is input into the tool's large model, which then generates a first text that conforms to the thought chain format and reflects the expert's reasoning.

[0012] In one implementation, the first text includes the following: a section reflecting how to understand the content, a section reflecting how to understand the risk, and a section reflecting how to determine the risk.

[0013] In one implementation, the method further includes:

[0014] Inputting historical risk information into the second major model yields a third text representing the risk reasoning logic for prediction;

[0015] When the difference between the third text and the first text is greater than a preset threshold, the first text is taken as the text to be corrected.

[0016] Based on the expert's correction operation on the first text input, the corrected first text is obtained, and the second major model is trained using the historical risk content and the corrected first text.

[0017] In one implementation, the risk characterization rule includes risk keywords for recalling relevant content and detailed judgment rules for content identification; the step of identifying the first content to be identified as having risk according to the risk characterization rule includes:

[0018] When the first content matches the risk keyword, a second prompt word containing the discrimination rules, the first content, and the second task instruction is constructed through the risk control engine. The second prompt word is input into the third model, and the identification result of whether the first content belongs to risk content is obtained through the third model. The second task instruction is used to instruct whether the first content belongs to risk content according to the discrimination rules and to obtain the identification result.

[0019] In one implementation, before constructing the second prompt word, the method further includes: taking existing content published by the content platform as the first content to be identified, matching the first content with the risk keyword, and obtaining a result on whether the first content matches the risk keyword.

[0020] In one implementation, before constructing the second prompt word, the method further includes:

[0021] Add the aforementioned risk qualitative rules to the rule base;

[0022] The newly added content published by the content platform is taken as the first content to be identified, and the first content is matched with the risk keywords corresponding to several risk qualitative rules in the rule base;

[0023] The step of constructing a second prompt word that includes the discrimination rules, the first content, and the second task instruction includes:

[0024] When the first content matches several risk keywords, determine the k discrimination rules with the highest similarity to the first content from several discrimination rules associated with the several risk keywords;

[0025] A second prompt word is constructed, which includes the k discrimination rules, the first content, and the second task instruction. The second task instruction is specifically used to instruct whether the first content belongs to risky content according to the k discrimination rules, and to obtain the identification result.

[0026] In one implementation, the step of constructing a second prompt word that includes the discrimination rules, the first content, and the second task instruction includes:

[0027] Construct a second prompt word that includes the discrimination rules, the first content, the risk content sample, risk-related knowledge, and a second task instruction; wherein, the second task instruction is specifically used to instruct whether the first content belongs to risk content based on the discrimination rules, the risk content sample, and the risk-related knowledge, and to obtain the identification result.

[0028] Secondly, the embodiments provide a risk content identification device based on a large model, including:

[0029] The model training module is configured to generate a first text reflecting the expert's reasoning process based on the expert's analysis report on historical risk content; input the historical risk content into the first large model to obtain a second text representing the risk reasoning process for prediction; and update the parameters of the first large model based on the difference between the second text and the first text to obtain the second large model.

[0030] The prompt word construction module is configured to construct a first prompt word based on the risk supervision requirement text issued by the regulatory authority and the corresponding risk content sample, wherein the first prompt word includes a first task instruction, which is used to instruct the generation of risk qualitative rules for identifying risk content based on the risk supervision requirement text and the risk content sample.

[0031] The rule generation module is configured to input the first prompt word into the second large model to obtain risk qualitative rules;

[0032] The risk identification module is configured to identify the first content with a risk to be identified in the content platform according to the risk characterization rules.

[0033] Thirdly, the embodiments provide a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method described in any one of the first aspects.

[0034] Fourthly, an embodiment provides a computing device including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the method described in any one of the first aspects.

[0035] In the methods and apparatus provided in the embodiments of this specification, a first major model is trained using historical risk content and expert risk reasoning, enabling a second major model to possess the ability to think about risks. Upon receiving risk regulatory requirements texts and risk content samples issued by regulatory authorities, the second major model, based on its learned expert reasoning and the risk regulatory requirements texts and risk content samples, generates risk qualitative rules for identifying risky content. In these embodiments, utilizing a second major model with risk reasoning capabilities allows for more timely generation of risk qualitative rules with higher accuracy, thereby enabling more timely risk identification of content on the content platform. Attached Figure Description

[0036] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0037] Figure 1 This is a schematic diagram illustrating an implementation scenario of one embodiment disclosed in this application;

[0038] Figure 2 A flowchart illustrating a risk content identification method based on a large model is provided as an embodiment.

[0039] Figure 3 A flowchart illustrating a risk content identification method provided in one embodiment;

[0040] Figure 4 This is a schematic block diagram of a risk content identification device based on a large model, provided for an embodiment. Detailed Implementation

[0041] The solution provided in this specification will now be described with reference to the accompanying drawings.

[0042] Figure 1This is a schematic diagram illustrating an implementation scenario of one embodiment disclosed in this application. It includes a computing device and a content platform. Experts analyze historical risk content to obtain an analysis report. The computing device then generates text reflecting the expert's reasoning process based on this report. The computing device trains a large language model on multiple historical risk content sets and corresponding expert reasoning texts, enabling it to learn the expert's reasoning approach and acquire a certain level of risk assessment capability. When regulatory authorities issue risk supervision requirements and related risk samples, the computing device, through its locally deployed large language model, generates risk characterization rules for risk identification based on the risk supervision requirements and related risk content samples. The computing device can then use these risk characterization rules to promptly identify risks in the content on the content platform.

[0043] Content platforms are internet service platforms that connect content creators and users. Their core functions are content production, distribution, consumption, and management. Content platforms use algorithms or manual recommendations to match suitable content to interested users, while also assuming responsibility for content compliance and risk control. Content platforms showcase compliant content to users through content communities.

[0044] The content published on content platforms primarily includes text, images, videos, and audio. Text-based content includes articles, posts, comments, Q&A, and novels. Image-based content includes pictures, posters, emoticons, and mixed text and image formats. Video-based content includes short videos, long videos, live streams, and GIFs. Audio-based content includes podcasts, music, audiobooks, and voice messages. Content published on content platforms must undergo risk identification according to existing risk assessment rules and be confirmed to pose no risk before publication.

[0045] Regulatory authorities oversee content platforms to ensure that the content they publish is compliant and risk-free. These authorities distribute specific risk control requirements and corresponding risk samples to content platforms, enabling them to manage the risks associated with their content and ensure that all published content is risk-free. The risk content samples serve as reference data for the risk control requirements.

[0046] The risk management requirements issued by regulatory authorities can cover risks at different levels, such as illegality, violation of public order and good morals, and low quality. Correspondingly, risky content can include categories such as illegal content, content that violates public order and good morals, and low-quality content.

[0047] Figure 1This is merely one implementation scenario of the risk content identification method in this application. In practical applications, the data processing operations performed by the computing device can also be performed by the content platform. The computing device or content platform can be implemented by any device, equipment, platform, or cluster of devices with computing and processing capabilities. The following example illustrates a scenario where the computing device and content platform are implemented using different devices.

[0048] The computing device can generate corresponding risk assessment rules based on the risk supervision requirements and risk content samples issued by regulatory authorities, using a large language model, and then send these rules to the content platform. The content platform can then use these risk assessment rules to identify the risks in the content, and if the content is deemed risky, it will not be displayed to users in the content community. The purpose of risk control in the content community is to purify the community content by identifying, blocking, and handling illegal content in real time, preventing the spread of illegal information, ensuring the platform complies with laws and regulations, and avoiding legal risks. Simultaneously, it can also remove low-quality, harassing, fraudulent, and misleading content, maintaining a positive community atmosphere and ensuring users have a safe and high-quality browsing experience. Risk control acts as a filter and gatekeeper for the content community, ensuring its legality, health, and credibility by quickly removing harmful or low-quality content.

[0049] Risk characterization rules can be implemented as logical judgment standards to clearly define whether content violates regulations, including conditions (IF) and results (THEN). For example, if the "transfer" mentioned in the content is used to share a legitimate invoice reimbursement process, it is not a violation; if "transfer" is mentioned and accompanied by intentions such as "bypassing platform supervision" or "private discounts," it is determined to be a violation. The result can include violation or no violation. Risk characterization rules can be expressed in text form using natural language.

[0050] After regulatory authorities issue risk control requirements, content platforms need to promptly formulate risk assessment rules to quickly control content based on these rules. However, existing risk control solutions mainly rely on human experts to analyze key risk points and formulate rules. This process is lengthy, often taking several days from receiving regulatory instructions to completing risk analysis, rule training, and final implementation. This lag makes it difficult for content platforms to meet the stringent regulatory requirements for the timeliness of risk identification, easily leading to the spread of risky content during the handling window. This not only affects the efficiency of content ecosystem governance but may also expose the platform to compliance risks. Therefore, shortening the cycle from the issuance of regulatory requirements to the completion of full-scale content risk identification and improving the timeliness and completeness of risk control response has become an urgent problem to be solved in the field of content security management.

[0051] In this embodiment, the large language model is trained using historical risk content and corresponding reasoning texts provided by experts, enabling it to learn risk thinking ability. The large language model with risk thinking ability obtained through training can obtain risk identification rules more timely and accurately, thereby enabling more timely risk control of content and effectively shortening the cycle from the completion of full content risk identification under regulatory requirements.

[0052] Large Language Models (LLMs), also known simply as large models, are natural language processing models based on deep learning techniques. Their parameter count typically ranges from billions to hundreds of billions or even higher, giving them powerful language understanding and generation capabilities. LLMs can employ the Transformer architecture or its variants (such as GPT and BERT), which utilizes an attention mechanism to globally model sequential data, efficiently handling long-distance dependencies and thus performing exceptionally well in natural language tasks. By pre-training on large-scale corpora, LLMs learn the statistical features and semantic relationships of language, giving them excellent generalization abilities. They can be further trained on domain-specific datasets of a certain size to optimize their performance on specific tasks. These datasets can be labeled or unlabeled. After further training, the powerful generalization capabilities and flexibility of LLMs make them important tools in specific domains, providing efficient and accurate solutions for automated text generation and understanding.

[0053] Large language models are typically used in two modes: direct inference and fine-tuning. In direct inference mode, users guide the large language model to generate specific outputs by designing prompts. Prompts can be textual descriptions of the task or instructions used to stimulate the large language model's semantic understanding and generation capabilities.

[0054] In some embodiments, large language models can also understand and generate data from other modalities (such as visual and audio data). In this case, large language models can also be called multimodal large language models (MLLMs). MLLMs provide a richer and more natural interactive experience by integrating multiple types of input and output, such as text, images, and sound. The core advantage of MLLMs lies in their ability to process and understand information from different modalities and fuse this information to complete complex tasks. For example, MLLMs can analyze an image and generate descriptive text, or generate a corresponding image based on a text description. This cross-modal understanding and generation capability makes MLLMs widely applicable across multiple fields.

[0055] It should be noted that the key technologies of large language models can be found in the detailed description in the paper "A Survey of Large Language Models" (paper number: arXiv:2303.18223v16, published on March 11, 2025), and will not be repeated here. All the large models mentioned in this embodiment can be implemented based on the aforementioned large language models.

[0056] The following is combined with Figure 2 The embodiments are described in detail below.

[0057] Figure 2 This is a flowchart illustrating a risk content identification method based on a large model, provided as an embodiment. The method is executed via a computing device and specifically includes the following steps.

[0058] Step S210: Based on the expert's analysis report on historical risk content, generate a first text that reflects the expert's reasoning. Input the historical risk content into the first large model N1 to obtain a second text that represents the risk reasoning for prediction. Based on the difference between the second text and the first text, update the parameters of the first large model N1 to obtain the second large model N2.

[0059] The original historical risk content can be in the form of text, images, videos, or audio. The historical risk content input into the first major model N1 can be text described in natural language, or text after natural language recognition of images, videos, or audio.

[0060] Historical risk content refers to content that carries risk, and the corresponding risk types can be different levels of risk, such as illegal risk, risk of violating public order and good morals, and low-quality risk.

[0061] The aforementioned analysis report and the first text include the following sections: sections demonstrating how to understand the content, sections demonstrating how to understand the risks, and sections demonstrating how to determine the risks. The content here refers to historical risk information.

[0062] The first and second texts mentioned above can be considered chain of thought texts. A chain of thought (COT) is a process of progressively breaking down a complex problem into subproblems and solving them sequentially. The intermediate steps in the reasoning process of a large model on input data are called the chain of thought.

[0063] The analysis report includes the reasoning process by which experts assessed historical risks, but this reasoning process may not be arranged in an ordered manner. Therefore, in practical implementation, the expert analysis report on historical risks can be input into a large-scale tool model. The tool model then transforms the analysis report into a first text that conforms to the thought chain format and reflects the expert's reasoning process. This first text can serve as standard or labeled data for the thought chain.

[0064] The first large model N1 mentioned above is a pre-trained large model. The second large model N2 is a large model obtained by training the first large model N1 using historical risk content and the first text. The first large model N1 can be deployed locally on the computing device to ensure that the data is not sent out and to guarantee data security.

[0065] When updating the parameters of the first major model N1 using historical risk content and the first text, the prediction loss is determined based on the difference between the second text and the first text. This prediction loss is then used to update the parameters of the first major model N1, enabling it to learn expert reasoning strategies. Multiple historical risk content sets and their corresponding first texts can be used to iterate through multiple rounds of the first major model N1. After multiple iterations, a second major model N2 is obtained, which possesses risk-thinking capabilities.

[0066] The prediction loss can be obtained by calculating the cross-entropy between each token in the first text and each token in the second text. The first large model N1 can also output the judgment result of whether historical risk content belongs to risk content. The classification loss is determined based on the difference between the judgment result and the actual result. The parameters of the first large model N1 are updated based on the sum of the classification loss and the prediction loss.

[0067] In this embodiment, the second large model N2 obtained after training has a certain risk reasoning ability, so it can be used to detect the first text with low accuracy and provide it to experts for correction. The corrected first text can continue to train the second large model N2.

[0068] Step S220: Construct the first prompt word prompt1 based on the risk supervision requirements text issued by the regulatory authorities and the corresponding risk content samples.

[0069] The risk regulatory requirement text refers to regulatory requirements issued for at least one type of risk. The risk content sample is illustrative content containing that risk; it can be one or more, and can take the form of natural language text, images, videos, or audio. When constructing the first prompt word (prompt1), the image, video, or audio content can be converted into text expressed in natural language.

[0070] The first prompt word, prompt1, includes the text of the risk regulatory requirements, a sample of risk content, and the text of the first task instruction. The first task instruction is used to instruct the generation of risk characterization rules for identifying risk content based on the risk regulatory requirements text and the sample of risk content.

[0071] In practical applications, the risk regulatory requirement text and the corresponding risk content sample can be filled into the placeholders in the prompt template to obtain the first prompt word, prompt1. The prompt template contains placeholders representing the risk regulatory requirement text, placeholders representing the risk content sample, and the text of the first task instruction.

[0072] Step S230: Input the first prompt word prompt1 into the second large model N2 to obtain the risk qualitative rule R1.

[0073] The resulting risk qualitative rule R1 can be one or more. The second major model N2, based on the expert reasoning and risk reasoning capabilities learned from it, generates the risk qualitative rule R1 based on the input risk regulatory requirement text and risk content samples. The risk qualitative rule R1 embodies the risk reasoning approach (COT). The second major model N2 is a large model deployed locally on the computing device.

[0074] Step S240: Based on the aforementioned risk characterization rule R1, perform risk identification on the first content C1 in the content platform to be identified, and obtain the risk identification result.

[0075] The risk identification results include whether the first content C1 is a risky content or not.

[0076] The computing device can send the risk identification rule R1 to the content platform, so that the content platform can identify the first content C1 to be identified based on the risk identification rule R1 and obtain the risk identification result.

[0077] The computing device can also receive the first content C1 of the risk to be identified sent by the content platform, and perform risk identification on the first content C1 according to the above-mentioned risk characterization rule R1 to obtain the risk identification result.

[0078] When identifying risks in the first content C1, the text of the first content C1 can be matched with the condition part of the risk characterization rule R1 to determine whether it meets the condition. If it does, the conclusion in the risk characterization rule R1 is obtained. If it does not meet the condition, no action is taken. The risk characterization rule R1 is a rule that can qualitatively determine whether there is risk in the first content C1, and it is also a rule that judges whether it contains risk.

[0079] In this embodiment, the large model is trained to absorb the reasoning and thinking of experts, thereby enabling the large model to reason about risk content. Based on this, relevant information can be extracted quickly and completely and refined into risk qualitative rules, which are then quickly transmitted to the risk control engine as application material, thereby enabling rapid risk identification.

[0080] In another embodiment of this application, any risk characterization rule R1 includes several risk keywords K1 for recalling relevant content and several judgment rules X1 for content identification. The risk keywords K1 and the judgment rules X1 correspond to each other. "Several" includes one or more.

[0081] Risk keyword K1 is a feature identifier used to quickly filter out potentially relevant content from massive amounts of data. Its function is to improve retrieval efficiency by recalling relevant content, avoiding direct matching of all content with the judgment criteria, thereby reducing data processing costs. Risk keyword K1 can be in the form of words or phrases, such as transfers, private transactions, bypassing platforms, and no fees. Risk keyword K1 can be plain text or descriptive features of images or multimodal content.

[0082] The judgment criteria X1 is the logical basis for assessing the risks associated with the recall. It typically includes two parts: conditions and results, and sometimes also includes exemption conditions. For example, the judgment criteria follow the logic of "if condition A is met and condition B is met, then the result is C" or "if condition A is met and there is no exemption condition B, then the result is C".

[0083] In step S240, when identifying the first content C1 of the risk to be identified according to the risk characterization rule R1, the specific steps 1 and 2 can be performed.

[0084] Step 1: The risk control engine determines whether the first content C1 matches the aforementioned risk keyword K1. If the first content C1 contains the risk keyword K1, it is determined that the first content C1 matches the aforementioned risk keyword K1. If the first content C1 does not contain the risk keyword K1, it is determined that the first content C1 does not match the aforementioned risk keyword K1. The risk control engine is deployed on a computing device.

[0085] Step 2: When the first content C1 matches the aforementioned risk keyword K1, the risk control engine constructs a second prompt word prompt2, which includes the judgment details X1 corresponding to the aforementioned keyword, the first content C1, and the second task instruction text. The second prompt word prompt2 is input into the third major model N3, and the identification result of whether the first content belongs to risk content is obtained through the third major model N3.

[0086] The second task instruction is used to instruct whether the first content C1 is risky content according to the aforementioned discrimination rule X1, and to obtain the identification result. When constructing the second prompt word prompt2, the aforementioned discrimination rule X1 and the first content C1 can be filled into the placeholders of a pre-constructed prompt template. The prompt template also contains the text of the second task instruction.

[0087] In one implementation, a second prompt word, prompt2, can be constructed, comprising the aforementioned discrimination rules X1, the first content C1, a risk content sample, risk-related knowledge, and a second task instruction. The second task instruction specifically instructs the identification of whether the first content C1 is risky content based on the discrimination rules, the risk content sample, and the risk-related knowledge, and obtains the identification result. In practical applications, this second prompt word, prompt2, is input into the third major model N3 to obtain the identification result of whether the first content C1 is risky content. Combining the risk content sample and risk-related knowledge to judge the risk of the first content C1 yields a more accurate judgment result.

[0088] The third major model, N3, can be deployed locally to ensure data security. The third major model, N3, can be implemented using a pre-trained general-purpose large model.

[0089] Figure 3 This is a flowchart illustrating a risk content identification method provided in one embodiment. When a computing device receives a regulatory instruction, it can convert the instruction into a risk characterization rule R1 containing risk keywords K1 and corresponding judgment details X1 using a second major model N2. The regulatory instruction includes risk regulatory requirements texts issued by regulatory authorities and risk content samples, etc.

[0090] Once experts confirm the generated risk assessment rule R1, it can be applied to the existing content on the content platform to identify risks in the existing content.

[0091] Existing content refers to historical content that the content platform published and is currently displaying or storing before the current point in time. This existing content has not been used for risk identification using the risk assessment rule R1. When a new risk assessment rule R1 is generated, it can be used to identify risks in the platform's existing content, excluding content that matches the rule.

[0092] In practice, existing content published on the content platform can be used as the first content to be identified, C1. This first content C1 is then matched against the risk keyword K1 to determine whether it matches K1. The first content C1 that matches K1 is then used as the relevant existing content to be recalled, and a second prompt word, prompt2, is constructed. Prompt2 includes the relevant existing content to be recalled, the judgment rules X1, and the second task instruction.

[0093] Continue with step 2, which involves inputting the second prompt word (prompt2) into the third model (N3) to obtain the identification results of whether the recalled relevant existing content belongs to risky content.

[0094] When the first content C1 does not match the risk keyword K1, it can be excluded from matching with the judgment rule X1, that is, it will not be constructed as the second prompt word prompt2, and the first content C1 is considered not to be risky content.

[0095] Simultaneously, when generating a risk assessment rule R1, it can be added to the rule base. The rule base contains multiple risk assessment rules, including previous risk assessment rules and newly generated risk assessment rule R1. The rule base includes risk assessment rules corresponding to all risk monitoring requirements issued by regulatory authorities. Any publicly available content on the content platform needs to be matched against all risk assessment rules to ensure it does not constitute risky content. Risk keywords in the rule base can be used as indexes for risk assessment rules.

[0096] For any new content published on the content platform, this new content is taken as the first content to be identified, C1. This new content is then matched with the risk keywords corresponding to all risk qualitative rules in the rule base to obtain the result of whether the new content matches the risk keyword K1.

[0097] When the newly added content matches several risk keywords K1, the k judgment rules with the highest similarity to the newly added content are determined from several judgment rules associated with several risk keywords K1, and a second prompt word prompt2 containing the k judgment rules, the newly added content and the second task instruction is constructed.

[0098] The second task instruction specifically instructs the identification of whether the newly added content constitutes risk content based on the k discrimination rules, and obtains the identification result. The risk keyword K1 matched by the newly added content may be one or more, meaning it may match one or more risk qualitative rules. k is an integer greater than 1.

[0099] When determining the similarity between the discrimination rules and the newly added content, the semantic similarity between the condition part of the discrimination rules and the newly added content can be determined.

[0100] After constructing the second prompt word (prompt2), step 2 can be executed, inputting the second prompt word (prompt2) into the third model (N3) to obtain the risk identification result of whether the newly added content belongs to risky content. Simultaneously, the third model (N3) can also output the risk reasoning process (COT) data.

[0101] The original new content can be in the form of natural language text, images, videos, or audio. When identifying risks associated with this new content, it can be converted into the corresponding natural language text for risk assessment.

[0102] Upon receiving the identification result that the first content C1 is risky content, an audit task can be generated for the qualitatively identified risky content. This audit task is then transferred to audit experts for review and confirmation of whether the first content C1 is indeed risky content. The audit task may include the first content C1, relevant risk identification rules, and COT data output by the third major model N3.

[0103] During the review process, when review experts are unfamiliar with the risk characterization rules, they can use a locally deployed AI Q&A assistant for clarification. This assistant can provide review experts with Q&A services based on the risk inference rules it has learned. The AI ​​Q&A assistant can be a different large model than the second and third large models, N2 and N3.

[0104] In this embodiment, the reasoning process and related rules for risk content determination by the third major model N3 are output to the review experts, thereby reducing the memory burden on them. When review experts need to elaborate on the detailed content of specific rules, they can quickly query the rules through the rule Q&A assistant.

[0105] It is understood that before or during the use of the technical solutions of the various embodiments in this disclosure, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner in accordance with relevant laws and regulations, and user authorization will be obtained.

[0106] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations of this disclosed technical solution.

[0107] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0108] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0109] In this specification, the terms "first," "first text," "first prompt word," and "first content," as well as the corresponding "second" (if present) in the text, are used merely for ease of distinction and description and do not have any limiting meaning.

[0110] The foregoing description describes specific embodiments of this specification; other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than those shown in the embodiments, and the desired result may still be achieved. Furthermore, the processes depicted in the drawings do not necessarily need to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0111] Figure 4 This is a schematic block diagram of a risk content recognition device based on a large model, provided for an embodiment. This device embodiment is related to... Figure 2 The method embodiment shown corresponds to this. The device 400, deployed in a computing device, includes the following modules: A model training module 410, configured to generate a first text reflecting the expert's reasoning based on an expert's analysis report on historical risk content; input the historical risk content into a first large model to obtain a second text representing the predicted risk reasoning; and update the parameters of the first large model based on the difference between the second text and the first text to obtain a second large model. A prompt word construction module 420, configured to construct a first prompt word based on risk regulatory requirements text issued by regulatory authorities and corresponding risk content samples, wherein the first prompt word contains a first task instruction, which instructs the generation of risk qualitative rules for identifying risk content based on the risk regulatory requirements text and the risk content samples. A rule generation module 430, configured to input the first prompt word into the second large model to obtain risk qualitative rules. A risk identification module 440, configured to perform risk identification on the first content to be identified in the content platform according to the risk qualitative rules.

[0112] In one implementation, the first text and the second text are thought chain texts. The model training module 410, when generating the first text reflecting the expert's reasoning based on the expert's analysis report on historical risk content, includes: inputting the expert's analysis report on historical risk content into the tool model, and obtaining the first text conforming to the thought chain form and reflecting the expert's reasoning through the tool model.

[0113] In one implementation, the first text includes the following: a section reflecting how to understand the content, a section reflecting how to understand the risk, and a section reflecting how to determine the risk.

[0114] In one implementation, the device 400 further includes the following modules: a model prediction module (not shown in the figure), configured to input the historical risk content into a second large model to obtain a third text representing the risk reasoning logic for the prediction; a correction determination module (not shown in the figure), configured to use the first text as the text to be corrected when the difference between the third text and the first text is greater than a preset threshold; and a text correction module (not shown in the figure), configured to obtain a corrected first text based on the correction operation performed by an expert on the first text input, and to train the second large model using the historical risk content and the corrected first text.

[0115] In one implementation, the risk qualitative rules include risk keywords for recalling relevant content and discrimination rules for content identification. The risk identification module 440 includes the following sub-modules: a construction sub-module 41, configured to construct a second prompt word containing the discrimination rules, the first content, and a second task instruction when the first content matches the risk keyword; and a qualitative sub-module 43, configured to input the second prompt word into a third model to obtain an identification result indicating whether the first content is risky content. The second task instruction is used to instruct whether the first content is risky content according to the discrimination rules and to obtain the identification result.

[0116] In one implementation, the device 400 further includes: a first matching module 450, configured to, before constructing the second prompt word, take existing content published by the content platform as the first content to be identified, match the first content with the risk keyword, and obtain a result of whether the first content matches the risk keyword.

[0117] In one implementation, the device 400 further includes the following modules: a rule entry module 460, configured to add the risk qualitative rules to a rule base before constructing the second prompt word; a second matching module 470, configured to use newly published content from the content platform as the first content to be identified, and match the first content with risk keywords corresponding to several risk qualitative rules in the rule base; and a construction submodule 41 including the following units: a selection unit (not shown in the figure), configured to determine the k discrimination rules with the highest similarity to the first content from several discrimination rules associated with the several risk keywords when the first content matches several risk keywords; and a construction unit (not shown in the figure), configured to construct a second prompt word containing the k discrimination rules, the first content, and a second task instruction, wherein the second task instruction is specifically used to instruct whether the first content belongs to risk content according to the k discrimination rules, and to obtain the identification result.

[0118] In one implementation, the construction submodule 41 is specifically configured to: construct a second prompt word containing the discrimination rules, the first content, the risk content sample, risk-related knowledge, and a second task instruction; wherein, the second task instruction is specifically used to instruct whether the first content belongs to risk content according to the discrimination rules, the risk content sample, and the risk-related knowledge, and to obtain the identification result.

[0119] The above-described apparatus embodiments correspond to the method embodiments, and detailed descriptions can be found in the description of the method embodiments section, which will not be repeated here. The apparatus embodiments are derived based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments; detailed descriptions can be found in the corresponding method embodiments.

[0120] This specification also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform... Figures 1 to 3 Any one of the methods described.

[0121] This specification also provides a computing device, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement... Figures 1 to 3 Any one of the methods described.

[0122] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for storage media and computing devices are basically similar to the method embodiments, so they are described more simply; relevant parts can be referred to the descriptions of the method embodiments.

[0123] Those skilled in the art will recognize that the functions described in the embodiments of the present invention in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0124] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, or improvements made based on the technical solutions of the present invention should be included within the scope of protection of the present invention.

Claims

1. A risk content identification method based on a large model, comprising: Based on the expert analysis report on historical risk content, a first text reflecting the expert's reasoning is generated; The historical risk content is input into the first large model to obtain the second text representing the risk reasoning idea for prediction. Based on the difference between the second text and the first text, the parameters of the first large model are updated to obtain the second large model. A first prompt word is constructed based on the risk supervision requirements text issued by the regulatory authorities and the corresponding risk content sample. The first prompt word contains a first task instruction, which is used to instruct the generation of risk qualitative rules for identifying risk content based on the risk supervision requirements text and the risk content sample. Input the first prompt word into the second large model to obtain the risk qualitative rule; Based on the aforementioned risk characterization rules, risk identification is performed on the first piece of content in the content platform that is subject to risk identification.

2. The method according to claim 1, wherein the first text and the second text are thought chain texts; the step of generating the first text reflecting the expert's reasoning based on the expert's analysis report on historical risk content includes: The analysis report of experts on historical risk content is input into the tool's large model, which then generates a first text that conforms to the thought chain format and reflects the expert's reasoning.

3. The method according to claim 2, wherein the first text includes the following: a part reflecting how to understand the content, a part reflecting how to understand the risk, and a part reflecting how to determine the risk.

4. The method according to claim 1, further comprising: Inputting the historical risk content into the second major model yields a third text representing the risk reasoning logic for prediction; When the difference between the third text and the first text is greater than a preset threshold, the first text is taken as the text to be corrected. Based on the expert's correction operation on the first text input, the corrected first text is obtained, and the second major model is trained using the historical risk content and the corrected first text.

5. The method according to claim 1, wherein the risk qualitative rules include risk keywords for recalling relevant content and discrimination rules for content discrimination; the step of risk identification of the first content to be identified based on the risk qualitative rules includes: When the first content matches the risk keyword, a second prompt word containing the discrimination rules, the first content, and the second task instruction is constructed through the risk control engine. The second prompt word is input into the third model, and the identification result of whether the first content belongs to risk content is obtained through the third model. The second task instruction is used to instruct whether the first content belongs to risk content according to the discrimination rules and to obtain the identification result.

6. The method according to claim 5, further comprising, before constructing the second prompt word: The existing content published on the content platform is used as the first content to be identified. The first content is matched with the risk keywords to obtain the result of whether the first content matches the risk keywords.

7. The method according to claim 5, further comprising, before constructing the second prompt word: Add the aforementioned risk qualitative rules to the rule base; The newly added content published by the content platform is taken as the first content to be identified, and the first content is matched with the risk keywords corresponding to several risk qualitative rules in the rule base; The step of constructing a second prompt word that includes the discrimination rules, the first content, and the second task instruction includes: When the first content matches several risk keywords, determine the k discrimination rules with the highest similarity to the first content from several discrimination rules associated with the several risk keywords; A second prompt word is constructed, which includes the k discrimination rules, the first content, and the second task instruction. The second task instruction is specifically used to instruct whether the first content belongs to risky content according to the k discrimination rules, and to obtain the identification result.

8. The method according to claim 5, wherein the step of constructing a second prompt word comprising the discrimination rules, the first content, and the second task instruction comprises: Construct a second prompt word that includes the discrimination rules, the first content, the risk content sample, risk-related knowledge, and a second task instruction; wherein, the second task instruction is specifically used to instruct whether the first content belongs to risk content based on the discrimination rules, the risk content sample, and the risk-related knowledge, and to obtain the identification result.

9. A risk content identification device based on a large model, comprising: The model training module is configured to generate a first text reflecting the expert's reasoning process based on the expert's analysis report on historical risk content. The historical risk content is input into the first large model to obtain the second text representing the risk reasoning idea for prediction. Based on the difference between the second text and the first text, the parameters of the first large model are updated to obtain the second large model. The prompt word construction module is configured to construct a first prompt word based on the risk supervision requirement text issued by the regulatory authority and the corresponding risk content sample, wherein the first prompt word includes a first task instruction, which is used to instruct the generation of risk qualitative rules for identifying risk content based on the risk supervision requirement text and the risk content sample. The rule generation module is configured to input the first prompt word into the second large model to obtain risk qualitative rules; The risk identification module is configured to identify the first content with a risk to be identified in the content platform according to the risk characterization rules.

10. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-8.

Images