Risk identification method and device, storage medium and computing equipment

Abstract

The embodiment of the invention provides a risk identification method and device, a storage medium and computing equipment. Comprising the following steps: acquiring target content to be identified and business characteristics related to the target content; determining whether the target content belongs to a risk business range or not according to the business characteristics; if the target content does not belong to the risk business range, determining whether the target content belongs to a risk content range or not according to content features of the target content; and if the target content does not belong to the risk content range, inputting the target content as a cue word into a large language model to obtain an identification result of whether the target content output by the large language model has a risk.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology. More specifically, embodiments of the present disclosure relate to a risk identification method, apparatus, storage medium, and computing device. Background Art

[0002] This section aims to provide background or context for the embodiments of the present disclosure. The descriptions herein are not admitted to be prior art merely because they are included in this section.

[0003] In the related art, traditional risk identification can be divided into two methods: manual review and machine review. These two methods can be carried out separately or in combination.

[0004] The manual review method has the problems of low efficiency, high cost, poor timeliness, and difficulty in coping with risk identification of a large amount of content such as the Internet.

[0005] The machine review method can identify risks in the content to be identified through methods such as sensitive words, rules, and deep learning. However, it is difficult to achieve a high recognition accuracy, and it is difficult to independently realize automatic processing, which is prone to misjudgment. Summary of the Invention

[0006] In a first aspect of the embodiments of the present disclosure, a risk identification method is provided. The method includes:

[0007] Obtain the target content to be identified and the business features related to the target content;

[0008] Determine whether the target content belongs to the risk business scope according to the business features;

[0009] If the target content does not belong to the risk business scope, determine whether the target content belongs to the risk content scope according to the content features of the target content;

[0010] If the target content does not belong to the risk content scope, input the target content as a prompt word into a large language model to obtain the recognition result of whether the target content has risks output by the large language model.

[0011] Optionally, the determining whether the target content belongs to the risk business scope according to the business features includes:

[0012] Match the business features with a preset risk feature library to determine whether the business features hit the risk features in the risk feature library; wherein, the risk features include the business features extracted from each risk business;

[0013] If the business feature does not match the risk feature in the risk feature library, it is determined that the target content does not fall within the scope of risk operations.

[0014] Optionally, determining whether the target content belongs to the scope of risk content according to the content feature of the target content includes:

[0015] Input the target content into a preset risk identification model, and calculate whether the content feature of the target content belongs to the scope of risk content.

[0016] Optionally, inputting the target content into a preset risk identification model includes:

[0017] Perform content classification on the target content to determine the content type of the target content;

[0018] Input the target content into the risk identification model corresponding to the content type.

[0019] Optionally, inputting the target content as a prompt into a large language model includes:

[0020] Obtain a basic prompt describing the review rules and input-output structure;

[0021] Fill the target content as input data into the input structure of the basic prompt to obtain a question prompt, and input the question prompt into the large language model.

[0022] Optionally, the method further includes:

[0023] If the target content belongs to the scope of risk operations, or the target content belongs to the scope of risk content, or the large language model outputs an identification result indicating that the target content is at risk, perform risk processing on the target content.

[0024] Optionally, obtaining the target content to be identified includes:

[0025] Based on a preset screening rule, determine whether the business content generated in the business system matches the screening rule;

[0026] If the screening rule is matched, determine the business content as the target content to be identified.

[0027] Optionally, the target content to be identified includes real-time target content generated in online operations; or non-real-time target content collected in offline operations.

[0028] In the second aspect of the embodiments of the present disclosure, a risk identification device is provided, and the device includes:

[0029] An acquisition unit that acquires the target content to be recognized and the business characteristics related to the target content;

[0030] A first-layer recognition unit that determines whether the target content belongs to the scope of risk business according to the business characteristics;

[0031] A second-layer recognition unit that, if the target content does not belong to the scope of risk business, determines whether the target content belongs to the scope of risk content according to the content characteristics of the target content;

[0032] A third-layer recognition unit that, if the target content does not belong to the scope of risk content, inputs the target content as a prompt word into a large language model to obtain the recognition result of whether there is a risk in the target content output by the large language model.

[0033] Optionally, the first-layer recognition unit includes:

[0034] A matching subunit that matches the business characteristics with a preset risk feature library to determine whether the business characteristics hit the risk features in the risk feature library; wherein the risk features include the business characteristics extracted from each risk business;

[0035] A determining subunit that, if the business characteristics do not hit the risk features in the risk feature library, determines that the target content does not belong to the scope of risk business.

[0036] Optionally, the second-layer recognition unit is further configured to input the target content into a preset risk recognition model to calculate whether the content characteristics of the target content belong to the scope of risk content.

[0037] Optionally, the second-layer recognition unit is further configured to classify the content of the target content to determine the content type of the target content; and input the target content into a risk recognition model corresponding to the content type.

[0038] Optionally, the third-layer recognition unit further includes:

[0039] An acquisition subunit that acquires the basic prompt word describing the review rules and the input-output structure;

[0040] A construction subunit that fills the target content as input data into the input structure of the basic prompt word to obtain a question prompt word, and inputs the question prompt word into the large language model.

[0041] Optionally, the device further includes:

[0042] A processing unit, if the target content belongs to the scope of risk services, or the target content belongs to the scope of risk content, or there is a risk recognition result for the target content output by the large language model, performs risk processing on the target content.

[0043] Optionally, the obtaining unit is further configured to determine, based on a preset screening rule, whether the service content generated in the service system hits the screening rule. If the screening rule is hit, the service content is determined as the target content to be recognized.

[0044] Optionally, the target content to be recognized includes real-time target content generated in online services; or non-real-time target content collected in offline services.

[0045] In a third aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, including:

[0046] When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the risk recognition method as described in any one of the preceding items.

[0047] In a fourth aspect of the embodiments of the present disclosure, a computing device is provided, including:

[0048] A processor;

[0049] A memory for storing executable instructions of the processor;

[0050] Wherein, the processor is configured to execute the executable instructions to implement the risk recognition method as described in any one of the preceding items.

[0051] According to the risk recognition solution provided by the embodiments of the present disclosure, through cascaded multi-layer recognition, risk content is sequentially recognized layer by layer according to service characteristics, content characteristics, and large language models, which can avoid the influence of the data long-tail effect and thus improve the recognition accuracy. Description of the Drawings

[0052] By reading the following detailed description with reference to the drawings, the above and other objects, features, and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the drawings, several embodiments of the present disclosure are shown in an exemplary rather than restrictive manner, where:

[0053] Figure 1 Schematically shows a data long-tail schematic diagram provided by the present disclosure;

[0054] Figure 2 Schematically shows a multi-layer risk recognition architecture schematic diagram provided by the present disclosure;

[0055] Figure 3Schematically shows a schematic diagram of the risk identification method provided by the present disclosure;

[0056] Figure 4 Schematically shows a schematic diagram of the development process of the risk identification model and the large language model provided by the present disclosure;

[0057] Figure 5 Schematically shows a schematic diagram of the medium provided by the present disclosure;

[0058] Figure 6 Schematically shows a schematic diagram of the risk identification device provided by the present disclosure;

[0059] Figure 7 Schematically shows a schematic diagram of the computing device provided by the present disclosure.

[0060] In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed implementation manners

[0061] Next, the principles and spirit of the present disclosure will be described with reference to several exemplary implementation manners. It should be understood that these implementation manners are only provided to enable those skilled in the art to better understand and then implement the present disclosure, rather than limiting the scope of the present disclosure in any way. On the contrary, these implementation manners are provided to make the present disclosure more thorough and complete, and to be able to fully convey the scope of the present disclosure to those skilled in the art.

[0062] Those skilled in the art know that the implementation manners of the present disclosure can be implemented as a system, a device, an equipment, a method, or a computer program product. Therefore, the present disclosure can be specifically implemented in the following forms: completely hardware, completely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0063] According to the implementation manners of the present disclosure, a risk identification method, a computer-readable storage medium, a device, and a computing device are proposed.

[0064] In this article, it should be understood that the number of any element in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0065] Next, with reference to several representative implementation manners of the present disclosure, the principles and spirit of the present disclosure will be elaborated in detail.

[0066] The data involved in the present disclosure can be data authorized by users or fully authorized by all parties. The collection, dissemination, use, etc. of the data all comply with the requirements of relevant national laws and regulations. The implementation manners / embodiments of the present disclosure can be combined with each other.

[0067] Overview of Application Scenarios

[0068] As described above, the risk identification method using manual review has problems of low efficiency, high cost, and poor timeliness, while the risk identification method using machine review has insufficient identification accuracy. When using the combined machine review + manual review risk identification method, it mainly relies on manual review, so there are the same problems of low efficiency, high cost, and poor timeliness as those of manual review.

[0069] To address the above problems, the present disclosure does not adopt the manual review method at first, but only improves the machine review method, thereby avoiding the problems of low efficiency, high cost, and poor timeliness existing in the manual review method or the combined machine review + manual review method.

[0070] Furthermore, by analyzing the reasons for the insufficient accuracy caused by the traditional machine review method, it is found that in the overall content, the number of risk contents is often much less than the number of healthy contents. Therefore, when conducting machine review on all contents, even if most healthy contents and risk contents can be correctly classified, it is still very likely to misjudge a large number of normal contents, which is the common data long-tail phenomenon or data long-tail problem in the field of data analysis.

[0071] Please refer to the following Figure 1 schematic diagram of the data long-tail problem under the traditional machine review method.

[0072] As Figure 1 shown, for 500,000 healthy contents, even if 99% of the healthy contents are correctly identified, there are still 5,000 healthy contents misjudged as risk contents. And the number of risk contents is much less than the number of healthy contents. For 1,000 risk contents, if 99% of the risk samples are correctly identified, there are 990 risk contents. Since the identified risk contents (including the 5,000 misjudged healthy contents and the 990 correctly identified risk contents) need to be further processed, from the perspective of processing accuracy, the actual number of misprocessed contents will reach 5,000 (i.e., the 5,000 misjudged healthy contents), and the number of correctly processed contents is only 990 (i.e., the 990 correctly identified risk contents). Overall, the final processing accuracy is only 990 / 5,000 = 19.8%; and this is achieved on the basis that the identification accuracy during machine review reaches 99%.

[0073] Based on this, to solve the problem of low processing accuracy caused by the data long-tail phenomenon in the traditional machine review method, the present disclosure proposes a multi-layer risk identification architecture schematic diagram as Figure 2 shown. The multi-layer risk identification architecture includes a scope layer, a recall layer, an accuracy layer, and a processing layer from top to bottom. The scope layer, recall layer, accuracy layer, and processing layer can be connected in series in the form of a data funnel as Figure 2 shown to achieve high-precision machine review.

[0074] Among them, the scope layer is used to combine business characteristics to identify risk content belonging to the scope of risk business. The role of the scope layer is reflected in strengthening the coverage of risks for each business type and stripping content that appears healthy in terms of business, so as to input this content that appears healthy in terms of business into the recall layer of the next layer for further identification.

[0075] The recall layer is used to combine content characteristics to identify risk content belonging to the scope of risk content. The role of the recall layer is reflected in covering the main business scenarios and violation types and stripping content that appears healthy in terms of content, so as to input this content that appears healthy in terms of content into the precision layer of the next layer for further identification.

[0076] The precision layer is used to leverage the capabilities of the large language model. Since the large language model has stronger logical thinking ability and deep semantic understanding ability, by re-checking and auditing the content hit by the recall layer through the large language model and performing cross-verification with the recall layer, the accuracy of machine review can be improved. The precision layer conducts a detailed review of the stripped content.

[0077] The processing layer is mainly implemented relying on rules. For business value, combining business experience and data analysis results, different differential processing means are formed for different types of users. Optionally, based on business experience and data analysis, the probability of a user publishing risk content and the user value can be obtained to form positive or negative user lists such as white lists, high-value lists, and suspected account theft lists, and different processing strategies can be configured.

[0078] Through the machine review process of the multi-layer risk identification architecture as Figure 2 shown, healthy content can be stripped layer by layer to solve the data long-tail problem in traditional machine review methods, thereby achieving high-precision risk identification in machine review.

[0079] It is worth mentioning that the embodiments provided in this disclosure can be applied to any application scenario that requires risk identification, such as but not limited to social content published by users on social platforms, product content published by merchants on e-commerce platforms, multimedia content (such as videos, audios, etc.) published by users on entertainment platforms, etc. Since the application scenarios are difficult to enumerate, they will not be elaborated here one by one.

[0080] It should be noted that the above application scenarios are only shown for the convenience of understanding the spirit and principle of this disclosure, and the embodiments of this disclosure are not restricted in this regard. On the contrary, the embodiments of this disclosure can be applied to any applicable scenario.

[0081] Exemplary Method

[0082] Next, please refer to Figure 3 the flowchart of the risk identification method according to the exemplary embodiment of this disclosure shown. As Figure 3As shown, the risk identification method can be applied to the aforementioned multi-layer risk identification architecture and may include the following steps:

[0083] Step 310: Obtain the target content to be identified and the business characteristics related to the target content.

[0084] Exemplarily, based on a preset screening rule, it can be determined whether the business content generated in the business system hits the screening rule;

[0085] If it hits the screening rule, the business content is determined as the target content to be identified.

[0086] In this example, in the process of obtaining the target content, the target content to be identified can be screened out from various business contents based on a preset screening rule. For example, screening is performed based on the characteristic information related to the target content (such as business characteristics, content characteristics, etc.), and the business contents with characteristic information such as business characteristics and content characteristics are screened out and used as the target content to be identified; while the business contents without characteristic information such as business characteristics and content characteristics can be regarded as invalid contents and filtered out. In addition, for duplicate target contents, the most recent one can be selected to reduce the processing volume of risk identification.

[0087] The embodiments of the present disclosure have high applicability, that is, they can be used for online services and offline services; for online services, the target content to be identified can include real-time target content generated in online services; while for offline services, the target content to be identified can include non-real-time target content collected in offline services.

[0088] Exemplarily, in offline services, the accumulated business content can be periodically obtained and the target content can be screened out from it. For example, the target content can be obtained once every hour. In online services, the business content can be collected in real time and the target content can be screened.

[0089] Step 320: Determine whether the target content belongs to the scope of risk services according to the business characteristics.

[0090] After obtaining the business characteristics of the target content, it can be determined whether the target content belongs to the scope of risk services according to the business characteristics.

[0091] The following combines the aforementioned Figure 2 As shown in the multi-layer risk identification architecture, step 320 can be executed in the scope layer. In Figure 2 it, the scope layer can deploy a variety of different screening strategies, such as including but not limited to screening strategies for business characteristics, screening strategies for users, etc.

[0092] [Screening strategy for business characteristics]

[0093] In the scope layer, it is necessary to predetermine the risk business scope required for screening. For example, the risk characteristics of various risk businesses in the industry can be collected and these risk characteristics can be used to build a preset risk characteristic library. In this way, the above determination of whether the target content belongs to the risk business scope based on the business characteristics can further include:

[0094] Matching the business feature with a preset risk feature library to determine whether the business feature matches the risk feature in the risk feature library; wherein the risk feature includes business features extracted from various risky businesses;

[0095] If the business feature does not match the risk feature in the risk feature library, it is determined that the target content does not belong to the risk business scope.

[0096] Through the above-mentioned embodiments, the risk feature library can be used to cover various business type risks and strip off the content that appears healthy in terms of business, so as to input these content that appears healthy in terms of business into the recall layer of the next layer for further identification.

[0097] [Filtering strategy for users]

[0098] In the scope layer, the screening strategy for screening risky users can be predetermined. For example, a list of historical illegal users can be used. If the user who publishes the target content is on the list of historical illegal users, the target content is determined to be risky content. Specifically, a list of historical illegal users can be constructed by identifying illegal users such as user feature analysis, text clustering, user behavior sequence analysis, junk registration identification model, and gang mining.

[0099] Taking user feature analysis as an example, the features of illegal users are determined based on historical data. For example, a user who has been processed M times in the past N days can be considered an illegal user, so the user features of the illegal user are extracted and added to the list of historical illegal users. The content posted by the illegal user can be directly blocked.

[0100] It should be noted that when there are no good empirical features or when more features are sought to achieve better recognition effects, business features can be widely used and constructed, and machine learning models such as gradient boosted decision trees (GBDT) and improved GBDT such as LightGBM (Light Gradient Boosting Machine) can be trained to classify user features.

[0101] In actual applications, the scope layer can use a single screening strategy or a combination of multiple screening strategies to identify the target content. If the target content does not fall into the risk range defined by the screening strategy (such as not belonging to the risky business scope, not belonging to the list of historical illegal users, etc.), it can be further handed over to the downstream recall layer for processing.

[0102] In some embodiments, when there are sufficient computing resources and audit accuracy at the downstream level (such as the recall level and the precision level) and risk identification is performed on the entire content, step 320 can be skipped and the entire content can be directly processed at the downstream level.

[0103] Step 330: If the target content does not belong to the risky business scope, determine whether the target content belongs to the risky content scope according to the content characteristics of the target content;

[0104] For target content that does not fall within the risk business scope, it can be further determined whether the target content falls within the risk content scope based on its content characteristics.

[0105] In combination with the above Figure 2 In the multi-layer risk identification architecture shown, step 330 can be performed in the recall layer. Figure 2 In the recall layer, a risk identification model may be deployed. Thus, the above-mentioned determination of whether the target content belongs to the risk content range according to the content characteristics of the target content may further include:

[0106] The target content is input into a preset risk identification model to calculate whether the content features of the target content belong to the risk content range.

[0107] Through the above embodiment, with the help of the risk identification model, the content characteristics of the target content are calculated to determine whether the target content belongs to the risk content range; the target content that belongs to the risk content range is processed, and the target content that does not belong to the risk content range is further identified by the next layer of precision layer.

[0108] In this embodiment, different risk identification models can be developed for target contents of different content types. When identifying a specific content type, the corresponding risk identification model can be used for identification.

[0109] Based on this, the above-mentioned inputting the target content into the preset risk identification model may further include:

[0110] Classifying the target content to determine the content type of the target content;

[0111] The target content is input into a risk identification model corresponding to the content type.

[0112] In this embodiment, the risk identification model can cover text classification models, image classification models, audio-video classification models, multimodal classification models, etc. according to the content type of the target content.

[0113] Taking text content as an example, supervised text classification models such as Bidirectional Encoder Representations from Transformers (BERT), ERNIE 1.0, ERNIE 2.0, or ERNIE 3.0 of the Enhanced Representation through Knowledge Integration (ERNIE) can be trained for risk identification. Taking image content as an example, a supervised image classification model can be trained for risk identification, or the OCR (Optical Character Recognition) capability can be used to first extract the text in the image and then input the text into the text classification model for risk identification.

[0114] By developing risk identification models for different content types, the recall layer can cover the content risk identification of various business scenarios. In the development of the recall layer, the recall ability of the model can be strengthened to avoid missing the release of risk content. Through the risk identification model, the target content that appears healthy in content can be stripped and input into the next precise layer for further identification.

[0115] Step 340: If the target content does not fall within the scope of risk content, input the target content as a prompt into the large language model to obtain the identification result of whether the target content has risks output by the large language model.

[0116] For the target content that does not fall within the scope of the risk content, further input the target content into the large language model, and rely on the large language model to determine whether the target content has risks.

[0117] Combined with the foregoing Figure 2 shown multi-layer risk identification architecture, step 340 can be executed in the precise layer. The large language model can be locally deployed or connected to a third-party large language model in the precise layer.

[0118] Before using the large language model, it is necessary to develop the machine review ability of the large language model. Specifically, based on the type of violation, develop basic prompt descriptions of the review rules and input-output structures, and require the large language model to perform machine review classification on the input target content. Thus, the above inputting the target content as a prompt into the large language model can further include:

[0119] Get basic prompt words that describe audit rules and input and output structures;

[0120] The target content is filled into the input structure of the basic prompt word as input data to obtain the question prompt word, and the question prompt word is input into the large language model.

[0121] By developing the basic prompt words of the large language model, the large language model is trained to identify whether the target content contains risky content, so that the ability of the large language model can be used. Since the large language model has stronger logical thinking ability and deep semantic understanding ability, the content hit by the recall layer can be verified and reviewed again by the large language model, and cross-validated with the recall layer to improve the accuracy of the machine review. The precision layer conducts a precise review of the stripped content.

[0122] In practical applications, when developing prompts to describe review rules and input-output structures and requiring a large language model to perform machine review and classification on the input target content, the review rules and input-output structures can be adjusted by measuring the review recall and precision of the large language model under the current prompt to achieve better recall precision. This process can be called prompt engineering.

[0123] Since prompt engineering often makes it difficult for large language models to achieve high audit performance, and large language models have hallucinations and are prone to cause more accidental injuries, the large language model can be fine-tuned to calibrate the audit standards and improve the audit accuracy of the large language model.

[0124] Please refer to the following Figure 4 Schematic diagram of the risk identification model and large language model development and training process shown.

[0125] In the process of developing and training risk identification models and large language models, operators, labelers, algorithm developers, engineers and other roles can participate together. First, the relevant business parties and operators reach a consensus on the risk definition and related machine review standards for the risk scenarios to be managed. It should be noted that the machine review standards may differ slightly from the human review standards, but they are generally consistent. The machine review standards mainly focus on the content types of model reviews and provide unified review classification standards.

[0126] After that, the algorithm will determine the demand level of each type of risk content and specify the labeling plan. In the early stage of labeling, the content of the market can be sampled to determine the level of the risk problem being managed on the market; the same type of risk content that has been intercepted historically on the platform can be sampled to determine the interception level and interception accuracy of historical capabilities. During the labeling process, the labeled content should be continuously quality-checked and confirmed to ensure that the labeling standards are consistent with the machine review labeling and risk definition.

[0127] Using the above-mentioned annotated content, train the first version of the risk identification model (hereinafter referred to as the model) in a supervised learning manner. After completing the model training, release the model to a small number of users for trial operation, continuously annotate the content in the trial operation to count the business metrics of the model online. If the metrics meet the expectations, proceed to the large language model development stage. If the model does not meet the expectations, optimize and iterate the model for the error samples in the content annotated during the trial operation.

[0128] In the development stage of the large language model's machine review ability, the open-source large language model can be deployed in a synchronous local deployment manner. Exemplarily, Qwen2 can be used as the large language model base. After completing the deployment of the large language model, design prompts to require the large language model to conduct reviews as the machine review demo of the initial version of the large language model. Thereafter, similar to the development mode of the risk identification model in the previous stage, release the large language model to a small number of users for trial operation, and conduct a secondary machine review on the target content input to the risk identification model. Further, annotate the results of the large language model's machine review during the trial operation to count the performance metrics of the large language model's machine review. If the metrics do not meet the expectations, optimize and iterate the large language model's machine review ability through methods such as fine-tuning the large language model.

[0129] Exemplarily, Llama Factory can be used to fine-tune the large language model. The fine-tuning is based on the annotated content of the classification task. By combining the task description, samples, and labels of the prompts, the question-and-answer pairs of the large language model are constructed, and the large language model is fine-tuned through the question-and-answer pairs. Among them, the lora (Low-Rank Adaptation) method can be adopted. LoRA can greatly improve the fine-tuning speed and reduce the demand for computing resources through low-rank decomposition and a small number of adaptive parameter updates.

[0130] After fine-tuning the large language model to an ideal effect and after sufficient verification during the trial operation, it is confirmed by the operator that the metrics during the trial operation meet the online standards, and the large language model is officially launched.

[0131] In an exemplary embodiment, the method further includes:

[0132] If the target content belongs to the scope of risk services, or the target content belongs to the scope of risk content, or the risk identification result of the target content output by the large language model is risky, perform risk handling on the target content.

[0133] Combined with the above-mentioned Figure 2 As shown in the multi-layer risk identification architecture, the risk handling of the target content can be executed by the processing layer, and the processing layer mainly relies on risk handling methods to achieve. Different risk handling methods can be developed according to business value, business experience, data analysis, different users, etc., so as to build diversified and hierarchical risk handling capabilities.

[0134] For example, based on business experience and data analysis, the probability of a user posting risky content and the user value are obtained, and positive or negative user lists such as a whitelist, a high-value list, and a suspected account hijacking list are formed, and different processing methods are configured accordingly.

[0135] When performing risk processing, a processing message can be sent to send the risky content or information related to the risky content (such as the content ID) to the general resource processing interface, and the business status of the risky content is modified to implement content processing.

[0136] As Figure 4 shown, after the large language model is officially launched, an online automatic processing mode based on the Nydus processing message model can be adopted. For the recognition result that the target content output by the large language model has risks, it is sent to the resource processing interface through Nydus messages, and the target content with risks is deleted, made only visible to oneself, or not recommended to other users, etc.

[0137] Exemplary Medium

[0138] After introducing the methods of the exemplary embodiments of the present disclosure, next, reference Figure 5 is made to illustrate the media of the exemplary embodiments of the present disclosure.

[0139] In this exemplary embodiment, the above method can be implemented by a program product. For example, a portable compact disc read-only memory (CD-ROM) can be used and includes program code, and can run on a device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device.

[0140] The program product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0141] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than a readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0142] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0143] The program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the C language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).

[0144] In summary, the present disclosure may provide a computer-readable storage medium, which when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, may enable the electronic device to execute the foregoing embodiments of the risk identification method.

[0145] Exemplary Device

[0146] After introducing the medium of the exemplary embodiments of the present disclosure, next, reference will be made to Figure 6 to describe the apparatus of the exemplary embodiments of the present disclosure.

[0147] Figure 6 A block diagram of a risk identification apparatus according to an embodiment of the present disclosure is schematically shown, corresponding to the foregoing Figure 2 shown method embodiments. The risk identification apparatus may include:

[0148] An acquisition unit 610, which acquires the target content to be identified and the service features related to the target content;

[0149] A first-layer identification unit 620, which determines whether the target content belongs to the risk service scope according to the service features;

[0150] The second - layer recognition unit 630, if the target content does not belong to the risk business scope, determines whether the target content belongs to the risk content scope according to the content characteristics of the target content.

[0151] The third - layer recognition unit 640, if the target content does not belong to the risk content scope, inputs the target content as a prompt word into the large - language model to obtain the recognition result of whether the target content has risks output by the large - language model.

[0152] Optionally, the first - layer recognition unit 620 includes:

[0153] The matching subunit 621 matches the business characteristics with a preset risk feature library to determine whether the business characteristics hit the risk features in the risk feature library; wherein, the risk features include business characteristics extracted from various risk businesses.

[0154] The determination subunit 623, if the business characteristics do not hit the risk features in the risk feature library, determines that the target content does not belong to the risk business scope.

[0155] Optionally, the second - layer recognition unit 630 is further configured to input the target content into a preset risk recognition model to calculate whether the content characteristics of the target content belong to the risk content scope.

[0156] Optionally, the second - layer recognition unit 640 is further configured to classify the content of the target content to determine the content type of the target content; and input the target content into a risk recognition model corresponding to the content type.

[0157] Optionally, the third - layer recognition unit 640 further includes:

[0158] The acquisition subunit 641 acquires a basic prompt word describing the review rules and the input - output structure.

[0159] The construction subunit 643 fills the target content as input data into the input structure of the basic prompt word to obtain a question prompt word, and inputs the question prompt word into the large - language model.

[0160] Optionally, the device further includes:

[0161] The processing unit 650, if the target content belongs to the risk business scope, or the target content belongs to the risk content scope, or the recognition result of the target content having risks output by the large - language model, performs risk processing on the target content.

[0162] Optionally, the obtaining unit 610 is further configured to determine whether the service content generated in the service system hits the preset screening rule based on the preset screening rule. If the service content hits the screening rule, the service content is determined as the target content to be recognized.

[0163] Optionally, the target content to be recognized includes real-time target content generated in online services; or non-real-time target content collected in offline services.

[0164] Exemplary Computing Device

[0165] After introducing the methods, media, and devices of the exemplary embodiments of the present disclosure, next, reference Figure 7 is made to illustrate the computing device of the exemplary embodiments of the present disclosure.

[0166] Figure 7 The displayed computing device 1500 is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0167] As Figure 7 shown, the computing device 1500 is presented in the form of a general-purpose computing device. The components of the computing device 1500 may include, but are not limited to: at least one processing unit 1501, at least one storage unit 1502, and a bus 1503 connecting different system components (including the processing unit 1501 and the storage unit 1502).

[0168] The bus 1503 includes a data bus, a control bus, and an address bus.

[0169] The storage unit 1502 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 15021 and / or a cache memory 15022, and may further include a readable medium in the form of a non-volatile memory, such as a read-only memory (ROM) 15023.

[0170] The storage unit 1502 may further include a program / utilities 15025 having a set (at least one) of program modules 15024. Such program modules 15024 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. The implementation of a network environment may be included in each or some combination of these examples.

[0171] The computing device 1500 may also communicate with one or more external devices 1504 (such as a keyboard, a pointing device, etc.).

[0172] This communication can be carried out through the input / output (I / O) interface 1505. Also, the computing device 1500 can further communicate with one or more networks (such as local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) through the network adapter 1506. As Figure 7 shown, the network adapter 1506 communicates with other modules of the computing device 1500 through the bus 1503. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the computing device 1500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0173] Through the computing device 1500 as Figure 7 shown, the foregoing risk identification method can be implemented. More specifically, the storage unit 1502 stores instructions executable by the processing unit 1501, and when the processing unit 1501 executes the instructions, the foregoing risk identification method is implemented.

[0174] It should be noted that although several units / modules or sub-units / modules of the risk identification device are mentioned in the foregoing detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more of the above-described units / modules can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0175] In addition, although the operations of the method of the present disclosure are described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the shown operations must be performed to achieve the desired result. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution.

[0176] Although the spirit and principles of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of each aspect does not mean that the features in these aspects cannot be combined for benefit. This division is only for the convenience of expression. The present disclosure aims to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A risk identification method, comprising: Obtaining a target content to be identified and business characteristics related to the target content; Determining whether the target content belongs to the scope of risk business according to the business characteristics; If the target content does not belong to the scope of risk business, determining whether the target content belongs to the scope of risk content according to the content characteristics of the target content; If the target content does not belong to the scope of risk content, inputting the target content as a prompt word into a large language model to obtain an identification result of whether there is a risk in the target content output by the large language model.

2. The method according to claim 1, wherein the determining whether the target content belongs to the scope of risk business according to the business characteristics comprises: Matching the business characteristics with a preset risk feature library to determine whether the business characteristics hit the risk features in the risk feature library; wherein the risk features include business characteristics extracted from various risk businesses; If the business characteristics do not hit the risk features in the risk feature library, determining that the target content does not belong to the scope of risk business.

3. The method according to claim 1, wherein the determining whether the target content belongs to the scope of risk content according to the content characteristics of the target content comprises: Inputting the target content into a preset risk identification model and calculating whether the content characteristics of the target content belong to the scope of risk content.

4. The method according to claim 3, wherein the inputting the target content into a preset risk identification model comprises: Performing content classification on the target content to determine the content type of the target content; Inputting the target content into a risk identification model corresponding to the content type.

5. The method according to claim 1, wherein the inputting the target content as a prompt word into a large language model comprises: Obtaining a basic prompt word describing the review rules and the input-output structure; Filling the target content as input data into the input structure of the basic prompt word to obtain a question prompt word, and inputting the question prompt word into the large language model.

6. The method according to claim 1, further comprising: If the target content belongs to the scope of risk business, or the target content belongs to the scope of risk content, or the identification result of the target content output by the large language model indicates that there is a risk, performing risk processing on the target content.

7. The method according to claim 1, wherein the obtaining the target content to be identified comprises: Based on a preset screening rule, determining whether the business content generated in the business system hits the screening rule; If it hits the screening rule, determining the business content as the target content to be identified.

8. A risk identification device, the device comprising: An obtaining unit, which obtains a target content to be identified and business characteristics related to the target content; A first-layer identification unit, which determines whether the target content belongs to the scope of risk business according to the business characteristics; The second-level recognition unit, if the target content does not belong to the scope of risk services, determines whether the target content belongs to the scope of risk content according to the content characteristics of the target content; The third-level recognition unit, if the target content does not belong to the scope of risk content, inputs the target content as a prompt word into the large language model to obtain the recognition result of whether there is a risk in the target content output by the large language model.

9. A computer-readable storage medium, comprising: When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device can execute the risk recognition method according to any one of claims 1-7.

10. A computing device, comprising: A processor; A memory for storing the executable instructions of the processor; Wherein, the processor is configured to execute the executable instructions to implement the risk recognition method according to any one of claims 1-7.

Images