A risk detection method, apparatus and device

Abstract

Embodiments of the present specification provide a risk detection method, device and equipment, wherein the method comprises: obtaining sample data corresponding to a target service and a risk label corresponding to the sample data, using a preset large language model to perform risk detection on the sample data to obtain a first risk detection result and a probability value of the first risk detection result belonging to each preset confidence interval, determining a first loss value based on the first risk detection result and the risk label, determining a second loss value based on a matching degree between the first risk detection result and the risk label and the probability value of the first risk detection result belonging to each preset confidence interval, training the preset large language model based on the first loss value and the second loss value to obtain a trained large language model.

Description

Technical Field

[0001] This document relates to the field of computer technology, and in particular to a risk detection method, apparatus, and equipment. Background Technology

[0002] With the continuous development of internet technology and e-commerce, in order to protect user privacy and data security, models based on artificial intelligence algorithms, such as large language models, can be used to control risks in the business execution process.

[0003] Because AI algorithms may overconfident models, meaning they may assign excessively high confidence levels to their own generated detection results, supervised learning requires manual labeling of confidence levels. However, as the number and types of training samples grow larger, the efficiency and accuracy of manual labeling become low, affecting the training effect of the model. Therefore, a faster and more reliable model training solution is needed to improve the efficiency and accuracy of business risk detection. Summary of the Invention

[0004] The purpose of the embodiments in this specification is to provide a faster and more reliable model training scheme to improve the efficiency and accuracy of business risk detection.

[0005] To achieve the above technical solution, the embodiments in this specification are implemented as follows: This specification provides a risk detection method, comprising: acquiring sample data corresponding to a target business and risk labels corresponding to the sample data; performing risk detection on the sample data using a preset large language model to obtain a first risk detection result and a probability value of the first risk detection result belonging to each preset confidence interval; determining a first loss value based on the first risk detection result and the risk labels, and determining a second loss value based on the matching degree between the first risk detection result and the risk labels, and the probability value of the first risk detection result belonging to each preset confidence interval; training the preset large language model based on the first loss value and the second loss value to obtain a trained large language model, wherein the trained large language model is used to determine the risk detection result and confidence level corresponding to the execution of the target business, and the risk detection result and confidence level are used to determine whether there is a risk in executing the target business.

[0006] This specification provides a risk detection device, comprising: a data acquisition module for acquiring sample data corresponding to a target business and risk labels corresponding to the sample data; a first detection module for performing risk detection on the sample data using a preset large language model to obtain a first risk detection result and a probability value of the first risk detection result belonging to each preset confidence interval; a loss determination module for determining a first loss value based on the first risk detection result and the risk labels, and determining a second loss value based on the matching degree between the first risk detection result and the risk labels, and the probability value of the first risk detection result belonging to each preset confidence interval; and a model training module for training the preset large language model based on the first loss value and the second loss value to obtain a trained large language model, wherein the trained large language model is used to determine the risk detection result and confidence level corresponding to the execution of the target business, and the risk detection result and confidence level are used to determine whether there is a risk in executing the target business.

[0007] This specification provides an embodiment of a risk detection device, comprising: a processor; and a memory arranged to store computer-executable instructions, wherein when executed, the executable instructions cause the processor to: acquire sample data corresponding to a target service and risk labels corresponding to the sample data; perform risk detection on the sample data using a preset large language model to obtain a first risk detection result and a probability value of the first risk detection result belonging to each preset confidence interval; determine a first loss value based on the first risk detection result and the risk labels, and determine a second loss value based on the matching degree between the first risk detection result and the risk labels and the probability value of the first risk detection result belonging to each preset confidence interval; and train the preset large language model based on the first loss value and the second loss value to obtain a trained large language model, wherein the trained large language model is used to determine the risk detection result and confidence level corresponding to the execution of the target service, and the risk detection result and confidence level are used to determine whether there is a risk in executing the target service.

[0008] This specification also provides a storage medium for storing computer-executable instructions. When executed by a processor, the executable instructions implement the following process: acquiring sample data corresponding to a target service and risk labels corresponding to the sample data; performing risk detection on the sample data using a preset large language model to obtain a first risk detection result and a probability value of the first risk detection result belonging to each preset confidence interval; determining a first loss value based on the first risk detection result and the risk labels, and determining a second loss value based on the matching degree between the first risk detection result and the risk labels, and the probability value of the first risk detection result belonging to each preset confidence interval; training the preset large language model based on the first loss value and the second loss value to obtain a trained large language model, wherein the trained large language model is used to determine the risk detection result and confidence level corresponding to the execution of the target service, and the risk detection result and confidence level are used to determine whether there is a risk in executing the target service.

[0009] This specification also provides a computer program product, including a computer program that, when executed by a processor, implements the following process: acquiring sample data corresponding to a target business and risk labels corresponding to the sample data; using a preset large language model to perform risk detection on the sample data, obtaining a first risk detection result and a probability value of the first risk detection result belonging to each preset confidence interval; determining a first loss value based on the first risk detection result and the risk labels, and determining a second loss value based on the matching degree between the first risk detection result and the risk labels, and the probability value of the first risk detection result belonging to each preset confidence interval; training the preset large language model based on the first loss value and the second loss value to obtain a trained large language model, wherein the trained large language model is used to determine the risk detection result and confidence level corresponding to the execution of the target business, and the risk detection result and confidence level are used to determine whether there is a risk in executing the target business. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is a schematic diagram of the implementation environment for one of the risk detection methods described in this specification; Figure 2This is a flowchart illustrating the processing procedure of one risk detection method described in this specification. Figure 3 This is a schematic diagram of the input and output data of a large language model as described in this specification; Figure 4 This is a schematic diagram of a confidence probability distribution in this specification; Figure 5 This is a flowchart illustrating the process for determining a second loss value as described in this specification. Figure 6 This is a schematic diagram of a supervised fine-tuning process described in this specification; Figure 7 This is a flowchart illustrating the process for determining a first detection result as described in this specification. Figure 8 This is a flowchart illustrating the risk detection process for one target business described in this specification. Figure 9 This is a flowchart illustrating the process of constructing a second prompt message as described in this specification. Figure 10 This is a flowchart illustrating the process of constructing another type of second prompt information in this specification; Figure 11 This is a flowchart illustrating a reinforcement learning training process as described in this specification. Figure 12 This is a schematic diagram of a risk detection device described in this specification; Figure 13 This is a schematic diagram of a risk detection device described in this manual. Detailed Implementation

[0011] This specification provides a risk detection method, apparatus, and device through its embodiments.

[0012] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0013] This specification provides a faster and more reliable model training scheme to improve the efficiency and accuracy of business risk detection. In the context of rapid development in fintech, artificial intelligence, especially large language models, is increasingly widely used in risk control. However, models built with AI algorithms suffer from a key problem: overconfidence. This means the model may assign excessively high confidence levels to its own generated answers, including potentially erroneous judgments. This problem is particularly prominent in compliance-critical scenarios, potentially leading to serious misjudgment risks. In practical applications, supervised training can be performed by manually labeling confidence levels. However, high-quality confidence labeling requires domain experts, is costly, and consistency is difficult to guarantee, hindering scalability. Furthermore, due to the dynamic changes in data distribution in financial risk control scenarios, models trained on historical data struggle to adapt to new risk patterns, exhibiting insufficient generalization ability. Alternatively, one could ask the model to output confidence levels using pre-designed prompts. However, this essentially only makes the model "say" the confidence level rather than truly calibrating its inherent confidence mechanism. Different prompts can lead to significant differences in results, resulting in unstable calibration and failing to address the model's inherent overconfidence problem. This poses a high risk in scenarios with high reliability requirements, such as financial risk control. Another approach is to use ensemble learning and multiple sampling to estimate confidence levels by statistically analyzing the consistency of multiple generated results. However, this approach faces significant practical challenges. For example, it requires 5-10 times the inference computation cost, making the response speed in real-time risk control scenarios unacceptable to business needs. Furthermore, random noise introduced during sampling affects stability, and this approach is only suitable for specific task types such as factual question answering, with limited effectiveness in complex financial risk inference scenarios. To address this, this specification provides a faster and more reliable model training scheme. In this scheme, a second loss value, determined based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval, allows the model to learn the probability distribution of discrete preset confidence regions during the model training phase. This eliminates the need for manual labeling of confidence labels, enabling the training of a large language model using sample data and corresponding risk labels. This saves on manual labeling costs, improves model training efficiency while ensuring accuracy, and ultimately enhances the effectiveness of business risk control using the trained large language model. Specific details can be found in the following embodiments.

[0014] The risk detection methods described in one or more embodiments of this specification are applicable to the data acquisition implementation environment, such as... Figure 1 As shown, the implementation environment includes at least: Client 100 and server 200. Furthermore, server 200 can be configured with various network models and algorithms, among which: Client 100 can run on terminal devices, which can be mobile phones, personal computers, tablets, e-book readers, wearable devices, devices that interact with information based on AR (Augmented Reality) and VR (Virtual Reality), and laptop computers, etc. Client 100 can be installed on terminal devices, and Client 100 can be an application, a browser, or a subroutine embedded in an application, etc.

[0015] Server 200 can run on a server, which can be one or more servers, a server cluster consisting of several servers, or a cloud server on a cloud computing platform. Server 200 can be installed on the server. Server 200 can be an application or a subroutine embedded in an application. Various network models and algorithms can be integrated into server 200, or server 200 can call one or more of various network models and algorithms to perform corresponding operations.

[0016] In addition, it may include a database 300, which may be set in the server on which the server 200 runs, or outside the server on which the server 200 runs. The database 300 may store sample data and corresponding risk labels, model parameters of the large language model, and a set of confidence granularities containing multiple granularities.

[0017] In this implementation environment, server 200 can obtain sample data corresponding to the target business and risk labels corresponding to the sample data. Server 200 can use a preset large language model to perform risk detection on the sample data, obtain a first risk detection result and the probability value of the first risk detection result belonging to each preset confidence interval. Based on the first risk detection result and risk labels, a first loss value is determined. Based on the matching degree between the first risk detection result and risk labels, and the probability value of the first risk detection result belonging to each preset confidence interval, a second loss value is determined. Server 200 can train the preset large language model based on the first loss value and the second loss value to obtain a trained large language model. Users can trigger the execution of the target business through client 100. Server 200 can use the trained large language model to determine the risk detection result and confidence level corresponding to the execution of the target business, and judge whether there is a risk in executing the target business based on the risk detection result and confidence level, and return the judgment result to client 100.

[0018] like Figure 2As shown in the embodiments of this specification, a risk detection method is provided. The execution subject of this method can be a server, which can be a single independent server or a server cluster composed of multiple servers. The server can be a backend server for resource-related businesses such as resource transfer or resource lending, or a backend server for a resource-related business application. The method specifically includes the following steps: In step S202, sample data corresponding to the target business and risk labels corresponding to the sample data are obtained.

[0019] The target business can be any business involving resources, such as resource transfer, resource lending, or resource management. The sample data can be the business data required to execute the target business. For example, if the target business is a resource transfer business, the sample data corresponding to the target business can include data such as the quantity of resources transferred, the time of resource transfer, the object of resource transfer, and the resource transfer platform. If the target business is a resource lending business, the sample data corresponding to the target business can include the quantity of resources lent, collateral information, and user credit information. The risk labels corresponding to the sample data can include type labels used to characterize the degree of risk (such as whether a risk type exists or not).

[0020] In implementation, the server can receive a fine-tuning instruction triggered by the user for a preset large language model, or, when a preset model fine-tuning period is reached, the server can trigger a fine-tuning instruction for the preset large language model, or, when it is detected that the amount of historical business data of the target business exceeds a preset data amount threshold within a preset period, the server can trigger a fine-tuning instruction for the preset large language model. There are also various ways to trigger the fine-tuning instruction, and different triggering methods can be selected according to different actual application scenarios. This specification does not specifically limit this in the embodiments.

[0021] In response to this fine-tuning instruction, the server can identify historical business data corresponding to the target business as sample data, and determine the risk label corresponding to the sample data based on the execution status of the historical business data. Alternatively, when the amount of historical business data is large, to improve model training efficiency, data quality assessment can be performed on the historical business data, and the historical business data can be filtered based on the data quality assessment results to identify the filtered historical business data as sample data.

[0022] Alternatively, the server can also use a pre-trained sample generation model to generate sample data corresponding to the target business and the risk label corresponding to the sample data, based on the model training requirements corresponding to the target business. The sample generation model can be a model built based on a preset generation algorithm.

[0023] Furthermore, the method for obtaining the risk label corresponding to the sample data of the aforementioned target business is an optional and feasible method. In actual application scenarios, there can be a variety of different methods. Different methods can be selected according to different actual application scenarios. This specification does not specifically limit this embodiment.

[0024] In step S204, a preset large language model is used to perform risk detection on the sample data to obtain a first risk detection result and the probability value of the first risk detection result belonging to each preset confidence interval.

[0025] Among them, the pre-defined large language model can be a large language model with complex logical reasoning capabilities, capable of multi-step reasoning, logical analysis and problem solving. Compared with ordinary language models, it performs better in advanced cognitive tasks such as mathematical reasoning and causal inference.

[0026] The preset confidence interval can be multiple confidence intervals obtained by dividing based on a preset granularity. For example, the preset confidence interval can include preset confidence interval 1 (i.e., [0~20], which is used to represent the interval with a confidence level of not less than 0 and less than 20), preset confidence interval 2 (i.e., [20~40]), preset confidence interval 3 (i.e., [4~60]), preset confidence interval 4 (i.e., [60~80]), and preset confidence interval 5 (i.e., [80~100]).

[0027] In implementation, the server can construct a preset prompt message based on the business background of the target business, the standard operating procedure (SOP) for review, and the format requirements. The preset prompt message can be used to control the output of the large language model, which includes the judgment key points, judgment analysis, judgment conclusion, and confidence level. The confidence level can be determined based on the probability value of the detection result belonging to each preset confidence level interval.

[0028] For example, such as Figure 3 As shown, the server can input preset prompt information into the preset large language model, and the preset large language model can output the first risk detection result, which includes the judgment key points, judgment analysis, judgment conclusion, and confidence level.

[0029] In addition, the server can also obtain the confidence token probability distribution, that is, the probability value of the first risk detection result belonging to each preset confidence interval. For example, assuming the preset confidence interval is 10 discrete intervals obtained by dividing the confidence interval [0%, 100%] by 10 granularities, and the sum of the probability values ​​of the first risk detection result belonging to each preset confidence interval can be 1, the probability value of the first risk detection result belonging to each preset confidence interval can be as follows: Figure 4 As shown, the server can determine the confidence level corresponding to the first detection result based on the probability distribution of the confidence level token.

[0030] In step S206, a first loss value is determined based on the first risk detection result and the risk label, and a second loss value is determined based on the matching degree between the first risk detection result and the risk label, as well as the probability value of the first risk detection result belonging to each preset confidence interval.

[0031] In implementation, the server can determine the first loss value based on the matching between the first risk detection result and the risk label. For example, if the first risk detection result and the risk label match, that is, both indicate that there is no risk or that there is risk, then the first loss value can be 1, meaning that the answer to the first risk detection result is correct. Alternatively, if the first risk detection result and the risk label do not match (e.g., the first risk detection result indicates that there is no risk, while the risk label indicates that there is risk), then the first loss value can be 0, meaning that the answer to the first risk detection result is incorrect.

[0032] The server can determine the second loss value based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval. For example, the server can filter out the target confidence interval based on the probability value of the first risk detection result belonging to each preset confidence interval, and determine the second loss value based on the preset correspondence between the probability value, the matching degree, and the loss value, the probability value corresponding to the target confidence interval, the matching degree between the first risk detection result and the risk label, and the above preset correspondence.

[0033] Furthermore, the method for determining the second loss value described above is an optional and feasible method. In practical application scenarios, there can be a variety of different methods. Different methods can be selected according to different practical application scenarios. This specification does not specifically limit this method in the embodiments.

[0034] In this way, the confidence level of the model output can be evaluated using the second loss value. Confidence evaluation is a process of systematically quantifying and evaluating the confidence level of the model output. By establishing an evaluation index system, the accuracy and reliability of the confidence level expression can be measured, providing a basis for model optimization.

[0035] In step S208, the preset large language model is trained based on the first loss value and the second loss value to obtain the trained large language model.

[0036] Among them, the trained large language model can be used to determine the risk detection results and confidence level corresponding to the execution of the target business. The risk detection results and confidence level can be used to determine whether there is a risk in executing the target business.

[0037] In implementation, the server can determine the target loss value based on the first loss value and the second loss value, and determine whether the preset large language model has converged based on the target loss value. If it is determined that the preset large language model has not converged, the server can continue to train the preset large language model based on the sample data and the corresponding risk labels until the preset large language model converges, thus obtaining the trained large language model.

[0038] By calibrating the confidence level during model training, the model can more accurately reflect the true probability of correctness, reducing the problems of overconfidence or underconfidence.

[0039] After obtaining the trained large language model, when the server receives a risk detection request for the execution of the target business, in response to the risk detection request, the server can obtain the business data required for the execution of the target business, and use the trained large language model to perform risk detection on the business data, obtain a second detection result and the corresponding confidence level, and then determine whether there is a risk in the execution of the target business based on the second detection result and the corresponding confidence level.

[0040] Risk detection for the target business can include a complete set of compliance processes and technical systems that, in accordance with pre-set regulatory requirements, prevent, detect and block risky behaviors such as using the financial system to conceal or hide criminal proceeds through means such as customer identification, transaction monitoring, suspicious reporting, and record keeping.

[0041] This specification provides a risk detection method that can acquire sample data corresponding to a target business and risk labels corresponding to the sample data. Using a preset large language model, risk detection is performed on the sample data to obtain a first risk detection result and the probability value of the first risk detection result belonging to each preset confidence interval. Based on the first risk detection result and the risk labels, a first loss value is determined. Based on the matching degree between the first risk detection result and the risk labels, and the probability value of the first risk detection result belonging to each preset confidence interval, a second loss value is determined. Based on the first loss value and the second loss value, the preset large language model is trained to obtain a trained large language model. The trained large language model can be used to determine the risk detection result and confidence level corresponding to the execution of the target business. The risk detection result and confidence level can be used to determine whether there is a risk in executing the target business. In this way, the second loss value, determined based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval, can allow the model to learn the probability distribution of discrete preset confidence regions during the model training phase. Without the need for manual labeling of confidence labels, the large language model can be trained using sample data and corresponding risk labels, saving manual labeling costs. This improves model training efficiency while ensuring the accuracy of model training, thereby enhancing the effectiveness of business risk control by utilizing the trained large language model.

[0042] In practical applications, the specific processing method for determining the second loss value in step S206 above, based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval, can be varied. The following provides one optional processing method, such as... Figure 5 As shown, the specific process may include the following steps S2062 to S2064.

[0043] In step S2062, the first confidence level of the first risk detection result belonging to each preset confidence interval is obtained, and the difference between the matching degree and each first confidence level is determined.

[0044] In step S2064, the second loss value is determined based on the product between the difference and the probability value corresponding to the first confidence level.

[0045] In implementation, the server can determine the second loss value using the following formula, based on the product of the difference and the probability value corresponding to the first confidence level:

[0046] in, The second loss value, 'i', can represent the level number of the pre-set confidence interval, ranging from 0 to N integers. Each 'i' corresponds to a discrete confidence level. The first confidence value corresponding to each pre-set confidence interval can be i / N, where N represents the number of pre-set confidence intervals, i.e., the highest level number. For example, assuming there are 10 pre-set confidence intervals, then N can be 10, meaning the confidence interval [0%, 100%] can be evenly divided into N discrete levels. q i This is the probability value output by the model for "confidence level i", that is, the probability value that the first detection result belongs to the preset confidence interval i. i This can form a discrete distribution vector q, i.e., q = [q0, q1...q]. N ], satisfying Σq i =1, y can be used to characterize the matching degree between the first risk detection result and the risk label. That is, if the first risk detection result matches the risk label, then y can be 1, and if the first risk detection result does not match the risk label, then y can be 0.

[0047] like Figure 6 As shown, taking the fine-tuning of a large language model through supervised fine-tuning (SFT) as an example, the server can input preset prompts into the preset large language model to control the output of the first risk detection result and the corresponding confidence level. At the same time, the server can also obtain the confidence token distribution (i.e., the probability value of the first risk detection result belonging to each preset confidence interval). In this way, the server can use the above formula to determine the second loss value based on the difference between the first confidence level of the first risk detection result belonging to each preset confidence interval and the matching degree, as well as the probability value of the first risk detection result belonging to each preset confidence interval. Then, through the first loss value and the second loss value, the preset large language model is subjected to supervised fine-tuning. That is, the server can use the confidence token distribution to calculate the second loss value according to the true correctness of the generated answer, so as to effectively punish the improperly calibrated confidence level through the second loss value, so as to obtain the trained preset large language model.

[0048] In the above formula, (yi / N)^2 can be used to represent the squared error of the current sample data. This error will occur if the model predicts i as its confidence level, since the model has q. iThe probability of generating a confidence label i is used. This summation can be used to calculate the expected error of the model on its predicted distribution. The second loss value calculated by the above formula can simultaneously penalize both overconfidence and underconfidence of the model. For example, when the answer is incorrect (i.e., y=0), the squared error term (yi / N)^2 can degenerate into (i / N)^2, which can take a minimum value of 0 when i=0 and a maximum value of 1 when i=N. Therefore, to minimize the second loss value... The model can be incentivized to assign high probabilities to the logit q0 corresponding to 0 confidence and low probabilities to the logit q0 corresponding to full confidence. N For intermediate confidence levels, the model also tends to assign higher probabilities to lower confidence levels and lower probabilities to higher confidence levels. Conversely, when the answer is correct (i.e., y=1), the squared error term becomes (1-i / N)^2, which can take a minimum of 0 when i=N and a maximum of 1 when i=0, thus driving the model to concentrate the probability quality on q. N Through the above mechanism, the second loss value can continuously update the model parameters during the model fine-tuning process, so that the confidence label of the model output can be well calibrated with the correctness of the answer. This allows the large language model to calibrate the confidence of the judgment conclusion while giving the judgment conclusion, so as to judge the reliability of the model's reasoning conclusion.

[0049] In this way, during the fine-tuning phase, the model can directly learn the probability distribution q of discrete confidence levels (i.e., multiple pre-set confidence intervals obtained by uniformly dividing 0%–100%), without the need for pre-configuration of manual confidence labels, thereby overcoming the following problems: 1. Sensitive to prompt words: The second loss value can be uniquely solved by the above formula, which means that true calibration can be achieved; 2. High inference cost: Only a single forward propagation is needed to output the gear probability; 3. Expensive labeling: Unsupervised mode can train the model by utilizing the correctness of the sample data itself (i.e., y∈{0,1}), saving the labeling cost of domain experts.

[0050] By synthesizing sample data containing confidence levels, unsupervised calibration is then performed using a method based on appropriate scoring rules. Unlike traditional methods, the risk detection method provided in this specification does not require true confidence level labels. Instead, it uses a tokenized Brier-score loss function to determine the second loss value, thereby automatically calibrating the model's confidence level. This solution can be used in resource transaction risk detection scenarios for target businesses, providing risk assessment conclusions and confidence scores for each risk alert, helping reviewers prioritize high-risk, high-confidence alerts to improve the efficiency of risk detection.

[0051] In practical applications, the specific processing methods for step S204 above, which utilizes a preset large language model to perform risk detection on the sample data to obtain the first risk detection result and the probability value of the first risk detection result belonging to each preset confidence interval, can vary. The following provides one optional processing method, such as... Figure 7 As shown, the specific process may include the following steps S2042 to S2044.

[0052] In step S2042, the set of confidence granularities corresponding to the target business is obtained, and the target granularity is selected from the set of confidence granularities.

[0053] The confidence granularity set can contain multiple granularities. The granularity can be used to divide the confidence interval of 0%–100% to obtain multiple preset confidence intervals. For example, if the granularity is 50%, the corresponding preset confidence interval can include confidence interval 1 (i.e., [0%~50%]) and confidence interval 2 ([50%~100%]).

[0054] In step S2044, based on the preset confidence interval determined by the target granularity, a first prompt message is constructed, and a preset large language model is used to perform risk detection on the sample data based on the first prompt message to obtain a first risk detection result and the probability value of the first risk detection result belonging to each preset confidence interval.

[0055] In implementation, the server can randomly select a granularity from the confidence granularity set as the target granularity and embed this target granularity into preset prompt information to obtain the first prompt information. For example, assuming the confidence granularity set is {5, 10, 20, 50}, the first prompt information could be: "Based on the above risk analysis, please give a judgment conclusion and conduct a confidence assessment with a target granularity of 5, i.e., each 20% as a preset confidence interval." Alternatively, the first prompt information could also be: "Based on the above risk analysis, please give a judgment conclusion and conduct a confidence assessment with a target granularity of 50, i.e., each 1% as a preset confidence interval."

[0056] Then, the server can perform grouped batch iteration on the large language model. That is, to ensure the efficiency and mathematical consistency of training, a training strategy of the same granularity and the same batch is adopted during the model iteration process. 1. Sampling by target granularity: During each training iteration, samples with the same target granularity can be drawn from the data pool to form a batch.

[0057] 2. Adaptive loss function calculation: The loss function layer reads the target granularity parameters corresponding to the current batch in real time.

[0058] Through the above training scheme, the large language model can learn the mapping relationship between the target granularity and the output Logits dimension in the first prompt information, as well as the probability distribution of the confidence token. This avoids the problem that the model can only output a fixed granularity of confidence due to the fixed pre-set confidence interval. It enables the large language model to flexibly perform self-calibration of confidence at different granularities according to the specific confidence interval requirements in the first prompt information.

[0059] In practical applications, the trained large language model can also be used to detect whether there are risks in executing the target business. There are various ways to handle risk detection; the following provides one optional method, such as... Figure 8 As shown, the specific process may include the following steps S802 to S806.

[0060] In step S802, in response to a risk detection request for performing the target business, the business data required for performing the target business is obtained.

[0061] The business data required to execute the target business can include target business data, user operation data, client information, etc. For example, taking the target business as a resource transfer business, the business data required to execute the target business can include target business data such as the number of resources transferred, the time of resource transfer, and the object of resource transfer, user operation data such as user operation trajectory and user operation time, and client information such as client model and client location information.

[0062] In step S804, the trained large language model is used to perform risk detection on the business data based on the second prompt information, and the second detection result and the corresponding confidence level are obtained.

[0063] In implementation, the server can construct a second prompt message based on the business background of the target business, the standard operating procedure (SOP) for review, and the format requirements. The second prompt message and business data are then input into a trained large language model to perform risk detection on the business data and obtain the second detection result and the corresponding confidence level.

[0064] In step S806, based on the second detection result and the corresponding confidence level, it is determined whether there is a risk in executing the target business.

[0065] In practice, the server can send the second detection result and the corresponding confidence level to a preset reviewer to determine whether there is a risk in executing the target business through manual review.

[0066] In practical applications, the specific processing methods for determining whether there is a risk in executing the target business based on the second detection result and the corresponding confidence level in step S806 above can be varied. The following provides an optional processing method, which may specifically include the processing of step A1.

[0067] In step A1, if the confidence level corresponding to the second detection result is greater than the preset confidence threshold, it is determined whether there is a risk in executing the target business based on the second detection result.

[0068] In practice, if the confidence level corresponding to the second detection result is greater than the preset confidence threshold, and the second detection result indicates that there is a risk, then it can be determined that there is a risk in executing the target business; conversely, if the second detection result indicates that there is no risk, then it can be determined that there is no risk in executing the target business.

[0069] Alternatively, the server can determine the review method based on whether the confidence level is not greater than a preset confidence threshold. For example, if the confidence level is not greater than the preset confidence threshold, the server can determine whether there is a risk in executing the target business through manual review. If the confidence level is greater than the preset confidence threshold, the server can directly determine whether there is a risk in executing the target business based on the second detection result.

[0070] In practical applications, a second prompt message can also be pre-built. There are various ways to build this second prompt message; the following provides one optional method: Figure 9 As shown, the specific process may include the following steps S902 to S904.

[0071] In step S902, the user's review intensity requirements for the target business are obtained, and the first granularity is determined based on the review intensity requirements.

[0072] In implementation, the server can configure the first granularity according to the user's actual needs, that is, the user's review intensity requirements for the target business. For example, the initial review may require a coarse-grained distribution, while the secondary review may require a fine-grained probability. That is, the first granularity configured for the review intensity requirements of the initial review can be greater than the first granularity configured for the review intensity requirements of the secondary review.

[0073] In step S904, a second prompt message is constructed based on the confidence interval determined by the first granularity.

[0074] In implementation, the server can construct a second prompt based on the confidence interval determined by the first granularity. The second prompt can then be used to control the probability distribution of the corresponding confidence interval output by the large language model. In other words, the prompt can be dynamically adjusted according to the user's actual needs.

[0075] In practical applications, a second prompt message can also be pre-built. There are various ways to build this second prompt message; the following provides one optional method: Figure 10 As shown, the specific process may include the following steps S1002 to S1006.

[0076] In step S1002, the second loss value corresponding to each granularity in the confidence granularity set is obtained during the training of the large language model.

[0077] In step S1004, the second granularity in the confidence granularity set is determined based on the second loss value corresponding to each granularity.

[0078] In practice, the server can obtain the second loss value corresponding to each granularity in the confidence granularity set during the training of the large language model, and can determine the granularity corresponding to the smallest second loss value as the second granularity.

[0079] In step S1006, a second prompt message is constructed based on the confidence interval determined by the second granularity.

[0080] In practice, by using the above-mentioned strength screening, the confidence interval can be determined based on the second granularity with the least loss, so as to improve the efficiency and accuracy of risk detection for the target business in the actual application stage of the model.

[0081] In practical applications, the above step S208, which trains the preset large language model based on the first and second loss values, can be processed in various ways. One optional processing method is provided below. Figure 11 As shown, the specific process may include the following steps S2082 to S2088.

[0082] In step S2082, the reward coefficient is determined based on the first loss value.

[0083] In practice, the server can use reinforcement learning frameworks (such as PPO or DPO frameworks) to fine-tune the large language model, that is, it can guide the model to automatically calibrate the confidence of its output on the premise of giving the correct judgment.

[0084] First, the server can perform multi-track sampling (Rollout), that is, for the same sample data, it can perform K independent inferences (Rollout) using the current policy model. Each inference requires the model to output: [judgment conclusion y, confidence level c]. In this way, we can obtain the judgment set Y={y1,y2,…yK} and the corresponding confidence set C={c1,c2,…cK}.

[0085] The server can determine the reward coefficient based on the first loss value. For example, if the conclusion is correct, that is, the first risk detection result matches the risk label, the reward coefficient Racc can be 1. Conversely, if the conclusion is incorrect, that is, the first risk detection result does not match the risk label, the reward coefficient Racc can be -1.

[0086] In step S2084, the confidence level corresponding to the first risk detection result is obtained, and the third loss value is determined based on the reward coefficient and the confidence level.

[0087] In implementation, if the reward coefficient is 1, that is, the first risk detection result matches the risk label, then the third loss value can be Rconf=αci. If the reward coefficient is 0, that is, the first risk detection result does not match the risk label, then the third loss value can be Rconf=-βci, where α and β are preset adjustment coefficients.

[0088] In step S2086, the target reward value is determined based on the first loss value, the second loss value, and the third loss value.

[0089] In practice, the server can use preset weights to weight the first loss value, the second loss value, and the third loss value, and can determine the weighted value as the target reward value.

[0090] In step S2088, the policy gradient is calculated based on the target reward value, and the large language model is trained using reinforcement learning based on the policy gradient to obtain the trained large language model.

[0091] In practice, the server can use the target reward value R to calculate the policy gradient, and then extract and update the model parameters based on the calculated policy. In this way, during the evolution process, the model will adjust the confidence level when uncertain and increase the confidence level when certain in order to maximize the reward.

[0092] In summary, by discretizing the continuous confidence interval into N confidence levels (0%–100%, i.e., pre-set confidence intervals), the second loss value is determined as the differentiable expectation l(q,y)=Σq. i (y i / N)², can directly perform backpropagation on the logits of the LLM, achieving proper calibration without manual labels.

[0093] Furthermore, since only the correctness of the sample data itself, y∈{0,1}, is needed as the supervision signal, the loss function constructed by the above formula drives the model to self-adjust the confidence distribution. This can eliminate the need for high-cost domain expert annotation, adapt to the dynamic changes in the distribution of business data in the target business scenario, and achieve an unsupervised self-calibration mechanism.

[0094] This specification provides a risk detection method that can acquire sample data corresponding to a target business and risk labels corresponding to the sample data. Using a preset large language model, risk detection is performed on the sample data to obtain a first risk detection result and the probability value of the first risk detection result belonging to each preset confidence interval. Based on the first risk detection result and the risk labels, a first loss value is determined. Based on the matching degree between the first risk detection result and the risk labels, and the probability value of the first risk detection result belonging to each preset confidence interval, a second loss value is determined. Based on the first loss value and the second loss value, the preset large language model is trained to obtain a trained large language model. The trained large language model can be used to determine the risk detection result and confidence level corresponding to the execution of the target business. The risk detection result and confidence level can be used to determine whether there is a risk in executing the target business. In this way, the second loss value, determined based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval, can allow the model to learn the probability distribution of discrete preset confidence regions during the model training phase. Without the need for manual labeling of confidence labels, the large language model can be trained using sample data and corresponding risk labels, saving manual labeling costs. This improves model training efficiency while ensuring the accuracy of model training, thereby enhancing the effectiveness of business risk control by utilizing the trained large language model.

[0095] The above describes the risk detection method provided in the embodiments of this specification. Based on the same idea, the embodiments of this specification also provide a risk detection device, such as... Figure 12 As shown.

[0096] The risk detection device includes: a data acquisition module 1201, a first detection module 1202, a loss determination module 1203, and a model training module 1204, wherein: The data acquisition module 1201 acquires sample data corresponding to the target business, as well as risk labels corresponding to the sample data; The first detection module 1202 uses a preset large language model to perform risk detection on the sample data, and obtains a first risk detection result and the probability value of the first risk detection result belonging to each preset confidence interval. The loss determination module 1203 determines a first loss value based on the first risk detection result and the risk label, and determines a second loss value based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval. The model training module 1204 trains the preset large language model based on the first loss value and the second loss value to obtain the trained large language model. The trained large language model is used to determine the risk detection result and confidence level corresponding to the execution of the target business. The risk detection result and confidence level are used to determine whether there is a risk in executing the target business.

[0097] In the embodiments of this specification, the loss determination module 1203 is used for: Obtain the first confidence level of the first risk detection result belonging to each preset confidence interval, and determine the difference between the matching degree and each of the first confidence levels; The second loss value is determined based on the product of the difference and the probability value corresponding to the first confidence level.

[0098] In this embodiment of the specification, the first detection module 1202 is used for: Obtain the set of confidence granularities corresponding to the target business, and select the target granularity from the set of confidence granularities; Based on the preset confidence intervals determined by the target granularity, a first prompt message is constructed, and the preset large language model is used to perform risk detection on the sample data based on the first prompt message to obtain the first risk detection result and the probability value of the first risk detection result belonging to each preset confidence interval.

[0099] In the embodiments described in this specification, the device further includes: The acquisition module, in response to a risk detection request for executing the target service, acquires the business data required for executing the target service; The detection module uses the trained large language model to perform risk detection on the business data based on the second prompt information, and obtains the second detection result and the corresponding confidence level. The judgment module determines whether there is a risk in executing the target service based on the second detection result and the corresponding confidence level.

[0100] In the embodiments described in this specification, the device further includes: The first determining module obtains the user's review intensity requirements for the target business, and determines the first granularity based on the review intensity requirements; The first construction module constructs the second prompt information based on the confidence interval determined by the first granularity.

[0101] In the embodiments described in this specification, the device further includes: The loss acquisition module acquires the second loss value corresponding to each granularity in the confidence granularity set during the training of the large language model. The second determining module determines the second granularity in the confidence granularity set based on the second loss value corresponding to each granularity; The second construction module constructs the second prompt information based on the confidence interval determined by the second granularity.

[0102] In the embodiments described in this specification, the model training module 1204 is used for: Based on the first loss value, determine the reward coefficient; Obtain the confidence level corresponding to the first risk detection result, and determine the third loss value based on the reward coefficient and the confidence level; Based on the first loss value, the second loss value, and the third loss value, the target reward value is determined; The policy gradient is calculated based on the target reward value, and the large language model is trained using reinforcement learning based on the policy gradient to obtain the trained large language model.

[0103] In the embodiments of this specification, the determination module is used for: If the confidence level corresponding to the second detection result is greater than the preset confidence threshold, a determination is made based on the second detection result to determine whether there is a risk in executing the target service.

[0104] This specification provides a risk detection device that can acquire sample data corresponding to a target business and risk labels corresponding to the sample data. Using a preset large language model, it performs risk detection on the sample data to obtain a first risk detection result and a probability value for the first risk detection result belonging to each preset confidence interval. Based on the first risk detection result and the risk labels, a first loss value is determined. Based on the matching degree between the first risk detection result and the risk labels, and the probability value for the first risk detection result belonging to each preset confidence interval, a second loss value is determined. Based on the first loss value and the second loss value, the preset large language model is trained to obtain a trained large language model. The trained large language model can be used to determine the risk detection result and confidence level corresponding to the execution of the target business. The risk detection result and confidence level can be used to determine whether there is a risk in executing the target business. In this way, the second loss value, determined based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval, can allow the model to learn the probability distribution of discrete preset confidence regions during the model training phase. Without the need for manual labeling of confidence labels, the large language model can be trained using sample data and corresponding risk labels, saving manual labeling costs. This improves model training efficiency while ensuring the accuracy of model training, thereby enhancing the effectiveness of business risk control by utilizing the trained large language model.

[0105] The above are risk detection devices provided in the embodiments of this specification. Based on the same idea, the embodiments of this specification also provide a risk detection device, such as... Figure 13 As shown.

[0106] The risk detection device can provide terminal equipment or servers, etc., for the above embodiments.

[0107] like Figure 13As shown, device 1300 mainly consists of a communication interface 1302, a user interface 1304, a processor 1306, and a data storage 1308. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 1310. The communication interface 1302 enables device 1300 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 1302 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 1302 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 1302 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 1302 may also include multiple physical communication interfaces, such as Wi-Fi, Bluetooth, and wide-area wireless interfaces.

[0108] User interface 1304 includes receiving user input and providing output to the user. Therefore, user interface 1304 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 1304 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 1304 may include software, circuitry, or other forms of logic capable of transmitting and receiving data from external user input / output devices. Additionally or alternatively, device 1300 may support remote access from other devices via communication interface 1302 or another physical interface (not shown). User interface 1304 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 1304 can also be configured as a display device for rendering or displaying text fragments.

[0109] Processor 1306 may include one or more general-purpose processors and / or special-purpose processors.

[0110] Data storage 1308 may include one or more volatile storage components and may be integrated wholly or partially with processor 1306. Data storage 1308 may include removable and non-removable components.

[0111] Processor 1306 is capable of executing program instructions 1318 (e.g., compiled or uncompiled program logic and / or machine code) stored in data store 1308 to perform the various functions described herein. Data store 1308 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 1300, enable device 1300 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Execution of program instructions 1318 by processor 1306 may result in processor 1306 using data 1312.

[0112] For example, program instructions 1318 may include an operating system 1322 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 1300 and one or more applications 1320 (e.g., a browser, social application, or game application). Similarly, data 1312 may include operating system data 1316 and application data 1314. Operating system data 1316 is primarily accessible to the operating system 1322, while application data 1314 is primarily accessible to one or more applications 1320. Application data 1314 may reside in a file system visible or hidden from the user of device 1300.

[0113] Application 1320 can communicate with operating system 1312 through one or more application programming interfaces (APIs). These APIs help application 1320 read and / or write application data 1314, transmit or receive information via communication interface 1302, receive or display information on user interface 1304, etc.

[0114] In some terminology, application 1320 may be simply referred to as "app". Furthermore, application 1320 can be downloaded to device 1300 through one or more online app stores or app markets. However, applications can also be installed on device 1300 in other ways, such as through a web browser or a physical interface on device 1300 (e.g., a USB port).

[0115] Specifically, in this embodiment, the risk detection device includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the risk detection device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following: Obtain sample data corresponding to the target business, and risk labels corresponding to the sample data; Using a pre-set large language model, risk detection is performed on the sample data to obtain a first risk detection result and the probability value of the first risk detection result belonging to each pre-set confidence interval; Based on the first risk detection result and the risk label, a first loss value is determined, and based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval, a second loss value is determined. Based on the first loss value and the second loss value, the preset large language model is trained to obtain the trained large language model. The trained large language model is used to determine the risk detection result and confidence level corresponding to the execution of the target business. The risk detection result and confidence level are used to determine whether there is a risk in executing the target business.

[0116] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the risk detection device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0117] This specification provides a risk detection device that can acquire sample data corresponding to a target business and risk labels corresponding to the sample data. Using a preset large language model, it performs risk detection on the sample data to obtain a first risk detection result and the probability value of the first risk detection result belonging to each preset confidence interval. Based on the first risk detection result and the risk labels, a first loss value is determined. Based on the matching degree between the first risk detection result and the risk labels, and the probability value of the first risk detection result belonging to each preset confidence interval, a second loss value is determined. Based on the first loss value and the second loss value, the preset large language model is trained to obtain a trained large language model. The trained large language model can be used to determine the risk detection result and confidence level corresponding to the execution of the target business. The risk detection result and confidence level can be used to determine whether there is a risk in executing the target business. In this way, the second loss value, determined based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval, can allow the model to learn the probability distribution of discrete preset confidence regions during the model training phase. Without the need for manual labeling of confidence labels, the large language model can be trained using sample data and corresponding risk labels, saving manual labeling costs. This improves model training efficiency while ensuring the accuracy of model training, thereby enhancing the effectiveness of business risk control by utilizing the trained large language model.

[0118] Furthermore, based on the above Figures 1 to 11 This specification also provides a storage medium for storing computer-executable instruction information in one or more embodiments. In one specific embodiment, the storage medium may be a USB flash drive, optical disc, hard disk, etc. When the computer-executable instruction information stored in the storage medium is executed by a processor, it can realize the following process: Obtain sample data corresponding to the target business, and risk labels corresponding to the sample data; Using a pre-set large language model, risk detection is performed on the sample data to obtain a first risk detection result and the probability value of the first risk detection result belonging to each pre-set confidence interval; Based on the first risk detection result and the risk label, a first loss value is determined, and based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval, a second loss value is determined. Based on the first loss value and the second loss value, the preset large language model is trained to obtain the trained large language model. The trained large language model is used to determine the risk detection result and confidence level corresponding to the execution of the target business. The risk detection result and confidence level are used to determine whether there is a risk in executing the target business.

[0119] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the above-described storage medium embodiment is basically similar to the method embodiment, so the description is relatively simple; relevant parts can be referred to the description of the method embodiment.

[0120] This specification provides a storage medium that can acquire sample data corresponding to a target business and risk labels corresponding to the sample data. Using a preset large language model, risk detection is performed on the sample data to obtain a first risk detection result and the probability value of the first risk detection result belonging to each preset confidence interval. Based on the first risk detection result and the risk labels, a first loss value is determined. Based on the matching degree between the first risk detection result and the risk labels, and the probability value of the first risk detection result belonging to each preset confidence interval, a second loss value is determined. Based on the first loss value and the second loss value, the preset large language model is trained to obtain a trained large language model. The trained large language model can be used to determine the risk detection result and confidence level corresponding to the execution of the target business. The risk detection result and confidence level can be used to determine whether there is a risk in executing the target business. In this way, the second loss value, determined based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval, can allow the model to learn the probability distribution of discrete preset confidence regions during the model training phase. Without the need for manual labeling of confidence labels, the large language model can be trained using sample data and corresponding risk labels, saving manual labeling costs. This improves model training efficiency while ensuring the accuracy of model training, thereby enhancing the effectiveness of business risk control by utilizing the trained large language model.

[0121] Furthermore, based on the above Figures 1 to 11 This specification also provides one or more embodiments of a computer program product, including a computer program, which, when executed by a processor, can perform the following processes: Obtain sample data corresponding to the target business, and risk labels corresponding to the sample data; Using a pre-set large language model, risk detection is performed on the sample data to obtain a first risk detection result and the probability value of the first risk detection result belonging to each pre-set confidence interval; Based on the first risk detection result and the risk label, a first loss value is determined, and based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval, a second loss value is determined. Based on the first loss value and the second loss value, the preset large language model is trained to obtain the trained large language model. The trained large language model is used to determine the risk detection result and confidence level corresponding to the execution of the target business. The risk detection result and confidence level are used to determine whether there is a risk in executing the target business.

[0122] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the above-described embodiment of a computer program product is relatively simple in description because it is fundamentally similar to the method embodiment; relevant parts can be referred to the description of the method embodiment.

[0123] This specification provides a computer program product that can acquire sample data corresponding to a target business and risk labels corresponding to the sample data. Using a preset large language model, it performs risk detection on the sample data to obtain a first risk detection result and the probability value of the first risk detection result belonging to each preset confidence interval. Based on the first risk detection result and the risk labels, a first loss value is determined. Based on the matching degree between the first risk detection result and the risk labels, and the probability value of the first risk detection result belonging to each preset confidence interval, a second loss value is determined. Based on the first loss value and the second loss value, the preset large language model is trained to obtain a trained large language model. The trained large language model can be used to determine the risk detection result and confidence level corresponding to the execution of the target business. The risk detection result and confidence level can be used to determine whether there is a risk in executing the target business. In this way, the second loss value, determined based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval, can allow the model to learn the probability distribution of discrete preset confidence regions during the model training phase. Without the need for manual labeling of confidence labels, the large language model can be trained using sample data and corresponding risk labels, saving manual labeling costs. This improves model training efficiency while ensuring the accuracy of model training, thereby enhancing the effectiveness of business risk control by utilizing the trained large language model.

[0124] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0125] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using a hardware physical module. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0126] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0127] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0128] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0129] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0130] The embodiments described herein are illustrated with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0131] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0132] These computer program instructions may also be loaded onto a computer or other programmable device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0133] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0134] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0135] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0136] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitations, the presence of additional identical elements in the process, method, article, or apparatus that includes said elements is not excluded.

[0137] It should also be noted that the terms "one," "an," and "the" do not specifically refer to the singular; they can also include the plural. Ordinal numbers such as "first," "second," etc., do not necessarily indicate order; often they are used to distinguish between objects. For example, "first server" and "second server" usually refer to two servers. To differentiate between these two servers, they are described as "first server" and "second server." Of course, sometimes these two servers may be the same server.

[0138] In this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can also mean indirect receiving and sending. For example, A receiving data sent by B can be understood as A directly receiving data sent by B, or it can be understood as A indirectly receiving data sent by B through other entities such as C. Similarly, B sending data to A can be understood as B sending data directly to A, or it can be understood as B indirectly sending data to A through other entities such as C. Here, C can be a single subject, or two or more subjects.

[0139] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations within this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0140] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is only one of many possible execution orders and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims.

[0141] It should be noted that the user data obtained in this manual is authorized by the user and does not involve user privacy.

[0142] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0144] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0145] The above description is merely an embodiment of this specification and is not intended to limit this document. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A risk detection method, comprising: Obtain sample data corresponding to the target business, and risk labels corresponding to the sample data; Using a pre-set large language model, risk detection is performed on the sample data to obtain a first risk detection result and the probability value of the first risk detection result belonging to each pre-set confidence interval; Based on the first risk detection result and the risk label, a first loss value is determined, and based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval, a second loss value is determined. Based on the first loss value and the second loss value, the preset large language model is trained to obtain the trained large language model. The trained large language model is used to determine the risk detection result and confidence level corresponding to the execution of the target business. The risk detection result and confidence level are used to determine whether there is a risk in executing the target business.

2. The method according to claim 1, wherein determining the second loss value based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval, comprises: Obtain the first confidence level of the first risk detection result belonging to each preset confidence interval, and determine the difference between the matching degree and each of the first confidence levels; The second loss value is determined based on the product of the difference and the probability value corresponding to the first confidence level.

3. The method according to claim 1, wherein the step of using a preset large language model to perform risk detection on the sample data to obtain a first risk detection result, and the probability value of the first risk detection result belonging to each preset confidence interval, includes: Obtain the set of confidence granularities corresponding to the target business, and select the target granularity from the set of confidence granularities; Based on the preset confidence intervals determined by the target granularity, a first prompt message is constructed, and the preset large language model is used to perform risk detection on the sample data based on the first prompt message to obtain the first risk detection result and the probability value of the first risk detection result belonging to each preset confidence interval.

4. The method according to claim 1, further comprising: In response to a risk detection request for performing the target service, obtain the business data required to perform the target service; Using the trained large language model, risk detection is performed on the business data based on the second prompt information to obtain the second detection result and the corresponding confidence level; Based on the second detection result and the corresponding confidence level, it is determined whether there is a risk in executing the target service.

5. The method according to claim 4, before performing risk detection on the business data based on the second prompt information using the trained large language model to obtain the second detection result and the corresponding confidence level, further includes: Obtain the user's review intensity requirements for the target business, and determine the first granularity based on the review intensity requirements; The second prompt information is constructed based on the confidence interval determined by the first granularity.

6. The method according to claim 4, before performing risk detection on the business data using the trained large language model to obtain a second detection result and the corresponding confidence level, further comprising: During the training of the large language model, obtain the second loss value corresponding to each granularity in the set of confidence granularities; Based on the second loss value corresponding to each granularity, the second granularity in the confidence granularity set is determined; The second prompt message is constructed based on the confidence interval determined by the second granularity.

7. The method according to claim 1, wherein training the preset large language model based on the first loss value and the second loss value to obtain the trained large language model includes: Based on the first loss value, determine the reward coefficient; Obtain the confidence level corresponding to the first risk detection result, and determine the third loss value based on the reward coefficient and the confidence level; Based on the first loss value, the second loss value, and the third loss value, the target reward value is determined; The policy gradient is calculated based on the target reward value, and the large language model is trained using reinforcement learning based on the policy gradient to obtain the trained large language model.

8. The method according to claim 4, wherein determining whether there is a risk in executing the target service based on the second detection result and the corresponding confidence level includes: If the confidence level corresponding to the second detection result is greater than the preset confidence threshold, a determination is made based on the second detection result to determine whether there is a risk in executing the target service.

9. A risk detection device, comprising: The data acquisition module acquires sample data corresponding to the target business, as well as risk labels corresponding to the sample data; The first detection module uses a preset large language model to perform risk detection on the sample data, and obtains a first risk detection result and the probability value of the first risk detection result belonging to each preset confidence interval. The loss determination module determines a first loss value based on the first risk detection result and the risk label, and determines a second loss value based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval. The model training module trains the preset large language model based on the first loss value and the second loss value to obtain the trained large language model. The trained large language model is used to determine the risk detection result and confidence level corresponding to the execution of the target business. The risk detection result and confidence level are used to determine whether there is a risk in executing the target business.

10. A risk detection device, wherein the risk detection includes: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to: Obtain sample data corresponding to the target business, and risk labels corresponding to the sample data; Using a pre-set large language model, risk detection is performed on the sample data to obtain a first risk detection result and the probability value of the first risk detection result belonging to each pre-set confidence interval; Based on the first risk detection result and the risk label, a first loss value is determined, and based on the matching degree between the first risk detection result and the risk label, and the probability value of the first risk detection result belonging to each preset confidence interval, a second loss value is determined. Based on the first loss value and the second loss value, the preset large language model is trained to obtain the trained large language model. The trained large language model is used to determine the risk detection result and confidence level corresponding to the execution of the target business. The risk detection result and confidence level are used to determine whether there is a risk in executing the target business.

Images