Risk control rule mining method, device, storage medium and program product

Abstract

This specification provides a risk control rule mining method, device, storage medium, and program product. The risk control rule mining method includes: acquiring a sample set containing multiple samples, each sample including the value of at least one risk feature variable and associated risk labels; binning each risk feature variable according to binning parameters and the value of the risk feature variable in the sample set, and performing statistical analysis based on the distribution of risk labels associated with samples in each bin to obtain binning feature information of the current binning method; constructing a first prompt word based on preset binning decision knowledge, semantic information of risk feature variables, and binning feature information; inputting the first prompt word into a language model to enable the language model to perform binning rationality analysis, outputting binning parameters for the next binning process when the current binning method is determined to be unreasonable, or outputting risk control rules for the risk feature variables when the current binning method is determined to be reasonable.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a risk control rule mining method, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] In the field of risk control management, risk control rule mining is the core link to achieving automated risk identification and decision-making.

[0003] Currently, the mainstream methods for generating risk control rules mainly rely on the following two implementation methods: In one implementation method, risk control experts manually write and configure rules based on their experience. This method is highly dependent on personal experience, subjective, time-consuming, and difficult to scale. When the application scenario or risk model changes, the rules are updated and iterated in a lag, making it difficult to quickly adapt to new risk characteristics.

[0004] Another approach involves using machine learning-based rule mining methods (such as decision trees and association rule mining) to automatically extract risk control rules from historical data. While these methods improve mining efficiency, their process relies entirely on statistical significance, lacking an inherent consideration of the application scenario's processing logic and semantic rationality. Therefore, the resulting rule sets often contain a large number of logically uninterpretable, difficult-to-deploy, or even contradictory risk control rules, requiring significant manual intervention for screening, interpretation, and correction. In essence, this approach does not completely eliminate human intervention, and the quality and acceptability of the resulting rule sets remain a significant bottleneck. Summary of the Invention

[0005] In view of the above, one or more embodiments of this specification provide the following technical solutions: According to a first aspect of one or more embodiments of this specification, a risk control rule mining method is proposed, comprising: Obtain a sample set containing multiple samples, each sample including the value of at least one risk characteristic variable and an associated risk label; For each of the aforementioned risk characteristic variables, binning is performed according to the binning parameters and the values ​​of the risk characteristic variables in the sample set. Statistical analysis is then performed based on the distribution of risk labels associated with samples in each bin to obtain binning characteristic information of the current binning method. The first prompt word is constructed based on the preset binning decision knowledge, the semantic information of the risk characteristic variables, and the binning characteristic information; The first prompt word is input into the language model so that the language model can perform binning rationality analysis. If the current binning method is determined to be unreasonable, the binning parameters for the next binning process are output. Alternatively, if the current binning method is determined to be reasonable, the risk control rules for the risk characteristic variables are output. The binning parameters are determined based on the binning decision knowledge and the semantic information of the risk characteristic variables, and the risk control rules are determined based on the semantic information of the risk characteristic variables and the binning characteristic information.

[0006] According to a second aspect of the embodiments of this specification, an electronic device is provided, comprising: processor; Memory used to store processor-executable instructions; Wherein, when the processor executes the executable instructions, it is used to implement the method described in the first aspect.

[0007] According to a third aspect of the embodiments of this specification, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in the first aspect.

[0008] According to a fourth aspect of the embodiments of this specification, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0009] As can be seen from the above embodiments, this specification constructs a first prompt word by combining binning decision knowledge, semantic information of risk feature variables, and binning feature information of risk feature variables in the current binning method. The first prompt word drives the language model to perform reasoning, so that the rule mining process no longer relies on manual writing by experts, thus improving the efficiency of rule generation. By injecting binning decision knowledge and semantic information into the language model through the first prompt word, its reasoning process takes into account the interpretability and logical rationality of variables in real-world scenarios, thereby avoiding or reducing the generation of unreasonable rules. While improving the level of automation, it ensures the semantic rationality of the output risk control rules.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0011] Figure 1 This is a flowchart of a risk control rule mining method provided in an exemplary embodiment.

[0012] Figure 2 This is a timing diagram of a binning process provided in an exemplary embodiment.

[0013] Figure 3This is a timing diagram of a rule derivation process provided in an exemplary embodiment.

[0014] Figure 4 This is a timing diagram of a rule combination process provided in an exemplary embodiment.

[0015] Figure 5 This is a timing diagram of a rule-based filtering process provided in an exemplary embodiment.

[0016] Figure 6 This is a schematic diagram of risk control rules mined by combining two rule mining methods, provided in an exemplary embodiment.

[0017] Figure 7 This is a schematic diagram of the structure of a device provided in an exemplary embodiment. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0019] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of related data shall comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.

[0020] The following explains the relevant terms mentioned in the embodiments of this specification: (1) Language model.

[0021] In the various embodiments of this specification, the language model is used for binning rationality assessment, risk control rule derivation, risk control rule combination, and risk control rule screening.

[0022] Language models can be text processing models built based on machine learning or deep learning, such as sequence modeling models based on neural networks, models based on attention mechanisms, models based on encoder-decoder structures, or combinations of the above models. The embodiments in this specification do not limit the specific algorithm type, network structure, or parameter size of the language model.

[0023] Regarding model training and adaptation, one or more of the following methods can be used for training or optimization: ① Domain-adaptive training: Based on pre-training with general corpora, further pre-training or fine-tuning can be performed using textual data related to the risk control domain (including but not limited to feature variable description documents, rule configuration instructions, historical binning analysis reports, etc.) to enhance the model's ability to understand the semantics of the risk control domain. ② Prompt engineering and context learning: By carefully designing input prompt word templates and utilizing the model's context learning capabilities, the model can be guided to complete the risk control rule mining task without changing its parameters.

[0024] In terms of deployment and invocation, this language model can be deployed as a standalone service and invoked by the mining agent described in this manual via an API interface; it can also be integrated into the system as an embedded component.

[0025] (2) Risk control rules.

[0026] The components of a risk control rule include: ① Risk characteristic variables: These are the data inputs upon which the rules rely for logical judgment. They refer to quantitative or qualitative indicators extracted from users, transactions, devices, or historical behavior that can characterize the degree of risk, such as the amount of this transaction, the number of abnormal logins by a user in the past 30 days, or the distance between the application location and the user's usual location. Variables are the bridge connecting the raw data and the rule logic.

[0027] ② Judgment Condition: This is the core logical expression of the rule, specifying how to evaluate risk characteristic variables. It typically consists of an operator and a specific threshold (such as the numerical value 1000, or a list [location A, location B]). Operators include comparison operators (>, <, ≥, ≤, =) and logical connectors (& for AND, | for OR). For example, a transaction amount > 1000 or the application location not being in the frequently used location list. This condition determines when the rule will be triggered.

[0028] ③ Decision Result (Optional): This is the explicit action or output executed by the system after the rule is triggered (i.e., the judgment condition is true), representing the conclusion of this risk decision. Typical decision results include: rejecting the transaction, approving the application, transferring to manual review, sending an alert, or increasing the risk score by 10 points. It directly translates logical judgment into concrete action.

[0029] In the field of risk control management, risk control rule mining is a crucial foundation for risk identification and automated decision-making. Related methods mainly fall into two categories: manually written rules and machine learning-based automated rule mining. The former relies on expert experience, is highly subjective, inefficient, and lags behind in updating as scenarios change; the latter, while improving mining efficiency, is primarily based on statistical correlation, lacking consideration for business semantics and logical rationality, and is prone to generating rules that are difficult to interpret or deploy, still requiring extensive manual verification and correction, resulting in insufficient rule quality and practicality.

[0030] For example, in a credit card application risk control scenario, machine learning-based automatic mining methods may uncover a risk control rule like this: "More than 10 online transactions during non-working hours (e.g., 11:00 PM - 5:00 AM) in the three months prior to application, and age <25 → high risk."

[0031] While this risk control rule may show statistical significance in some training data, such as a high proportion of bad samples, its logical rationale is seriously insufficient and it is difficult to deploy: ① Unexplainable: This rule forcibly associates "nighttime transactions" with "youth" as high-risk, lacking any causal logic that can be understood by domain experts. Nighttime transactions may stem from time zone differences, overtime work expenses, or nighttime entertainment, and there is no recognized direct link between them and fraud risk.

[0032] ② Difficult to deploy: If this rule is implemented, it will mistakenly exclude a large number of legitimate users, such as young people working at night and overseas students, leading to customer complaints and questions about fairness. Furthermore, the rule is easily circumvented by fraudsters (who may then switch to daytime trading), failing to create an effective risk barrier.

[0033] ③ Potential conflict with other rules: The system may also have another rule: "Average monthly spending > 5000 yuan → Low risk". For a young user who frequently spends money at night, the two rules will give completely opposite risk assessments, leading to a decision conflict.

[0034] Based on this, please refer to Figure 1 as well as Figure 2 This specification provides a risk control rule mining method, which can be executed by a mining agent. The mining agent can be implemented as a software entity or logical function module running on an electronic device, used to coordinate the processing flow including sample acquisition, binning, prompt word construction, language model invocation, and risk control rule output. By introducing a mining agent, the multi-step processing logic involved in the risk control rule mining process can be centrally scheduled and automated, thereby reducing the degree of manual intervention, improving the stability and execution efficiency of the rule mining process, and facilitating deployment and expansion in different computing platforms or system environments. The risk control rule mining method includes: In S100, a sample set containing multiple samples is obtained, each sample including the value of at least one risk characteristic variable and an associated risk label.

[0035] In this embodiment, the sample set is used to characterize the characteristic state of historical objects during risk control judgment and their corresponding results. Each sample includes at least one risk characteristic variable value and a risk label corresponding to that sample. The risk characteristic variable describes the attribute state of the sample in a certain dimension, such as transaction amount, account registration duration, device usage frequency, IP address stability, historical default count, and number of operations in the recent period—continuous or discrete numerical variables. The risk label characterizes the risk outcome state of the sample, such as whether a default occurred, whether fraud was detected, or whether the manual review process was initiated—binary or multi-classification labels.

[0036] For example, a sample may include the following: the risk characteristic variable "transaction amount in the past 30 days" is 85,000 yuan, "account registration duration" is 12 days, and "number of operations in the past 7 days" is 63 times; the corresponding risk label is "high risk" or denoted as label value 1, which is used to indicate that the sample is judged to have default or abnormal behavior in subsequent verification.

[0037] For example, another sample could include: risk characteristic variables "transaction amount in the past 30 days" of 3,200 yuan, "account registration duration" of 540 days, and "number of operations in the past 7 days" of 5 times; the corresponding risk label is "low risk" or denoted as label value 0, which indicates that the sample has not experienced any abnormalities or defaults.

[0038] By simultaneously retaining risk feature variables and risk labels in the sample, a correspondence between feature values ​​and risk outcomes can be established in subsequent processing, providing a data foundation for binning statistical analysis and rule condition generation, thereby ensuring that the mined rules are verifiable and interpretable.

[0039] In S102, for each risk characteristic variable, binning is performed according to the binning parameters and the value of the risk characteristic variable in the sample set. Statistical analysis is then performed based on the distribution of risk labels associated with samples in each bin to obtain binning characteristic information of the current binning method.

[0040] In this embodiment, binning is performed independently for each risk characteristic variable to map continuous variables to a finite number of bin intervals. For specific implementation details, please refer to [link to implementation details]. Figure 2In sections 23-24, the mining agent can invoke binning tools, such as binning tools based on chi-square binning, equal-frequency binning, equal-distance binning, or optimal binning algorithms. The binning parameters and sample set are used as input parameters for the binning tool. For each risk feature variable, the binning tool performs binning processing on the value of the risk feature variable in the sample set according to the preset binning parameters, forming multiple binning intervals. Further, based on the risk label distribution of the samples in each bin, statistical analysis is performed to calculate various statistical indicators corresponding to the current binning method. Finally, the binning results are returned, including the boundary values ​​corresponding to each bin and various statistical indicators corresponding to the current binning method.

[0041] The binning parameters may include at least one of the following: a preset binning algorithm, the number of bins, a lower limit on the sample size in a single bin, whether a monotonic trend constraint is required, whether empty bins are allowed, a merging threshold condition, and a missing value handling strategy.

[0042] The merging threshold condition refers to the judgment criteria or constraints used in the container processing to determine whether adjacent container intervals need to be merged. For example, when the difference in risk indicators (such as badRate difference or lift difference) between adjacent container intervals is less than a preset threshold, the adjacent container intervals are merged.

[0043] Missing value handling strategies refer to the rules for handling missing samples by binning or numerical substitution when there are missing values ​​in risk characteristic variables; for example, dividing the missing value samples into a separate independent binning interval; or temporarily not using the missing value samples during binning and statistical analysis.

[0044] The binning parameters used in the initial binning process can be manually set; the binning parameters used in subsequent binning processes are output by the language model.

[0045] For example, suppose the risk characteristic variable is "transaction amount in the past 30 days," with a value range of 0 to 200,000 in the sample set, and a total sample size of 100,000. The data mining agent calls a binning tool to perform equal-frequency binning on this risk characteristic variable according to the preset initial bin number bin=10. That is, all samples are sorted according to the variable value from smallest to largest and divided into 10 bin intervals with roughly the same number of samples.

[0046] For example, the binning tool can generate the following binning intervals: Bin 1: [0, 1200]; Bin 2: (1200, 2800]; Bin 3: (2800, 5600]; Bin 4: (5600, 9500]; Bin 5: (9500, 15000]; Bin 6: (15000, 23000]; Bin 7: (23000, 35000]; Bin 8: (35000, 55000]; Bin 9: (55000, 90000]; Bin 10: (90 [000, 200000]. The sample size for each of the 10 binning intervals is around 10,000. After binning is completed, the binning tool further performs statistical analysis based on the risk labels associated with the samples in each binning interval to calculate binning-level statistical indicators such as the bad rate and lift for each binning interval. It also calculates variable-level statistical indicators such as the information value (IV) and KS value in combination with the overall sample distribution, thereby forming binning characteristic information that represents the current binning method, which is used for subsequent binning rationality analysis and rule generation processes.

[0047] After binning and statistical analysis of various risk characteristic variables using binning tools, the resulting binning feature information is used to characterize the current binning method at both structural and effectiveness levels. This binning feature information includes: (1) Binning structure information, used to describe the formal characteristics of the binning results, including at least one of the following: the binning algorithm applied, the number of bins, the boundary values ​​corresponding to each bin, the number of samples corresponding to each bin, and the lower limit of the allowed sample size for a single bin.

[0048] (2) Binning statistics, used to describe the risk differentiation ability of binning results, including variable-level indicators obtained from the overall dimension of risk characteristic variables, such as information value (IV) and Kolmogorov-Smirnov (KS) indicators, as well as binning-level indicators obtained from the binning dimension, such as the risk sample ratio (badRate) and risk lift corresponding to each bin.

[0049] The Information Value (IV) measures the ability of a risk characteristic variable to distinguish risk labels, reflecting the amount of information it provides when differentiating samples from different risk states. In this embodiment, the IV is calculated based on the distribution differences between the proportions of positive and negative samples within each binning interval. Generally, a larger IV value indicates a stronger ability of the risk characteristic variable to distinguish risk labels; a smaller IV value indicates a weaker ability. By introducing IV as a variable-level statistical indicator, it can be used to assess whether the risk characteristic variable has the potential to form effective risk control rules under the current binning method, thus providing a quantitative basis for whether to continue optimizing binning or generating rules.

[0050] The KS value is used to measure the degree of distribution difference of a risk characteristic variable across samples with different risk labels. In this embodiment, the KS value is obtained by comparing the cumulative distribution function differences between high-risk and low-risk samples in different binning intervals, and its value typically ranges from 0 to 1. The larger the KS value, the more significant the distinguishing effect of the characteristic variable on different risk categories. The KS value can be used to evaluate the distinguishing ability of the current binning method from the perspective of the overall distribution, helping to determine whether the binning structure can effectively distinguish samples with different risk states.

[0051] Lift measures the degree to which the risk level of a sample within a bin delimited area deviates from the overall average risk level. In this embodiment, the lift of each bin delimited area can be calculated as the ratio of the proportion of bad samples within that bin delimited area to the proportion of bad samples in the overall sample. When the lift is greater than 1, it indicates that the risk level of that bin delimited area is higher than the overall average risk level; when the lift is less than 1, it indicates that the risk level of that bin delimited area is lower than the overall average risk level. By analyzing the lift distribution of each bin delimited area, high-risk or low-risk concentration areas can be identified, providing a basis for determining rule thresholds and operators.

[0052] The bad rate represents the proportion of risky samples within a given binning interval out of all samples in that binning interval. In this embodiment, the bad rate is obtained by calculating the ratio between the number of samples in the binning interval with a preset high-risk label and the total number of samples in that binning interval. A higher bad rate indicates a greater likelihood of a risk event occurring within that binning interval. The bad rate provides a clear picture of the risk intensity corresponding to different binning intervals, helping the language model determine the trend of risk changes with variable values ​​during binning rationality analysis and rule generation.

[0053] It is understandable that different types of risk characteristic variables differ in data distribution and value characteristics. Therefore, the binning algorithm and corresponding binning parameters used when performing binning can be set differently according to the variable type.

[0054] For example, for continuous risk characteristic variables, equal-frequency binning, equal-interval binning, chi-square binning, or optimal binning algorithms can be used, with a larger initial number of bins set to more precisely characterize the relationship between the value range and the risk level; while for discrete risk characteristic variables, binning methods based on value enumeration or category merging can be used, with a smaller initial number of bins set accordingly, or different discrete values ​​can be directly used as candidate binning intervals.

[0055] Furthermore, different risk characteristic variables have different value ranges, distribution patterns, and missing values ​​in the sample set. The binning parameters used during the initial binning (such as the number of bins, the lower limit of the sample size in a single bin, and whether to enable monotonic trend constraints) can also be configured separately according to the characteristics of the variables, rather than being limited to uniform values. This improves the adaptability of the binning results to different variables and the stability of subsequent rule generation.

[0056] In S104, the first prompt word is constructed based on the preset binning decision knowledge, the semantic information of risk characteristic variables, and the binning characteristic information.

[0057] In this embodiment, to enable the language model to perform constrained analysis and judgment for the current binning method, please refer to [link to documentation]. Figure 2 In sections 25-27, the intelligent agent constructs a first prompt word based on preset binning decision knowledge, semantic information of risk feature variables, and binning feature information, and uses it as a structured or semi-structured text input language model.

[0058] Please refer to Figure 2 In step 22, the semantic information of the risk characteristic variables can be loaded from a pre-built variable dictionary. This loading can be done before the first prompt word is constructed, and this embodiment does not impose any restrictions on the timing of this loading. The variable dictionary can be stored in a structured file format, such as JSON or CSV, and maintain at least one of the following types of fields for each risk characteristic variable: variable name, variable meaning description, value range description, whether it is a count or proportional variable, and variable generation method, etc. By introducing the semantic information of the variables, the language model can distinguish the meaning of the variables at the actual semantic level, avoiding abstract judgments based solely on numerical distribution, thereby reducing the probability of semantically unreasonable binning.

[0059] Binning decision knowledge is used to provide stable external constraints and references to the language model, which may include at least one of the following: (1) pre-organized binning cases to describe the binning structure forms that have been verified to be effective in historical data or other scenarios; (2) binning processing logic defined based on the experience of experts in the risk control field, such as trend pattern constraint rules (monotonic, U-shaped or inverted U-shaped distribution judgment criteria), abnormal interval identification logic, high-discrimination interval priority retention principle, etc.; (3) binning rationality statistical standards corresponding to the current application scenario, such as IV threshold range, KS lower limit, binning number range and sample stability requirements, etc.

[0060] By integrating binning decision knowledge, semantic information of risk characteristic variables, and binning characteristic information into the first prompt word, a clear decision boundary can be constructed for the model without changing the language model parameters. This allows the model to consider statistical effects, semantic consistency of variables, and empirical constraints when analyzing binning results, thereby improving the stability and deployability of the output results.

[0061] In S106, the first prompt word is input into the language model so that the language model can perform binning rationality analysis. If the current binning method is determined to be unreasonable, the binning parameters for the next binning process are output. Alternatively, if the current binning method is determined to be reasonable, risk control rules for risk feature variables are output. The binning parameters are determined based on binning decision knowledge and semantic information of risk feature variables, and the risk control rules are determined based on semantic information of risk feature variables and binning feature information.

[0062] In this embodiment, after receiving the first prompt word, the language model performs a binning rationality analysis and reasoning process on the current binning method. This reasoning process essentially verifies the consistency and validity of binning feature information under the constraints of semantic information constrained by binning decision knowledge and risk characteristic variables.

[0063] When the language model determines that the current binning method does not meet the constraints, such as insufficient bin size, inadequate discrimination, mismatch between trend pattern and variable semantics, or statistical indicators below a preset threshold, it outputs binning parameters for the next binning process. These binning parameters may include adjustments to the number of bins, boundary positions of merged or split intervals, modifications to the lower limit of the sample size, or adjustments to the trend constraint strategy. The determination process comprehensively considers experience from binning decision-making knowledge and the meaning of the direction of value changes in the variable semantic information, thereby avoiding interval structures that conflict with the actual meaning of the variables.

[0064] When the language model determines that the current binning method is reasonable, it further generates corresponding risk control rules based on the semantic information of the risk feature variable and the binning feature information. Risk control rules can be described in the form of conditional expressions, for example, mapping binning intervals to judgment conditions, and combining the risk label distribution to determine the judgment rules for high-risk or low-risk intervals, thus forming risk control rules.

[0065] Through the above mechanism, this embodiment maintains the automation of the binning process while introducing a reasoning and verification process based on binning decision knowledge constraints, so that the final output binning parameters or risk control rules have statistical validity, semantic consistency and structural stability, thereby reducing the cost of manual post-processing and improving the usability of rules.

[0066] In some possible implementations, in order to further standardize the reasoning process of the language model and prevent its output from going out of control, the mining agent may also introduce iteration number information and predefined formats of risk control rules when constructing the first prompt word.

[0067] The iteration count information is used to characterize the number of binning operations performed for a specific risk feature variable and the maximum allowed number of iterations. For example, the iteration count information may include the current count value and the maximum threshold, both embedded in the first prompt word as structured fields or natural language descriptions. After receiving the first prompt word, the language model can determine whether further adjustments to the binning method are still allowed based on this iteration count information, thereby adaptively controlling the output content during inference.

[0068] The predefined format of risk control rules is used to define the structural specifications that the language model should follow when outputting risk control rules. For example, the predefined format can include the rule's constituent fields, field order, and field types. For instance, it may require that the rule at least include feature variable identifiers, decision operators, thresholds, and risk labels, organized in a preset structured format, such as a key-value pair structure or a JSON structure. By explicitly specifying format constraints in the first prompt, unstructured or inconsistently formatted text content output by the language model can be avoided, facilitating subsequent automatic parsing and deployment by the system.

[0069] Under the above constraint mechanism, the output process of the language model can be limited to one of the following three mutually exclusive cases: When the language model determines that the current binning method is unreasonable based on binning decision knowledge, semantic information of risk feature variables, and binning feature information, and the iteration count information indicates that the maximum number of iterations has not yet been reached, the language model outputs the binning parameters and reasons for binning for the next binning process according to the format constraints in the prompt words. Binning parameters include, for example, the new number of bins, the minimum sample size threshold, or the merging threshold condition, so that the mining agent can re-execute the binning task. By simultaneously outputting the reasons for binning, the mining agent can verify the rationality of the language model's output, reducing the risk of erroneous parameters being directly used in the next round of binning; it also facilitates tracing the binning adjustment path during system logs or debugging, improving the auditability and maintainability of the binning optimization process.

[0070] When the language model determines that the current binning method is unreasonable and the iteration count information indicates that the maximum number of iterations has been reached, the language model outputs a prompt message indicating that the risk feature variable does not have the ability to generate stable risk control rules. This causes the mining agent to terminate the subsequent binning optimization process for the variable, thus avoiding unnecessary consumption of computational resources.

[0071] When the language model determines that the current binning method is reasonable, the language model outputs risk control rules for the risk feature variable in a predefined format. These risk control rules can be directly parsed and stored by the mining agent, or used in subsequent rule evaluation, combination and deployment processes.

[0072] By introducing iteration count information and predefined rule formats into the prompt words, the free generation process of the language model can be transformed into a constrained reasoning and parameter output process, thereby improving the determinism and controllability of the binning optimization process and reducing the impact of abnormal outputs on the stability of the risk control rule mining process.

[0073] For example, for the risk characteristic variable "transaction amount in the past 30 days", the mining agent loads the first type of structured prompt template P1 and fills the corresponding slot in the first type of structured prompt template P1 with the following content, thereby generating the first prompt word: (1) Basic information about variables.

[0074] Variable name: Transaction amount in the last 30 days; Meaning in Chinese: The cumulative transaction amount of the account in the last 30 days; Value range: [0, 200000]; Missing rate: 0.7%.

[0075] (2) Statistical indicator data: IV: 0.42; KS: 0.36; Lift of each compartment: 0.45, 0.62, 0.81, 1.05, 1.32, 1.78, 2.3, 2.95, 3.60, 4.20; Bad Rate for each compartment: 0.6%, 0.9%, 1.2%, 1.8%, 2.4%, 3.1%, 4.5%, 6.8%, 9.2%, 12.6%.

[0076] (3) Knowledge of containerization decision-making: Monotonic variables: When risk increases monotonically with the increase of the variable, the degree of risk clustering in the tail interval should be the focus; U-shaped variables: High-risk intervals at both ends should be retained to avoid excessive merging; High-discrimination recognition logic: When the tail lift is continuously greater than 2 and the badRate gradient is obvious, it has the potential to generate rules.

[0077] (4) Reasoning guidance instructions: Iteration trigger condition: If the current binning method is unreasonable and the maximum number of iterations has not been reached, output the new binning parameters and the reason for binning; execute continuously for a maximum of 3 times; finally output the risk control rule (if the binning method is reasonable) or skip the variable.

[0078] Recommended number of bins: If the risk trend is unclear or the differentiation is insufficient, it is recommended to increase the number of bins; if the trend is monotonous and the high-risk range is stable, output the rule threshold; Recommended candidate set of bin numbers: {20, 50, 100}.

[0079] (5) Dynamic constraint parameters: current number of bins: 10; maximum number of iterations: 3; minimum sample size per bin: 2000.

[0080] (6) Output mandatory specifications (output in JSON format): Field: exitFlag (whether to terminate the iteration); Field: bin (number of bins to be split next, optional); Field: operator (operator in the risk control rule, optional); Field: threshold (threshold in the risk control rule, optional).

[0081] Under the constraint of the first cue word, the language model performs binning rationality analysis.

[0082] In one scenario, the language model determines that further binning is needed and outputs the binning parameters in the following JSON format: In one scenario, the language model determines that further binning is unnecessary and outputs the risk control rules in the following JSON format: Template P1 encodes binning decision knowledge, semantic information of risk characteristic variables, and binning feature information into structured natural language constraint inputs. This allows the language model to complete the following within a fixed inference space: determine whether further binning is needed (based on trend clarity and discriminative potential); if iteration is needed, recommend the next reasonable number of bins (e.g., 20 / 50 / 100); if no iteration is needed, output risk control rules; and the output strictly follows a predefined format. This transforms the binning optimization process, which originally relied on human experience, into a configurable, reproducible, and constrained automated inference process, enabling the language model to make binning decisions under constraints, rather than through open-ended text generation.

[0083] In this embodiment, the binning process is a dynamic iterative optimization process driven by a language model. The core of this process lies in leveraging the semantic understanding and logical reasoning capabilities of the language model to transform binning feature information into risk control rules. The language model not only evaluates the statistical indicators of binning (such as IV and KS), but more importantly, it verifies the business rationality of the binning structure by combining binning decision knowledge. When binning is determined to be unreasonable, the model outputs not only new parameters, but also the analysis results based on the conflict between the current data distribution and business semantics; when it is determined to be reasonable, the model executes the rule generation step, that is, mapping the discretized binning intervals into judgment conditions (operators + thresholds) with clear semantic boundaries. This process achieves an intelligent leap from statistical data distribution to interpretable business rules, solving the technical problem of the lack of semantic coherence in rules generated by traditional machine learning methods.

[0084] In some embodiments, after generating corresponding risk control rules for a certain risk characteristic variable through the binning process described above, these rules can be used as basic risk control rules to further construct derived risk control rules. The risk control rule corresponding to any risk characteristic variable includes judgment conditions for the risk characteristic variable, and these judgment conditions include operators and thresholds.

[0085] Please see Figure 3 In sections 31-33, for any risk feature variable that has generated risk control rules, the mining agent can construct a second prompt word based on preset rule-derived knowledge, the type and value range of the risk feature variable, and the risk control rule corresponding to the risk feature variable. Then, the second prompt word is input into the language model so that the language model generates at least one derived risk control rule.

[0086] Among these, rule-derived knowledge describes the knowledge governing the choice of derivative direction when expanding thresholds. For example, rule-derived knowledge can be predefined as follows: when the operator is "greater than or equal to", the derivative direction expands towards a larger value; when the operator is "less than or equal to", the derivative direction expands towards a smaller value; and when the operator is an interval type, it can expand towards either the outer or inner edge of the interval. This approach ensures that the threshold expansion direction aligns with the risk implications of the basic risk control rules.

[0087] Furthermore, the derived knowledge of rules can also define the derived step size calculation knowledge for threshold expansion, such as increasing or decreasing according to a fixed numerical step size, a percentage of the variable's value range, or the bin width. When constructing the second cue word, the mining agent can provide the derived step size calculation knowledge along with the type (continuous or discrete) and value range of the risk feature variable to the language model, enabling it to generate derived thresholds within the value space of the risk feature variable.

[0088] Under this mechanism, the derived risk control rules generated by the language model satisfy the following constraints: (1) At the structural level, each derivative risk control rule includes risk characteristic variables, operators and derivative thresholds. By uniformly limiting the rule structure, it can be ensured that the derivative rules are consistent with the basic risk control rules in terms of syntax and field composition, which facilitates unified parsing, storage and execution in the future.

[0089] (2) At the numerical generation level, the derived threshold is not arbitrarily given by the language model, but is obtained by starting from the original threshold in the basic risk control rules, expanding along the preset derivative direction and according to the preset derivative step size, while requiring the obtained derived threshold to be within the value range of the corresponding risk characteristic variable.

[0090] For example, for continuous variables, values ​​can be increased or decreased according to a fixed numerical step size or a range of values; for discrete variables, values ​​can be expanded discretely according to adjacent selectable values. This numerical constraint mechanism can avoid generating thresholds that exceed the variable's defined range or have no practical meaning.

[0091] (3) At the directional control level, the derived direction is determined by the derived direction selection knowledge defined in the rule-derived knowledge and the operators in the risk control rules. This directional constraint can ensure that the derived rules are consistent with the basic risk control rules in terms of risk meaning, and avoid the situation where the risk level changes in the opposite direction.

[0092] (4) At the step size control level, the derivative step size of the derivative threshold is determined by the derivative step size calculation knowledge defined in the rule derivative knowledge, combined with the type and value range of the risk characteristic variable. For example, for continuous variables with a large value range, a larger numerical step size can be used; for variables with a small value range or low dispersion, a smaller step size or expansion based on an enumeration set is used. In this way, a moderate set of derivative rules can be generated while ensuring that the rule gradient is reasonable.

[0093] Through the aforementioned multi-layered constraints, the language model is confined to a defined search space when generating derived risk control rules, thereby avoiding disordered expansion or semantic deviation and improving the standardization, controllability, and feasibility of subsequent deployment of the derived rule set. It enables the automatic generation of a set of derived risk control rules with gradient variations from existing risk control rules to cover wider or finer-grained value ranges, thus improving the expressive power of the rule set under different risk intensity ranges while maintaining the consistency and controllability of the rule structure, facilitating subsequent rule selection, combination, and deployment.

[0094] In some exemplary implementations, the mining agent can load a second type of structured prompt template P2 to guide the language model in the process of generating risk control rules.

[0095] The intelligent agent can construct a second prompt word by combining rule-derived experience in the risk control field, basic risk control rules, type information (continuous or discrete) and value range information of risk characteristic variables, and a second type of structured prompt template P2. This second prompt word is then input into a language model, enabling the language model to generate candidate derived risk control rules under the constraints of the rule-derived experience. The rule-derived experience includes, but is not limited to: rules for determining the threshold expansion direction, the calculation logic for the derived step size, and business rationality constraints.

[0096] In this implementation, the language model satisfies at least the following constraints when generating candidate derived risk control rules: 1) The derivative direction is determined by the operator in the basic risk control rules. When the operator is "greater than" or "greater than or equal to", the threshold is expanded in the direction of increasing value of the risk characteristic variable; when the operator is "less than" or "less than or equal to", the threshold is expanded in the direction of decreasing value.

[0097] 2) The derived step size is automatically determined based on the type and value range of the risk characteristic variable. When the risk characteristic variable is a continuous variable, the step size is calculated based on a preset ratio or a fixed numerical interval; when the risk characteristic variable is a discrete variable, the step size is determined based on adjacent discrete values ​​or mapping coding intervals.

[0098] 3) All generated derivative thresholds are limited to the valid range of risk characteristic variables to avoid generating invalid rules that exceed the scope of business definition.

[0099] 4) Each derived risk control rule output is represented in a unified rule string format, which is "variable name operator threshold", and is sorted and output according to the threshold size.

[0100] In this way, the intelligent agent can embed the rule-derived experience accumulated by risk control experts in long-term practice into the second prompt word, so that the language model can perform rule-derived reasoning in a controlled reasoning space, thereby achieving the repeatability, constrainability and semantic consistency of the rule generation process, and avoiding the language model from outputting free text content that does not conform to actual logic.

[0101] For example, suppose a certain risk characteristic variable is "user age", which is a continuous numerical variable with a value range of 18 to 70. The basic risk control rule determined by the aforementioned binning process is: user age > 30.

[0102] In this embodiment, the rule-derived knowledge is predefined as follows: when the operator is "greater than" or "greater than or equal to", the derivation direction is the direction of increasing value; when the variable type is a continuous variable and the value range is less than 100, the derivation step size is 5 or calculated proportionally based on the length of the value interval; the derivation threshold shall not exceed the maximum value of the variable, which is 70.

[0103] The language model performs reasoning under the constraints of the knowledge derived from the above rules, determines the direction of derivation to the right (i.e., the direction greater than 30), and performs threshold expansion based on a derivation step size of 5, generating the following derived risk control rules in sequence: user age > 35; user age > 40; user age > 45; user age > 50; user age > 55; user age > 60; user age > 65.

[0104] In some embodiments, the mining agent can further combine rules across variables to form combined risk control rules that can describe the joint constraints of multidimensional risk characteristics. For this, please refer to [link to relevant documentation]. Figure 4 In sections 41-43, the intelligent agent mines the semantic information of the pre-set rule combination knowledge, all currently generated risk control rules, and various risk characteristic variables to construct a third prompt word and input it into the language model so that the language model can generate at least one combined risk control rule.

[0105] Rule combination knowledge describes "which rules can be combined, how to combine them, and whether the combination is reasonable." This can include, but is not limited to, the following: allowed logical connector types (such as AND, OR, NOT), the maximum number of variables in the combined rules, the conditions for determining conflicting rules (such as mutually exclusive threshold intervals, and logical rules not being simultaneously valid), combination priority strategies (such as prioritizing high-discrimination variables and high-risk indicator variables), and the upper limit of the complexity after combination. This rule combination knowledge avoids unconstrained enumeration of rule combinations, thereby preventing the exponential expansion of the number of rules and the generation of semantically inconsistent combination results.

[0106] The semantic information of risk characteristic variables is used to describe the semantic roles and relationships of different variables in the risk formation mechanism, such as whether they belong to the same risk dimension, whether they reflect different attributes of the same subject, and whether they have causal or statistically strong correlations. The data mining agent can load this semantic information from a pre-built variable dictionary and use it as one of the criteria for judging the feasibility of combinations. For example, when both variables describe a user's solvency or behavioral stability, their combination is considered highly reasonable; while when the two variables describe unrelated system states and personal attributes respectively, their combination priority can be reduced or their direct combination can be prohibited.

[0107] When constructing the third prompt word, the mining agent encodes the rule combination knowledge into a natural language description in the form of binding instructions, provides the generated risk control rules in the form of a structured list, and embeds the semantic information of each risk feature variable into the prompt content, thereby limiting the language model to reasoning and generating only within the legal combination space.

[0108] Under the constraint of the third cue word, the language model performs semantic consistency analysis and logical satisfiability analysis on candidate rules, outputting at least one combined risk control rule. A combined risk control rule is composed of at least two univariate risk control rules combined through logical connectors, such as using an "AND" relation to construct joint triggering conditions or using an "OR" relation to construct alternative triggering conditions. Its combination logic is determined based on the semantic relationships between variables reflected by the semantic information of each risk characteristic variable, as well as knowledge of rule combination.

[0109] Through the above mechanism, multi-variable combination risk control rules that are semantically reasonable, logically executable, and engineering-deployable can be automatically generated without introducing manual enumeration and manual verification. This improves the risk control rule system's ability to identify complex risk patterns and reduces the degree of human involvement and the probability of errors in the design process of combination rules.

[0110] For example, a third prompt word can be used to guide the language model to output the combined reasons for each risk control rule combination, which helps to improve the interpretability and auditability of the combined rules, facilitates manual review and compliance checks, supports subsequent rule optimization and effect evaluation, and reduces the risk of rule misuse.

[0111] In one exemplary embodiment, after obtaining the basic risk control rules obtained through the binning process and / or the derived risk control rules obtained through the rule derivation process, the mining agent further performs rule combination processing to construct combined risk control rules that can simultaneously describe multiple risk characteristics. To this end, the mining agent can load a third type of structured prompt template P3 to construct a third prompt word for rule combination reasoning.

[0112] In this embodiment, template P3 includes at least the following information slots: (1) A list of rules has been generated to describe the risk characteristic variables, operators and thresholds corresponding to each risk control rule.

[0113] (2) The semantic information of risk characteristic variables is used to characterize the meaning of variables and the semantic relationship between variables.

[0114] (3) Rule combination knowledge, used to describe the types of combination logic that are allowed or preferred, such as logical AND, logical OR, mutually exclusive combination, hierarchical combination, etc.; it also includes the following empirical information: preferentially combine rules corresponding to variables that have semantic relationships, avoid combining rules with semantic conflicts or weak statistical correlations, limit the number of conditions of combination rules to not exceed the preset upper limit, and preferentially combine rules that can all point to the same risk type, etc.

[0115] (4) Combination constraints, such as the maximum number of conditions in the combined rules, the minimum sample coverage requirement of the combined rules, and the restriction rules to avoid generating semantic contradictions.

[0116] (5) Output specifications, used to define the expression format and logical connection method of combined risk control rules.

[0117] The intelligent agent fills the template P3 with semantic information of risk rules, risk feature variables and rule combination knowledge, generates a third prompt word, and inputs the third prompt word into the language model.

[0118] The language model performs rule combination reasoning under the constraint of the third cue word. For example, it selects rule pairs or rule sets that are composable based on the semantic relationship between risk feature variables; determines the logical connection between each rule based on rule combination knowledge; and performs consistency checks on the combined rules to avoid generating mutually exclusive, semantically conflicting or difficult-to-interpret combination results.

[0119] The language model outputs at least one combined risk control rule, which is formed by combining two or more pre-generated risk control rules through logical connectors, such as logical AND, OR, or OR. This approach allows for the construction of combined rules that characterize more complex risk patterns while maintaining the interpretability of individual rules. This improves the comprehensive identification capability of risk control rules for multi-dimensional risk characteristics, while avoiding rule redundancy or semantic conflicts caused by unconstrained combinations.

[0120] For example, all generated single-variable risk control rules include: Rule R1: User age > 50; Rule R2: Number of overdue payments in the past 6 months ≥ 2; Rule R3: Average account balance in the past 3 months < 1000 yuan.

[0121] The language model performs inference analysis under the constraint of the third prompt word, and judges that the rules R1, R2 and R3 come from different risk dimensions and have no logical conflict, which is in line with the combination priority strategy. Therefore, the following combination risk control rules are generated as output results: (1) Combination rule CR1: User age > 50 AND number of overdue payments in the past 6 months ≥ 2; (2) Combination rule CR2: User age > 50 AND number of overdue payments in the past 6 months ≥ 2 AND average account balance in the past 3 months < 1000. Among them, CR1 is used to characterize the medium-risk group of "middle-aged and elderly users with obvious historical default behavior", and CR2 is used to further characterize the higher-risk group of "accompanied by insufficient asset capacity".

[0122] It is understood that the execution order of the rule combination process and the rule derivation process is not limited. For example, if the risk control rules obtained from the above binning process are used as the basic risk control rules, rule combination processing can be performed first to obtain combined risk control rules based on the basic risk control rules, and then rule derivation processing can be performed on the basic risk control rules and / or combined risk control rules; alternatively, rule derivation processing can be performed first to obtain derived risk control rules based on the basic risk control rules, and then rule combination processing can be performed on the basic risk control rules and / or derived risk control rules. Derivation and combination can also be performed in parallel, and the result sets can be merged at the end. This specification does not limit this in the embodiments. In some embodiments, please refer to... Figure 5 In sections 51-55, the mining agent can input the sample set and the currently generated risk control rules into the rule evaluation tool. This allows the rule evaluation tool to quantitatively assess the actual distinguishing ability and stability of each risk control rule on the sample set, obtaining and returning the evaluation index values ​​for each risk control rule. The rule evaluation tool can be an evaluation component based on statistical analysis or model simulation. It compares and analyzes the sample subsets that meet the rule conditions with those that do not, outputting the evaluation index values ​​corresponding to each risk control rule, such as hit rate, bad sample rate, coverage, information gain, KS value, IV value, and stability index.

[0123] In this embodiment, the generated risk control rules may include any one or more of the following: basic risk control rules obtained through the binning process, combined risk control rules obtained through the rule combination process, and derived risk control rules obtained through the rule derivation process.

[0124] After obtaining the evaluation index values ​​corresponding to each risk control rule, the mining agent constructs a fourth prompt word based on pre-set risk control rule screening knowledge, multiple generated risk control rules, the evaluation index values ​​of each risk control rule, and the semantic information of the risk feature variables involved in the rules. The risk control rule screening knowledge can include evaluation index threshold requirements, such as minimum hit rate, minimum IV value, maximum false positive rate, minimum sample coverage, etc., as well as engineering constraints such as upper limits on rule complexity and the number of rules, and predefined screening experience as semantic rationality constraints. The semantic information of the risk feature variables is used to characterize the actual meaning, value range, statistical characteristics, and semantic relationships between variables.

[0125] By inputting the fourth prompt word into the language model, the model can jointly analyze the statistical performance and semantic rationality of each risk control rule. On the one hand, the language model analyzes the evaluation index values ​​of each risk control rule based on the threshold requirements of the evaluation indexes included in the knowledge selection of the risk control rules, thereby filtering out risk control rules with insufficient discriminative power, low sample support, or poor stability. On the other hand, the language model, based on the semantic rationality constraints of the knowledge selection of the risk control rules and the semantic information of the risk characteristic variables in each risk control rule, determines whether the risk control rule conforms to the actual physical or business meaning of the variables, avoiding the retention of risk control rules that are logically contradictory, semantically unreasonable, or difficult to interpret.

[0126] Finally, the language model outputs the risk control rule selection results. These results include risk control rules selected from all generated risk control rules based on a comprehensive analysis of the evaluation index values ​​and semantic information corresponding to each rule. These rules simultaneously meet the evaluation index threshold requirements and semantic rationality constraints included in the risk control rule selection knowledge. This approach ensures the statistical validity of the rules while improving the interpretability and consistency of the rule set at the semantic level, thereby reducing the workload of subsequent manual review and correction, and enhancing the stability and usability of the final deployed rule set.

[0127] In some embodiments, after obtaining multiple risk control rules and their corresponding evaluation index values, the mining agent can load a fourth type of structured prompt template P4 to construct a fourth prompt word for rule filtering.

[0128] In this embodiment, template P4 includes at least the following information slots: (1) Risk control rule list, which describes the risk characteristic variables, operators and thresholds corresponding to each risk control rule.

[0129] (2) Performance index data, used to describe the evaluation index values ​​of each risk control rule on the sample set, including lift, support and optional false positive rate, hit rate and other indicators.

[0130] (3) The semantic information of risk characteristic variables is used to characterize the actual meaning of each variable and its role in risk assessment.

[0131] (4) Evaluation indicator threshold requirements, including threshold parameters such as min_lift, min_support, and max_false_positive_rate. min_lift is used to represent the minimum lift threshold of a rule. When the lift corresponding to a risk control rule is less than min_lift, it indicates that the risk control rule has insufficient ability to distinguish risk samples and does not have a significant risk indication role, and therefore it is not retained.

[0132] `min_support` represents the minimum support threshold for a rule. When the support of a risk control rule is less than `min_support`, it indicates that the rule only covers a very small number of samples, the statistical results are easily affected by random factors, and the stability is poor. Even if its lift is high, it may not have practical deployment value, so it should not be retained or its priority should be reduced.

[0133] `max_false_positive_rate` represents the maximum false positive rate threshold allowed by a rule. When the false positive rate of a risk control rule is higher than `max_false_positive_rate`, it indicates that the rule may significantly interfere with normal samples, easily introducing excessive invalid interception or review costs, and is therefore judged as not meeting the engineering constraints.

[0134] (4) Reasoning guidance logic is used to indicate the threshold requirements of the language model comprehensive evaluation index and the semantic information of the variables, and to make usability judgment on the rules in combination with semantic rationality constraint information (such as business common sense).

[0135] For example, reasoning-guided logic includes the following reasoning tasks: Reasoning Task 1: Compare the lift and support indices of each rule with min_lift and min_support respectively to determine whether they meet the basic requirements of statistical validity and sample stability.

[0136] Inference Task 2: Compare the false positive rate with the maximum false positive rate to determine whether it meets the acceptable risk control accuracy requirements.

[0137] Reasoning Task 3: Combining the semantic information of the risk characteristic variables involved in the rules, determine whether the risk orientation of the rules conforms to the actual meaning of the variables and common sense logic. For example, whether an increase in the value of a variable reasonably corresponds to an increase or decrease in risk, and whether there are obvious semantic conflicts or situations that are difficult to explain.

[0138] Reasoning Task 4: Only when the risk control rule satisfies both the above statistical indicator constraints and semantic reasonableness judgment will the rule be output to the rule filtering results, and an explanation based on the specific indicator value and semantic analysis conclusion will be given in the reason field.

[0139] (5) Output mandatory specification, which is used to restrict the output format of the language model to conform to the predefined JSON format and to include at least the ruleList field and the reason field.

[0140] The mining agent fills the P4 template with the above information, generates a fourth prompt word, and inputs this fourth prompt word into the language model. The language model then performs the rule evaluation reasoning process under the constraint of the fourth prompt word.

[0141] During the output phase, the language model outputs only the set of rules that meet the evaluation criteria and generates risk control rule filtering results according to the predefined JSON format. The ruleList field is used to list the retained risk control rules, and the reason field is used to give an explanation of why the corresponding rule is retained or removed. For example: "This rule has lift=2.1 and support=0.012, which are higher than the minimum coverage threshold, and the semantics of the variables are consistent with the direction of risk increase, so it is retained."

[0142] By using the above methods, the rule selection and interpretation process, which originally relied on human experience, can be transformed into an automated reasoning process constrained by structured prompt templates. This allows the language model to perform rule evaluation and selection under controlled conditions, thereby improving the consistency, interpretability, and engineering reusability of the rule selection results and avoiding the model outputting free text conclusions that do not meet engineering requirements.

[0143] In some embodiments, please refer to Figure 6 The risk control rule selection results output by the language model can constitute the first candidate risk control rule set. Each risk control rule in the first candidate risk control rule set is associated with a confidence score, which characterizes the overall credibility of the language model's assessment of the rule's effectiveness under the current sample set and semantic constraints. The confidence score can be calculated or predicted by the language model during the inference process based on the evaluation index value corresponding to the risk control rule, semantic information, and the conditions satisfying the risk control rule selection knowledge, and is output as a quantitative representation of the rule's reliability.

[0144] The intelligent agent can also invoke a machine learning-based risk control rule mining module to perform automatic rule mining processing on the same sample set to obtain a second candidate risk control rule set. The machine learning-based risk control rule mining module includes at least one of the following: decision tree model, association rule mining model, gradient boosting model, or other rule learning model. Each risk control rule in the second candidate risk control rule set carries a confidence score, which is used to characterize the statistical reliability or contribution of the rule.

[0145] The intelligent agent can input the sample set and the second candidate risk control rule set into the rule evaluation tool, so that the rule evaluation tool can quantitatively evaluate the actual distinguishing ability and stability of each risk control rule in the second candidate risk control rule set on the sample set, obtain the evaluation index value of each risk control rule and return it.

[0146] The intelligent agent can perform rule-level comparisons between the first and second candidate risk control rule sets to determine the overlap and complementarity between the two sets of rules. Specifically: On the one hand, rules that exist simultaneously in both the first and second candidate risk control rule sets, and whose confidence levels in both sets exceed a pre-set confidence threshold, are identified as the first type of risk control rules. These rules are validated through both semantic and statistical evaluation using a language model and statistical learning using a machine learning model, demonstrating high reliability and stability, making them suitable for priority deployment or as core rules.

[0147] On the other hand, rules that exist only in the first candidate risk control rule set or only in the second candidate risk control rule set, and whose corresponding evaluation indicators meet preset conditions (e.g., IV not lower than the threshold, lift not lower than the threshold, coverage not lower than the threshold, etc.), are identified as the second type of risk control rules. Although this type of rule does not appear in both rule generation mechanisms, it still meets the set usable standards in terms of statistical performance or semantic reasonableness, and is therefore retained.

[0148] Other risk control rules that do not fall under Category I or Category II risk control rules can be included as alternatives in subsequent manual review or gray-scale verification processes.

[0149] Through the rule set fusion and hierarchical screening mechanism described above, the semantic reasoning ability of language models and the statistical learning ability of machine learning models can be comprehensively utilized under the same rule mining framework. This not only improves the overall reliability of the risk control rule set, but also avoids the bias or information omission caused by a single method, thereby enhancing the stability and interpretability of the final rule system in actual risk identification scenarios.

[0150] It should be noted that the transaction amount, user age, etc. mentioned above are merely illustrative examples of risk characteristic variables. The technical solution in this specification does not rely on the specific meaning of business fields, but rather on the combination of variable semantic information and statistical characteristics.

[0151] This embodiment maps the original data column names to natural language semantic descriptions through a variable dictionary, solving the problem of language models' misunderstanding of domain-specific abbreviations; it transforms binning decision knowledge into natural language constraint instructions, limiting the search space of the language model and preventing illusions; and it constrains the language model's output of structured data through a predefined JSON format, achieving an automated process loop.

[0152] Therefore, regardless of whether the application scenario is financial risk control, medical diagnosis or industrial quality inspection, as long as the feature variable-label data structure and the corresponding domain decision knowledge are available, the intelligent mining architecture described in this manual can be reused.

[0153] The various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, the arbitrary combination of various technical features in the above embodiments is also within the scope of this specification.

[0154] Figure 7 This is a schematic structural diagram of a device provided in an exemplary embodiment. For example... Figure 7 As shown, device 700 mainly consists of a communication interface 702, a user interface 704, a processor 706, and a data storage 708. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 710. Communication interface 702 enables device 700 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, communication interface 702 may include an antenna and other processing devices for wireless communication with a radio access network or access point. Furthermore, communication interface 702 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide area wireless interface (e.g., WiMAX or LTE). Of course, communication interface 702 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. Communication interface 702 may also include multiple physical communication interfaces, such as Wi-Fi interfaces, Bluetooth interfaces, and wide area wireless interfaces.

[0155] User interface 704 includes receiving user input and providing output to the user. Therefore, user interface 704 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 704 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 704 may include software, circuitry, or other forms of logic capable of transmitting and receiving data from external user input / output devices. Additionally or alternatively, device 700 may support remote access from other devices via communication interface 702 or another physical interface (not shown). User interface 704 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 704 may also be configured as a display device for rendering or displaying text fragments.

[0156] Processor 706 may contain one or more general-purpose processors and / or special-purpose processors.

[0157] Data storage 708 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 706. Data storage 708 may include removable and non-removable components.

[0158] Processor 706 is capable of executing program instructions 718 (e.g., compiled or uncompiled program logic and / or machine code) stored in data storage 708 to perform the various functions described herein. Data storage 708 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 700, enable device 700 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Processor 706 executing program instructions 718 may result in processor 706 using data 712.

[0159] For example, program instructions 718 may include an operating system 722 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 700 and one or more applications 720 (e.g., a browser, social application, or game application). Similarly, data 712 may include operating system data 716 and application data 714. Operating system data 716 is primarily accessible to the operating system 722, while application data 714 is primarily accessible to one or more applications 720. Application data 714 may reside in a file system visible or hidden from the user of device 700.

[0160] Application 720 can communicate with operating system 722 through one or more application programming interfaces (APIs). These APIs help application 720 read and / or write application data 714, transmit or receive information via communication interface 702, receive or display information on user interface 704, etc.

[0161] In some terminology, application 720 may be simply referred to as "app". Furthermore, application 720 can be downloaded to device 700 through one or more online app stores or app markets. However, applications can also be installed on device 700 in other ways, such as through a web browser or a physical interface on device 700 (e.g., a USB port).

[0162] In some embodiments, the risk control rule mining device can be applied to, for example... Figure 7 The device shown is used to implement the technical solution described in this specification. The risk control rule mining device may include: The sample acquisition module is used to acquire a sample set containing multiple samples. Each sample includes the value of at least one risk characteristic variable and is associated with a risk label.

[0163] The binning module is used to bin each risk characteristic variable according to the binning parameters and the value of the risk characteristic variable in the sample set, and to perform statistical analysis based on the distribution of risk labels associated with samples in each bin to obtain binning characteristic information of the current binning method.

[0164] The prompt word construction module is used to construct the first prompt word based on preset binning decision knowledge, semantic information of risk feature variables, and binning feature information.

[0165] The binning analysis module is used to input the first prompt word into the language model so that the language model can perform binning rationality analysis. If the current binning method is determined to be unreasonable, it outputs the binning parameters for the next binning process; or, if the current binning method is determined to be reasonable, it outputs risk control rules for risk feature variables. The binning parameters are determined based on binning decision knowledge and semantic information of risk feature variables, and the risk control rules are determined based on semantic information of risk feature variables and binning feature information.

[0166] In one implementation, the first prompt word also includes: iteration count information and a predefined format for risk control rules.

[0167] The binning analysis module is specifically used to input the first prompt word into the language model, so that the language model outputs according to the following constraints after performing binning rationality analysis: If the current binning method is determined to be unreasonable, and the iteration count information indicates that the current iteration round has not reached the maximum iteration round, output the binning parameters and the reason for binning in the next binning process; or, When the current binning method is deemed unreasonable and the iteration count information indicates that the current iteration round has reached the maximum iteration round, a prompt message is output indicating that the risk characteristic variable cannot generate risk control rules; or, When the current binning method is deemed reasonable, output risk control rules that conform to a predefined format.

[0168] In one implementation, the risk control rules include judgment conditions for risk characteristic variables, and the judgment conditions include operators and thresholds.

[0169] The device also includes a rule derivation module, used to construct a second prompt word for any risk feature variable based on preset rule derivation knowledge, the type and value range of the risk feature variable, and the risk control rule corresponding to the risk feature variable; input the second prompt word into a language model so that the language model generates at least one derived risk control rule; wherein, the derived risk control rule includes the risk feature variable, an operator, and a derived threshold; the derived threshold is obtained by extending the threshold in the risk control rule along the derivation direction and according to the derivation step size, and the derived threshold is within the value range of the risk feature variable; the derivation direction is determined by the derivation direction selection knowledge defined in the rule derivation knowledge and the operator in the risk control rule; the derivation step size of the derived threshold is determined by the derivation step size calculation knowledge defined in the rule derivation knowledge and combined with the type and value range of the risk feature variable.

[0170] In one implementation, a rule combination module is also included, which is used to construct a third prompt word based on preset rule combination knowledge, all generated risk control rules, and semantic information of each risk feature variable; the third prompt word is input into a language model so that the language model generates at least one combined risk control rule; wherein, the combined risk control rule is formed by combining at least two generated risk control rules through logical connectors, and the combination logic is determined based on the semantic relationship between variables reflected by the semantic information of each risk feature variable and the rule combination knowledge.

[0171] In one implementation, a rule filtering module is also included, which is used to input the sample set and each generated risk control rule into the rule evaluation tool to obtain the evaluation index value of each risk control rule; construct a fourth prompt word based on preset risk control rule filtering knowledge, multiple generated risk control rules, the evaluation index value of each risk control rule, and the semantic information of risk characteristic variables in the risk control rules; input the fourth prompt word into the language model so that the language model outputs the risk control rule filtering result; wherein, the risk control rules included in the risk control rule filtering result are selected from all generated risk control rules based on a comprehensive analysis of the evaluation index value and semantic information corresponding to each risk control rule, and simultaneously meet the evaluation index threshold requirements and semantic rationality constraints included in the risk control rule filtering knowledge.

[0172] In one implementation, the results of the risk control rule screening constitute the first candidate risk control rule set, and the confidence level of each risk control rule in the first candidate risk control rule set is associated with it.

[0173] The rule filtering module is also used to call the machine learning-based risk control rule mining module to perform mining processing on the sample set to obtain a second candidate risk control rule set. Each risk control rule in the second candidate risk control rule set carries a confidence level. The first candidate risk control rule set and the second candidate risk control rule set are compared, and the first type of risk control rule and the second type of risk control rule are retained. The first type of risk control rule exists in both the first candidate risk control rule set and the second candidate risk control rule set, and the confidence level of both exceeds the preset threshold. The second type of risk control rule exists in either the first candidate risk control rule set or the second candidate risk control rule set, and the evaluation index value of the risk control rule meets the preset conditions.

[0174] In one implementation, the binning feature information includes binning structure information and binning statistics. Binning structure information includes the number of bins, the boundary values ​​corresponding to each bin, and the lower limit of the sample size contained in a single bin. Binning statistics include: variable-level statistical indicators obtained from statistical analysis of the feature variables, and / or bin-level statistical indicators obtained from statistical analysis of the binning dimensions.

[0175] The knowledge for bin splitting decisions includes at least one of the following: at least one bin splitting case, bin splitting processing logic defined based on the experience of risk control experts, and statistical indicators of the rationality of bin splitting in the current risk control scenario.

[0176] The semantic information of risk characteristic variables includes at least one of the following: name, meaning, and value range.

[0177] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more of these specifications, the functions of each module or unit can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0178] Based on the same concept as the methods described above, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor performs the steps of the method as described in any of the above embodiments by executing the executable instructions.

[0179] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0180] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0181] What those skilled in the art will understand is: In this specification, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded.

[0182] In this specification, “a,” “an,” and “the” do not specifically refer to the singular, but may also include the plural.

[0183] In this specification, ordinal numbers such as "first," "second," etc., do not necessarily indicate order; they are often used to distinguish between objects. For example, "first server" and "second server" usually refer to two servers. To differentiate between these two servers, they are described as "first server" and "second server." Of course, sometimes these two servers may be the same server.

[0184] In this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can also mean indirect receiving and sending. For example, A receiving data sent by B can be understood as A directly receiving the data sent by B, or it can be understood as A indirectly receiving the data sent by B through other entities such as C. Similarly, B sending data to A can be understood as B sending the data directly to A, or it can be understood as B indirectly sending the data to A through other entities such as C. Here, C can be one entity, or it can be two or more entities.

[0185] In this specification, unless explicitly stated otherwise, the relationships between structures can be direct or indirect. For example, when describing "A is connected to B," unless it is explicitly stated that A and B are directly connected, it should be understood that A can be directly connected to B or indirectly connected to B. Similarly, when describing "A is on top of B," unless it is explicitly stated that A is directly above B (AB is adjacent and A is above B), it should be understood that A can be directly above B or indirectly above B (AB is separated by other elements, and A is above B). And so on.

[0186] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0187] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is only one of many possible execution orders and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims.

Claims

1. A risk control rule mining method, comprising: Obtain a sample set containing multiple samples, each sample including the value of at least one risk characteristic variable and an associated risk label; For each of the aforementioned risk characteristic variables, binning is performed according to the binning parameters and the values ​​of the risk characteristic variables in the sample set. Statistical analysis is then performed based on the distribution of risk labels associated with samples in each bin to obtain binning characteristic information of the current binning method. The first prompt word is constructed based on the preset binning decision knowledge, the semantic information of the risk characteristic variables, and the binning characteristic information; The first prompt word is input into the language model so that the language model can perform binning rationality analysis. If the current binning method is determined to be unreasonable, the binning parameters for the next binning process are output. Alternatively, if the current binning method is determined to be reasonable, the risk control rules for the risk characteristic variables are output. The binning parameters are determined based on the binning decision knowledge and the semantic information of the risk characteristic variables, and the risk control rules are determined based on the semantic information of the risk characteristic variables and the binning characteristic information.

2. The method according to claim 1, wherein the first prompt word further includes: Iteration count information and predefined formats for risk control rules; The step of inputting the first prompt word into a language model to enable the language model to perform binning rationality analysis, and outputting binning parameters for the next binning process when the current binning method is determined to be unreasonable, or outputting risk control rules for the risk characteristic variables when the current binning method is determined to be reasonable, includes: The first prompt word is input into the language model, so that the language model outputs according to the following constraints after performing binning rationality analysis: If the current binning method is determined to be unreasonable, and the iteration count information indicates that the current iteration round has not reached the maximum iteration round, output the binning parameters and the reason for binning in the next binning process; or, When the current binning method is determined to be unreasonable, and the iteration count information indicates that the current iteration round has reached the maximum iteration round, a prompt message is output indicating that the risk characteristic variable cannot generate risk control rules; or, When the current binning method is determined to be reasonable, risk control rules conforming to the predefined format are output.

3. The method according to claim 1, wherein the risk control rule includes judgment conditions for risk characteristic variables, and the judgment conditions include operators and thresholds; The method further includes: For any risk characteristic variable, a second prompt word is constructed based on preset rule-derived knowledge, the type and value range of the risk characteristic variable, and the risk control rule corresponding to the risk characteristic variable; The second prompt word is input into the language model so that the language model generates at least one derivative risk control rule; The derived risk control rule includes the risk feature variable, the operator, and the derived threshold; the derived threshold is obtained by extending the threshold in the risk control rule along the derived direction and according to the derived step size, and the derived threshold is within the value range of the risk feature variable; The derived direction is determined by the derived direction selection knowledge defined in the rule-derived knowledge and the operators in the risk control rules; The derivative step size of the derived threshold is determined by the derivative step size calculation knowledge defined in the rule-derived knowledge, combined with the type and value range of the risk characteristic variable.

4. The method according to claim 1, further comprising: Based on the pre-defined rule combination knowledge, all generated risk control rules, and semantic information of each risk characteristic variable, a third prompt word is constructed; The third prompt word is input into the language model so that the language model generates at least one combined risk control rule; The combined risk control rule is formed by combining at least two of the generated risk control rules through logical connectors, and the combination logic is determined based on the semantic relationship between variables reflected by the semantic information of each risk characteristic variable and the rule combination knowledge.

5. The method according to any one of claims 1 to 4, further comprising: Input the sample set and each generated risk control rule into the rule evaluation tool to obtain the evaluation index value of each risk control rule; The fourth prompt word is constructed based on the knowledge of the preset risk control rules, multiple generated risk control rules, the evaluation index values ​​of each risk control rule, and the semantic information of the risk characteristic variables in the risk control rules. The fourth prompt word is input into the language model so that the language model outputs the risk control rule filtering results. The risk control rules included in the risk control rule screening results are selected from all generated risk control rules based on a comprehensive analysis of the evaluation index values ​​and semantic information corresponding to each risk control rule, and simultaneously meet the evaluation index threshold requirements and semantic rationality constraints included in the risk control rule screening knowledge.

6. The method according to claim 5, wherein the risk control rule screening results constitute a first candidate risk control rule set, and each risk control rule in the first candidate risk control rule set is associated with a confidence level; The method further includes: The risk control rule mining module based on machine learning is invoked to perform mining processing on the sample set to obtain a second candidate risk control rule set; each risk control rule in the second candidate risk control rule set carries a confidence level; The first candidate risk control rule set and the second candidate risk control rule set are compared, and the first type of risk control rule and the second type of risk control rule are retained; The first type of risk control rule exists in both the first candidate risk control rule set and the second candidate risk control rule set, and the confidence level of both exceeds the preset threshold. The second type of risk control rule exists in the first candidate risk control rule set or the second candidate risk control rule set, and the evaluation index value of the risk control rule meets the preset conditions.

7. The method according to claim 1, wherein the compartment feature information includes compartment structure information and compartment statistics information; The binning structure information includes the number of bins, the boundary values ​​corresponding to each bin, and the lower limit of the sample size contained in a single bin; The sorting statistics include: Variable-level statistical indicators obtained from statistical analysis of feature variables, and / or bin-level statistical indicators obtained from statistical analysis of bins; The binning decision knowledge includes at least one of the following: at least one binning case, binning processing logic defined based on the experience of risk control experts, and statistical indicators of binning rationality in the current risk control scenario; The semantic information of the risk characteristic variables includes at least one of the following: name, meaning, and value range.

8. An electronic device, comprising: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method as described in any one of claims 1-7 by executing the executable instructions.

9. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1-7.

10. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1-7.

Images