Method and system for coverage range with constrained maximization risk detection

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for risk detection. An example method can include obtaining a first subset of a plurality of risk detection rules, the first subset associated with a first coverage score; constructing a lower bound data map based on the first subset, the lower bound data map outputting an approximate coverage score for an input subset; and constructing an upper bound data map including a set of parameters based on the first subset; and generating a third subset of the plurality of risk detection rules; and responsive to the first coverage score exceeding the third coverage score, selecting a rule in the first subset for use in risk detection of a new transaction.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to systems and methods for utilizing constraint maximization for risk detection coverage. BACKGROUND

[0002] Fraud prevention and risk detection are long-standing concerns for online service providers, such as online banks, online payment systems, etc. These tasks are typically handled by risk management systems that replay rules by the book. Each rule can include multiple conditions to evaluate a transaction. A transaction can be correctly or incorrectly identified as risky (or safe) due to the rules. Correct identification can boost risk detection coverage (e.g., the ability to detect various risky transactions), but incorrect identification can lead to higher customer disruption (e.g., customer dissatisfaction due to good transactions being rejected as malicious). Among these rules, some can be created based on practical experience, while some can be provided by machine learning models such as decision trees. Often, some rules are overly aggressive, covering not only fraudulent and risky transactions, but also disrupting good transactions (e.g., false positive identification leading to customer disruption). Therefore, it is desirable to design an effective way to select rules to cover as many fraudulent / risky transactions as possible while keeping the number of disrupted transactions below a predetermined value. SUMMARY

[0003] Various embodiments of the present specification can include systems, methods, and non-transitory computer-readable media for risk detection.

[0004] According to one aspect, the method for risk detection can include: obtaining a first subset of a plurality of risk detection rules, the first subset being associated with a first coverage score and a first break score, wherein: the first coverage score indicates a number of unique historical transactions that have been correctly identified by a plurality of risk detection rules in the first subset, and the first break score indicates a number of unique historical transactions that have been incorrectly identified by a plurality of risk detection rules in the first subset; based on the first subset, constructing a lower bound data mapping that outputs an approximate coverage score for an input subset, wherein: when the input subset is the first subset, the approximate coverage score is the same as the first coverage score, and when the input subset is a second subset that is different from the first subset, the approximate coverage score is not greater than a second coverage score associated with the second subset, the second coverage score indicating a number of unique historical transactions that have been correctly identified by risk detection rules in the second subset; and based on the first subset, constructing an upper bound data mapping that includes a set of parameters, wherein: the upper bound data mapping outputs an approximate break score for an input subset of the plurality of risk detection rules, when the input subset is the first subset, the output approximate break score is the same as the first break score, and when the input subset is a second subset that is different from the first subset, the output approximate break score is not less than a second break score associated with the second subset, the second break score indicating a number of unique historical transactions that have been incorrectly identified as risk transactions by risk detection rules in the second subset; and based on the first subset, the lower bound data mapping, and the upper bound data mapping, generating a third subset of the plurality of risk detection rules based at least on: the approximate coverage score output by the lower bound data mapping corresponding to the plurality of risk detection rules as input, and the set of parameters associated with the upper bound data mapping, wherein the third subset is associated with a third coverage score indicating a number of unique historical transactions that have been correctly identified by a plurality of risk detection rules in the third subset; comparing the first coverage score with the third coverage score; and in response to the first coverage score exceeding the third coverage score, selecting rules in the first subset for use in risk detection of new transactions.

[0005] In some embodiments, the method further comprises: in response to the first coverage score not exceeding the third coverage score, replacing the first subset with the third subset as an updated first subset, wherein the first coverage score is replaced with the third coverage score of the third subset accordingly; and performing one or more iterations of a process cyclically based on the constructing step and the generating step, the process comprising: updating the lower bound data mapping based on the updated first subset; generating an updated third subset associated with an updated third coverage score based on the updated first subset and the updated lower bound data mapping; and if the exit condition is not satisfied, replacing the updated first subset with the updated third subset and replacing the updated first coverage score with the updated third coverage score.

[0006] In some embodiments, the exit condition comprises at least one of: the updated first coverage score being greater than the updated third coverage score, and a number of the one or more iterations being greater than a preset number.

[0007] In some embodiments, the lower bound data mapping comprises a submodular function.

[0008] In some embodiments, the first subset is empty.

[0009] In some embodiments, the constructing the lower bound data mapping comprises: generating a sequence by reordering the plurality of risk detection rules, wherein a risk detection rule in the first subset is placed first in the sequence; constructing a list of time subsets S i , 0≤i≤n, based on the generated sequence, wherein: n is a number of the plurality of risk detection rules, time subset S0is empty, and for a given i, wherein 1≤i≤n, time subset S i includes the i-th risk detection rule in the sequence and all risk detection rules in time subset S i-1 ; determining an approximate coverage score for each risk detection rule in the generated sequence; and determining a coverage score for a given subset of the plurality of risk detection rules as a sum of individual approximate coverage scores for each risk detection rule in the given subset.

[0010] In some embodiments, the determining the individual approximate coverage score for each risk detection rule in the sequence comprises: for the i-th risk detection rule in the sequence, determining the individual approximate coverage score based on a difference between a coverage score of the time subset S i and a coverage score of the time subset S i-1 , wherein the time subset S ia coverage score of the first subset S i-1 a coverage score of the first subset S

[0011] In some embodiments, constructing the parameter set includes determining the parameter set by: for each risk detection rule in the plurality of risk detection rules: determining a first approximate coverage score based on the lower bound data mapping; determining a first disruption score increase associated with adding the each risk detection rule to a first set of risk detection rules based on a number of unique historical transactions that have been falsely identified by the each risk detection rule; and determining a first ratio of the each risk detection rule, wherein the approximate coverage score is a numerator and the first disruption score increase is a denominator; generating a sequence by ordering the plurality of risk detection rules in descending order according to the determined first ratios of the plurality of risk detection rules; selecting a maximum number of risk detection rules for which a first total disruption score increase is not greater than the preset threshold, wherein the first total disruption score is a sum of the first disruption score increases associated with the selected risk detection rules; and determining the parameter set as an intersection of the first subset and the selected risk detection rules.

[0012] In some embodiments, the first set is determined as the first subset if the each risk detection rule is not in the first subset, or the first set is determined as the first subset excluding the each risk detection rule if the each risk detection rule is in the first subset.

[0013] In some embodiments, the generating the third subset of the plurality of risk detection rules includes: ordering the plurality of risk detection rules based on the first subset, the parameter set, and the approximate coverage scores generated by the lower bound data mapping for the plurality of risk detection rules; and selecting one or more consecutive risk detection rules from a beginning of the ordered plurality of risk detection rules as the third subset.

[0014] In some embodiments, ranking the plurality of risk detection rules comprises, for each risk detection rule in the plurality of risk detection rules: determining a second approximate coverage score based on the lower bound data mapping; determining a second set of risk detection rules as the parameter set if the each risk detection rule is not in the first subset, or as the first subset excluding the each risk detection rule if the each risk detection rule is in the first subset; determining a second increase in disruption score associated with adding the each risk detection rule to the second set of risk detection rules; and determining a second ratio of the each risk detection rule, wherein the second approximate coverage score is a numerator, and the second increase in disruption score is a denominator; generating a sequence by ranking the plurality of risk detection rules in descending order according to the second ratio of the each risk detection rule; and wherein selecting one or more consecutive risk detection rules as the third subset comprises selecting a maximum number of risk detection rules for which a second total increase in disruption score is not greater than the pre-set threshold, wherein the second total increase in disruption score is a sum of the second increase in disruption score associated with the selected risk detection rules.

[0015] According to another aspect, a system for risk detection can include one or more processors and one or more non-transitory computer-readable memories coupled to the one or more processors and configured with instructions executable by the one or more processors to cause the system to perform operations including: obtaining a first subset of a plurality of risk detection rules, the first subset being associated with a first coverage score and a first break score, wherein: the first coverage score indicates a number of unique historical transactions that have been correctly identified by the plurality of risk detection rules in the first subset, and the first break score indicates a number of unique historical transactions that have been incorrectly identified by the plurality of risk detection rules in the first subset; based on the first subset, constructing a lower bound data mapping that outputs an approximate coverage score for an input subset, wherein: when the input subset is the first subset, the approximate coverage score is the same as the first coverage score, and when the input subset is a second subset that is different from the first subset, the approximate coverage score is not greater than a second coverage score associated with the second subset, the second coverage score indicating a number of unique historical transactions that have been correctly identified by risk detection rules in the second subset; and based on the first subset, constructing an upper bound data mapping that includes a set of parameters, wherein: the upper bound data mapping outputs an approximate break score for an input subset of the plurality of risk detection rules, when the input subset is the first subset, the output approximate break score is the same as the first break score, and when the input subset is a second subset that is different from the first subset, the output approximate break score is not less than a second break score associated with the second subset, the second break score indicating a number of unique historical transactions that have been incorrectly identified as risk transactions by risk detection rules in the second subset; and generating, based at least on: the approximate coverage score output by the lower bound data mapping corresponding to the plurality of risk detection rules as input, and the set of parameters associated with the upper bound data mapping, a third subset of the plurality of risk detection rules in accordance with the first subset, the lower bound data mapping, and the upper bound data mapping, wherein the third subset is associated with a third coverage score indicating a number of unique historical transactions that have been correctly identified by the plurality of risk detection rules in the third subset; comparing the first coverage score to the third coverage score; and responsive to the first coverage score exceeding the third coverage score, selecting rules in the first subset for use in risk detection of new transactions.

[0016] According to yet another aspect, a method for selecting a subset from a set of candidates can include obtaining a first subset of a plurality of candidates, the first subset associated with a first true positive score and a first false positive score, wherein: the first true positive score indicates a gain associated with a candidate in the first subset, and the first false positive score indicates a cost associated with a candidate in the first subset; constructing a lower bound data mapping based on the first subset, the lower bound data mapping outputting an approximate true positive score for an input subset, wherein: when the input subset is the first subset, the approximate true positive score is the same as the first true positive score, and when the input subset is a second subset different from the first subset, the approximate true positive score is not greater than a second true positive score associated with the second subset, the second true positive score indicating a gain associated with a candidate in the second subset; and constructing an upper bound data mapping comprising a set of parameters based on the first subset, wherein: the upper bound data mapping outputs an approximate false positive score for an input subset of the plurality of candidates, when the input subset is the first subset, the output approximate false positive score is the same as the first false positive score, and when the input subset is the second subset different from the first subset, the output approximate false positive score is not less than a second false positive score associated with the second subset, the second false positive score indicating a gain associated with a candidate in the second subset; and generating a third subset of the plurality of candidates based on at least: the approximate true positive score output by the lower bound data mapping corresponding to the plurality of candidates as input, and the set of parameters associated with the upper bound data mapping, according to the first subset, the lower bound data mapping, and the upper bound data mapping, wherein the third subset is associated with a third true positive score, the third true positive score indicating a gain associated with a plurality of candidates in the third subset; comparing the first true positive score with the third true positive score; and responsive to the first true positive score exceeding the third true positive score, selecting a candidate in the first subset for a new transaction.

[0017] The embodiments disclosed in this specification have one or more technical effects. In some embodiments, the problem of maximizing risk detection coverage with constraints is formulated as a generalized submodular optimization and solved by an iterative process. In this way, the embodiments disclosed in this specification allow for more flexible constraints and give on the analytical or on the value oracle model (e.g., a fixed value) rather than simple classical cardinality constraints in existing schemes. For example, existing schemes can only allow a risk management system to specify a number of rules that can be selected, while the embodiments disclosed in this specification can allow a risk management system to specify a variety of constraints, such as a maximum customer disruption rate, which is more practical and meaningful. In some embodiments, variational modular approximations of the involved submodular functions (e.g., risk detection coverage and customer disruption rate) are constructed so that the optimal solution for maximizing risk detection coverage can be iteratively explored. In comparison with regular sequential search and random search, the iterative method disclosed in this specification guarantees that the best set of risk detection rules can be determined at a faster speed. In some embodiments, the submodularity of risk detection coverage and customer disruption rate is fully explored to simplify the solution search process. For example, by considering the submodularity of risk detection coverage and customer disruption rate, an upper bound approximation of customer disruption rate (e.g., in the form of a cost function) and a lower bound approximation of risk detection coverage (e.g., in the form of an objective function) are constructed. These approximations are employed to construct an iterative method, in which a better solution is guaranteed to be obtained at each iteration.

[0018] These and other features of the systems, methods, and non-transitory computer-readable media disclosed herein, as well as the methods of operation and functions of the related elements of structure and the combination of parts and economies of manufacture, will become more apparent upon consideration of the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals designate similar parts in the various figures. It is to be expressly understood, however, that the drawings are for purposes of illustration only and are not intended as a definition of the limits of the application. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 A system 100 for maximizing risk detection coverage according to some embodiments is shown.

[0020] Figure 2A An exemplary setup for maximizing risk detection coverage according to some embodiments is shown.

[0021] Figure 2B An exemplary data query system for maximizing risk detection coverage according to some embodiments is shown.

[0022] Figure 3 An exemplary method for maximizing risk detection coverage according to some embodiments is shown.

[0023] Figure 4 An exemplary diagram for maximizing risk detection coverage is shown in accordance with some embodiments.

[0024] Figure 5 An example method for risk detection is shown in accordance with various embodiments.

[0025] Figure 6 A block diagram of a computer system for risk detection is shown in accordance with some embodiments.

[0026] Figure 7 An example computing device that can implement any of the embodiments described herein is shown. DETAILED DESCRIPTION

[0027] Submodularity is an important property that naturally exists in many real-world scenarios. As a specific example, diminishing returns (or diminishing marginal utility) in economics refers to the phenomenon that the marginal benefit of any given element tends to decrease as more elements are added. This concept also applies to risk management systems, where the marginal benefit of adding a risk detection rule tends to decrease as more risk detection rules are added. Here, the marginal benefit of adding a risk detection rule can refer to the increase in risk detection coverage (as the newly added rule can discover some new risks that the existing rules missed). Similarly, the marginal “cost” of adding a risk detection rule (e.g., customer dissatisfaction associated with additional customer disruption introduced by the newly added risk detection rule) also follows the same pattern: decreasing as more risk detection rules are added.

[0028] In online systems that involve risk detection (e.g., online payment systems, online banking systems), many rules can be prepared to detect risky transactions. For example, such rules can include “if the location of the transaction is not the usual location of the user,” “if the transaction amount is greater than a certain amount and the credit level of the user is below a certain threshold,” “if the user has entered the wrong password multiple times,” and the like. These rules can be deployed to determine whether to approve a credit card application, or whether a consumption transaction is fraudulent, or whether a login is malicious. A common task for a risk management system is to determine which rule (or which subset) to invoke in order to maximize risk detection coverage while keeping the side effects (e.g., customer disruption) below a preset threshold.

[0029] For simplicity of illustration, let [n] = {1, 2,..., n} be a finite ground set (e.g., a plurality of risk detection rules), and let 2 [n]Each subset can provide a risk detection coverage (e.g., benefit) and a customer disruption rate (e.g., cost), which can be quantified based on the number of historical transactions that are correctly and incorrectly identified as risky (or safe), respectively. This description does not limit the means of quantifying coverage and cost. By representing risk detection coverage as a function g(X), where X refers to a given subset of risk detection rules, and customer disruption rate as a function f(X), the task of maximizing risk detection coverage subject to customer disruption rate constraints can be expressed as a submodular maximization problem as follows:

[0030]

[0031] where “s.t.” stands for “subject to,” and b refers to a pre-specified threshold of customer disruption. Both g(X) and f(X) can be submodular and monotonic (e.g., the value of the function does not decrease as X increases). The submodularity of the function g(X) can be expressed as not in Y, which can be interpreted as: for two given subsets X and Y, X includes fewer rules than Y, and the marginal gain (e.g., risk detection coverage boost) of adding a single rule j to the boundary of X is greater than or equal to adding j to Y. The above description also applies to f(X). The monotonicity of g(X) and f(X) means that the value of the function does not decrease as X expands (e.g., risk detection coverage and customer disruption rate do not decrease by adding new rules).

[0032] In some embodiments, for a given subset of risk detection rules, the corresponding risk detection coverage (e.g., the value of g(X)) can be determined by querying historical information. For example, the coverage can be learned based on the number of historical transactions that have been correctly identified as risky (or safe) by the rules in the given subset. Similarly, the customer disruption corresponding to the given subset can be learned based on the number of historical transactions that have been incorrectly identified as risky (or safe) by the rules in the given subset. Whether a historical transaction has been correctly or incorrectly identified as risky (or safe) can be determined based on a comparison of the identification based on the rules before the transaction occurred and a manual or machine labeling of such transactions after the transaction was performed. In some other embodiments, the risk detection coverage and customer disruption rate can be learned by other means, such as a black-box server or service (e.g., an oracle) that responds to queries. This description does not limit the means for obtaining the values of g(X) and f(X) for a given subset of rules X.

[0033] The embodiments described in this description provide an iterative way to explore the optimal subset of risk detection rules to maximize risk detection coverage subject to customer disruption rate constraints.

[0034] Figure 1A system 100 for maximizing risk detection coverage according to some embodiments is shown. The components of the system 100 presented below are intended to be illustrative. Depending on the implementation, the system 100 can include additional, fewer, or alternative components.

[0035] In some embodiments, the system 100 can include a computing system 102, a computing device 104, and a computing device 106. It should be understood that although two computing devices are shown in Figure 1 any number of computing devices can be included in the system 100. The computing system 102 can be implemented in one or more networks (e.g., enterprise networks), one or more endpoints, one or more servers (e.g., server 130), or one or more clouds. The server 130 can include hardware or software that manages access to centralized resources or services in a network. The cloud can include a cluster of servers and other devices distributed across a network.

[0036] In some embodiments, the computing system 102 can include a first acquisition component 112, a second acquisition component 114, an approximation component 116, and an optimization component 118. The computing system 102 can include other components. The computing system 102 can include one or more processors (e.g., digital processor, analog processor, digital circuitry designed to process information, central processing unit, graphics processing unit, microcontroller, or microprocessor, analog circuitry designed to process information, state machine, and / or other mechanisms for electronically processing information) and one or more memories (e.g., persistent memory, temporary memory, non-transitory computer-readable storage medium). The one or more memories can be configured with instructions that are executable by the one or more processors. The processors can be configured to perform various operations by interpreting machine-readable instructions stored in the memory. The computing system 102 can be installed with appropriate software (e.g., platform programs, etc.) and / or hardware (e.g., wired, wireless connections, etc.) to access other devices of the system 100.

[0037] In some embodiments, the computing devices 104 and 106 can be implemented on or as various devices such as mobile phones, tablets, servers, desktop computers, laptop computers, etc. The computing system 102 can communicate with the computing devices 104 and 106 and other computing devices. Communication between devices can occur over the Internet, over a local network (e.g., LAN), through direct communication (e.g., BLUETOOTH TM , radio frequency, infrared), etc.

[0038] In some embodiments, system 100 can include a risk detection platform. For example, computing system 102 and / or other computing devices can implement the risk detection platform. The risk detection platform can include a plurality of rules for assessing risk and implementing applications thereof. The rules can be designed from empirical data or through machine learning methods. For example, the platform can obtain data (e.g., transactions associated with various features and labeled with different risk levels) from various sources such as computing device 104 through communications 122. Computing device 104 can have obtained or stored such data in advance or in real-time. The platform can use the obtained data to construct or learn risk detection (or assessment) rules. The rules can be deployed in a remote server, cloud, client-side device, etc. For example, computing device 106 can be installed with a software application, web application, application program interface (API), or another suitable interface for invoking the rules.

[0039] In some embodiments, the rules can be deployed in computing device 106 or server 130. Computing device 106 can obtain one or more transactions 126 from one or more devices (e.g., 140a, 140b, etc.). The one or more devices can include a mobile phone, tablet, server, desktop computer, laptop, etc. For example, device 140b can be a mobile phone used to conduct a transaction (e.g., a consumer transaction, a credit card application) that is submitted to computing device 106 before being approved. Computing device 106 can apply the rules deployed in computing device 106 or invoke the rules deployed in server 130 through communications 124. Computing device 106 can apply the rules to one or more transactions 126 to determine their risk levels. Based on the determined risk levels, computing device 106 can implement subsequent steps such as approving or rejecting the transaction (e.g., by sending instructions to a bank or seller’s device), requiring additional verification (e.g., sending a verification code or task to device 140b to verify identity), etc.

[0040] Although computing system 102 is shown as a single computing device, it can include multiple computing devices that are communicatively coupled to each other and / or to other devices. For example, computing system 102 can include a server that is communicatively coupled to a client device. The server can implement the risk detection platform and the client device can invoke the rules of the platform. Figure 1The computing system 102 is shown as a single entity, but this is merely for ease of reference and is not meant to be limiting. One or more components or one or more functions of the computing system 102 described herein can be implemented in a single computing device or multiple computing devices. For example, the computing system 102 can include the computing device 106, and vice versa. That is, each of the first obtaining component 112, the second obtaining component 114, the approximating component 116, and the optimizing component 118 can be implemented in the computing system 102 or the computing device 106. Similarly, the computing system 102 can be coupled to and associated with one or more other computing devices that implement a portion of the components or functions of the computing system 102. The computing device 106 can include one or more processors and one or more memories coupled to the processors, the one or more memories configured with instructions executable by the one or more processors to cause the one or more processors to perform the various steps described herein.

[0041] The various components of the system 100 can be configured to perform steps for maximizing risk detection coverage by constraints. In some embodiments, the first obtaining component 112 can be configured to obtain a plurality of historical transactions from historical information. Each of the historical transactions can have been determined as risky or safe by one or more risk detection rules prior to the transaction being performed, and labeled as risky or safe according to a post-transaction determination (e.g., after the transaction is performed or investigated). A false identification occurs if a transaction is identified as risky (or safe) by the rules but is labeled as safe based on the post-transaction determination. In one embodiment, the rules can include a plurality of conditions, such as a transaction time (e.g., a date and / or a time of day when the transaction is performed), a transaction location (e.g., a geographic location where the transaction is performed), a transaction frequency (e.g., a frequency of the same user making transactions, a frequency of the same type of transaction being performed), a user history (e.g., how long the user has been registered with the transaction platform, a user history of using services provided by the platform or other platforms), a transaction amount, and a risk level (e.g., risky, safe).

[0042] In some embodiments, some of the plurality of transactions are labeled as risky transactions, and some of the plurality of transactions are labeled as safe transactions. “Risky” and “safe” are relative terms indicating different risk level labels. There can be more than two labels for a transaction. For example, some transactions have a risk level of 9-10 (labeled as risky), some transactions have a risk level of 7-8 (labeled as possibly risky), some transactions have a risk level of 5-6 (labeled as risk neutral), some transactions have a risk level of 3-4 (labeled as possibly safe), and some transactions have a risk level of 1-2 (labeled as safe). For simplicity, the embodiments in this specification assume that a transaction can be labeled as “risky” or “safe.” In some embodiments, the techniques described herein can be extended and applied to use cases that use multiple risk levels (e.g., use different weights for different degrees of error identification). For example, identifying an actually risky transaction as level 1 (e.g., absolutely safe) should be treated more severely (e.g., bear a higher weight) than identifying an actually risky transaction as level 6 (e.g., risk neutral); similarly, identifying an actually safe transaction as level 10 (e.g., absolutely risky) should be treated more severely (e.g., bear a higher weight) than identifying an actually safe transaction as level 6 (e.g., risk neutral).

[0043] In some embodiments, the second obtaining component 114 can be configured to obtain an initial subset of the plurality of risk detection rules. The initial subset can be used as a starting point for the iterative exploration process in seeking the best subset with the maximum risk detection coverage while satisfying the preset customer break rate. This initial subset can be determined manually or randomly. In some embodiments, the initial subset can be configured as an empty set (e.g., including 0 rules). The risk detection coverage of the initial subset need not satisfy any requirement, but its customer break rate can need to be no greater than a preset threshold. As described above, the risk detection coverage and the customer break rate of the initial subset can be learned from historical data. For example, the risk detection coverage of the initial subset can be determined based on the number of unique historical transactions that have been correctly identified as risky or safe, and the customer break rate of the initial subset can be determined based on the number of unique historical transactions that have been incorrectly identified as risky or safe. Here, “unique” means that if two or more rules in the subset correctly identify a risky (or safe) transaction, the transaction is only counted once in determining the risk detection coverage or the customer break rate of the subset.

[0044] In some embodiments, the approximation component 116 can be configured to approximate the objective function g(x) and the cost function f(x) in equation (1) based on a given subset (e.g., an initial subset in a first iteration). The approximation of the objective function can be a lower bound approximation (e.g., a lower bound data mapping) that generates, for a given subset, an approximate risk detection coverage for the given subset. The approximation of the cost function can be an upper bound approximation (e.g., an upper bound data mapping) that generates, for a given subset, an approximate customer disruption rate for the given subset. For simplicity, the lower bound approximation of g(x) is denoted as where, for any given subset x, The upper bound approximation of f(x) is denoted as where, where, for any given subset x,

[0045] Learning One of the reasons for learning is to reduce the search space of the original optimization problem set forth in equation (1). Also, since is a lower bound of g(x), it follows that Therefore, if a given set achieves the maximum risk detection coverage in the search space of , it must achieve at least as high a risk detection coverage in the search space of as the risk detection coverage in the search space of . Similarly, the reason for learning is that if a given set keeps its disruption rate below a pre-set threshold b in , its disruption rate must also be below b in f(x) (i.e.,

[0046] In some embodiments, the optimization component 118 can be configured to work with the approximation component 116 to iteratively explore the optimal subset for maximizing the risk detection coverage subject to the constraints. In some embodiments, an iteration can involve using the approximation component 116 to obtain a transition subset (e.g., an estimation step), and then using the optimization component 118 to optimize the transition subset to learn a new subset X t+1 for the next iteration (e.g., an optimization step). The iterative process involving the approximation component 116 and the optimization component 118 can continue until the subset X t+1 obtained in a new iteration is not better than the subset X t from a previous iteration.

[0047] Figure 2A An exemplary setup for maximizing the risk detection coverage is shown in accordance with some embodiments. Figure 2AThe settings shown in the middle can refer to a risk management system, such as a credit card application screening system in which credit card applications should be deemed risky or safe (e.g., risky applications can result in less amount of credit line); a fraud detection system in which consumer transactions or money transfer transactions can be evaluated (e.g., risky transactions can require additional authentication); or another suitable risk management system. The risk management system can be composed of multiple risk detection rules. Figure 2A The squares R1, R2, R3, and R4 in the middle can refer to multiple rules, the circles T1, T2, T3, and T4 can refer to different transactions that have been correctly identified by the rules (e.g., risky transactions are identified as risky and safe transactions are identified as safe), and the triangles t1, t2, t3, and t4 can refer to different transactions that have been incorrectly identified by the rules (e.g., risky transactions are identified as safe and safe transactions are identified as risky). In particular, R1 has correctly identified two transactions T1 and T2, but incorrectly identified one transaction t1; R2 has correctly identified three transactions T1, T3, and T4, but incorrectly identified two transactions t1 and t3; R3 has correctly identified two transactions T1 and T3, but incorrectly identified one transaction t2; and R4 has correctly identified two transactions T3 and T3, but incorrectly identified one transaction t3.

[0048] For each rule, the number of correctly identified transactions can be converted to its risk detection coverage (which can be referred to as a true positive score), and the number of incorrectly identified transactions can be converted to its customer disruption rate (which can be referred to as a false positive score). For a subset of rules that includes more than one rule, the number of uniquely correctly identified transactions can be converted to the subset’s risk detection coverage, and the number of uniquely incorrectly identified transactions can be converted to the subset’s customer disruption rate. For example, if the subset includes rules R1 and R2, the number of uniquely correctly identified transactions by the subset includes T1, T2, T3, and T4. Even though both R1 and R2 correctly identified transaction T1, T1 is only counted once when determining the number of uniquely correctly identified transactions. Similarly, for the subset including R1 and R2, the number of uniquely incorrectly identified transactions is 2 (e.g., the subset’s false positives include t1 and t3). Even though both R1 and R2 incorrectly identified transaction t1, t1 only needs to be counted once when determining the subset including R1 and R2’s customer disruption rate. In some embodiments, not only the number of uniquely identified transactions is considered, but different transactions can also be assigned different weights (e.g., a completely incorrect identification can be assigned a higher penalty weight, and a less incorrect identification can be assigned a lower weight). The manner of converting the number to coverage or rate is not limited herein. For simplicity, the following description directly uses the number of uniquely correctly or incorrectly identified transactions by a given subset as its risk detection coverage or customer disruption rate, respectively.

[0049] In this context, the task to be solved by embodiments is to determine a subset (e.g., can be empty, part of or all rules) to maximize its risk detection coverage while being subject to a constraint on its customer disruption rate (e.g., the number of false-identified transactions can not exceed 2).

[0050] Figure 2B An exemplary data query system for maximizing risk detection coverage is shown in accordance with some embodiments. As mentioned above, for a given single rule or a given subset of multiple risk detection rules, the corresponding risk detection coverage and customer disruption rate can be learned from historical information. The historical information can include historical transactions that the rules have been applied to. Each historical transaction can have been identified by multiple rules, and each rule can have been applied to multiple transactions. The historical information can be collected from a pre-set time period.

[0051] Reference is made to Figure 2B , the server 240 can be configured to store historical information and respond to queries about risk detection coverage and customer disruption rate for a given single rule or a set of rules. Although Figure 2B The server 240 in Figure 2B is shown as one single entity, it can include multiple entities depending on the implementation. For example, it can include a database (e.g., centralized or distributed) for storing data and a computing system (e.g., used as an oracle) for responding to queries.

[0052] In some embodiments, the historical information can be logically organized as a table 250. The first column 252 of the table 250 lists the rules, the second column 254 of the table 250 lists the historical transactions that have been correctly identified by each of the rules, and the third column 256 of the table 260 lists the historical transactions that have been falsely identified by each of the rules. Figure 2B The content of the table 250 in Figure 2A corresponds to the exemplary setup shown in Figure 2A . It can be appreciated that the table 250 is a logical view of the historical information, and the actual layout of the database can be implemented in various ways depending on the structure of the storage system (e.g., centralized, distributed across multiple storage nodes, or cloud-based storage service).

[0053] In Figure 2BIn this example, query 260 is sent to server 240, which includes a pair of rules {R1, R2}. In response, server 240 returns an exemplary response 280 including two fields: a first field including unique historical transaction IDs of {R1, R2} that have been correctly identified (e.g., in this case, T1–T4, 4 transactions), and a second field including unique historical transaction IDs of {R1, R2} that have been incorrectly identified (e.g., in this case, t1 and t3, 2 transactions). In some embodiments, the response may include richer information beyond the transaction IDs. For example, the actual flag for each transaction (e.g., a manually verified risk rating) and the assessment of each rule applied to that transaction (e.g., a predicted risk rating) may be returned. The response may be to determine a given set of rules (e.g., in…). Figure 2B As shown, the risk detection coverage and customer breach rate of {R1, R2} are based on this.

[0054] Figure 3 An exemplary method 300 for maximizing risk detection coverage is shown according to some embodiments. Figure 3 Method 300 is intended to be illustrative and may include fewer, more, or alternative steps, such as Figure 3 As shown, this depends on the implementation method. Method 300 can be derived from... Figure 1 The computing system 102 in the middle is implemented and applied to Figure 2A and Figure 2B The problem shown.

[0055] As shown in the figure, method 300 may include an iterative exploration of seeking a subset of multiple risk detection rules to deploy risk detection on future transactions. The multiple risk detection rules can be understood as a candidate pool, and the subset may include some or all of the candidates. The subset to be selected from the candidates can form an optimal solution to maximize risk detection coverage while being constrained by the rate of customer disruption.

[0056] In some embodiments, method 300 may begin at step 310 by obtaining a first subset of multiple risk detection rules, the first subset being associated with a first coverage score, wherein the first coverage score indicates the number of unique historical transactions that have been correctly identified by the multiple risk detection rules in the first subset.

[0057] For example, Figure 3Step 310 in the method 300 includes initializing a subset of the plurality of risk detection rules. The initialized subset can be selected manually or generated randomly. It can be used as a starting point for the iterative process of the method 300. The coverage of risk detection provided by the initial subset (e.g., the number of unique historical transactions that have been correctly identified by the plurality of risk detection rules in the initial subset) is not required, but the customer breach rate associated with the initial subset can need to be less than a preset threshold. In some embodiments, the initial subset can be empty, as the customer breach rate associated with an empty subset can be considered 0, which is less than any given positive threshold.

[0058] In some embodiments, the step 320 of the method 300 can include constructing, based on the first subset, an output lower bound data mapping for an approximate coverage score for an input subset, wherein: when the input subset is the first subset, the approximate coverage score is the same as the first coverage score, and when the input subset is a second subset that is different from the first subset and corresponds to a second coverage score that indicates a number of unique historical transactions that have been correctly identified by the risk detection rules in the second subset, the approximate coverage score is not greater than the second coverage score.

[0059] In some embodiments, the lower bound approximation represented as can be a submodular and monotonic function determined by the following steps: generating a sequence by reordering the plurality of risk detection rules, wherein the risk detection rules in the first subset are placed first in the sequence; based on the generated sequence, constructing a list S i of temporal subsets, 0≤i≤n, wherein: n is the number of the plurality of risk detection rules, the temporal subset S0is empty, and for a given i, where 1≤i≤n, the temporal subset S i includes the i th risk detection rules in the sequence and all risk detection rules in the temporal subset S i-1 , and determining an approximate coverage score for each risk detection rule in the sequence; and learning a variational approximation function to determine a coverage score for a given subset of the plurality of risk detection rules based on a sum of individual approximate coverage scores for each risk detection rule in the given subset.

[0060] For example, assuming that the current iteration is t th and the subset is X t (e.g., the known subset), the permutation p of the risk detection rule [n] can be determined by placing the elements in X t first and then including the remaining rules in [n] where (e.g., not including the elements in X t ). The permutation p can be understood as a sequence represented as {p1, p2,... pn} by reordering the plurality of risk detection rules in [n]. For example, if X n = {1, 2, 3}, then the permutation p of the risk detection rule [n] = {1, 2, 3, 4, 5} can be determined by placing the elements in X = {1, 2, 3} first and then including the remaining rules in [n] where (e.g., not including the elements in X). The permutation p can be understood as a sequence represented as {p1, p2,... pn} by reordering the plurality of risk detection rules in [n]. For example, if Xt including [rulei, rule2], then pi in the sequence can be {rulei} and p2 can be {rule2} (e.g., placing X first in the sequence t the rules in X).

[0061] Next, a list of time subsets can be constructed based on the sequence in the following manner.

[0062]

[0063] This results in Thus, for a given subset X, can be given by:

[0064]

[0065] where j is any risk detection rule in the given subset X, and X is a subset of the plurality of risk detection rules [n]. The above equation can be understood as: for a given subset X, the value of is the sum of the values for each risk detection rule in the subset X. In some embodiments, determining an individual approximate coverage score for each risk detection rule j in the sequence includes, for i th risk detection rule in the sequence, determining an individual approximate coverage score based on a difference between a coverage score of a time subset S i and a coverage score of a time subset S i-1 , where the coverage score of the time subset S i and the coverage score of the time subset S i-1 are known by querying a database of historical transactions.

[0066] For example, for can be defined as:

[0067]

[0068] where the values of and can be obtained by examining historical transactions. For example, since the rules in X are known, the number of unique transactions that have been correctly identified (e.g., identified as risky transactions) by the rules in X can be known by querying historical information, and then As

[0069] Figure 3 AsAs shown in the middle, g(X) can refer to a black box service that takes a given rule set (as input X) and provides a corresponding risk detection coverage (as output) at step 320. For example, the value of g(X) for a given rule set can be queried by using the server 240 in Figure 2B

[0070] In some embodiments, step 330 of the method 300 can include constructing an upper bound data mapping having a set of parameters based on the first subset, wherein: the upper bound data mapping outputs an approximate breakage score for an input subset of the plurality of risk detection rules, the output approximate breakage score is the same as the first breakage score when the input subset is the first subset, and the output approximate breakage score is not less than a second breakage score when the input subset is a second subset different from the first subset and is associated with the second breakage score indicating a number of unique historical transactions that have been incorrectly identified as risky transactions by the risk detection rules in the second subset.

[0071] The following section describes an exemplary method of obtaining an upper bound approximation of the cost function f(x). In some embodiments, because the cost function f(x) is a submodular function, it satisfies:

[0072] f(X) + f(Y) > f(X U Y) + f(X n Y) (3)

[0073] where X and Y refer to two different subsets of the plurality of risk detection rules, X U Y refers to the union of X and Y, and X n Y refers to the intersection of X and Y. The above equation can be interpreted as: even if X + Y = X U Y + X n Y, f(X) + f(Y) is greater than f(X U Y) + f(X n Y), the marginal benefit of adding f(X n Y) to f(X U Y) is reduced due to the submodularity of f(x). Equation (3) can be rewritten in the following format:

[0074]

[0075] ​where θ = X∩Y denotes the intersection of X and Y, "X\Y" denotes elements (e.g., rules) in X but not in Y, "Y\X" denotes elements (e.g., rules) in Y but not in X, "f(j|X\j)" denotes the boundary cost resulting from adding rule j to the subset X that does not have j (e.g., if X does not include j, X\j is equal to X, but if X includes j, it is equal to X excluding j), and f(j|θ) denotes the boundary cost of adding rule j to θ (e.g., the increase in customer churn rate), which is X∩Y.

[0076] Based on equation (4), the upper bound approximation of the cost function f(x) can be expressed as the following equation:

[0077]

[0078] where b is the current threshold of customer churn rate, X t and X t+1 denote the solution candidate (e.g., subset) at iteration t and the solution candidate at iteration t+1, respectively, denotes the upper bound approximation of the cost function f(x) constructed based on X t at iteration t (e.g., it means that needs to be updated in each iteration), denotes the approximate customer churn rate generated for the solution candidate X t+1 , where θ = X t ∩X t+1 can be understood as a set of parameters used. In equation (5), is the breakdown of according to equation (4). In equation (4), the current iteration is assumed to be t, where X t is known, and X t+1 is unknown and to be explored. Thus, the part is all known constants, while the part is unknown and to be explored. In other words, in order to determine the value of for X t+1 ; θ needs to be learned or estimated. Here, θ can be understood as a set of parameters used.

[0079] Referring back to equation (5), even though is the upper bound of f(X t+1 ) (e.g., It is desirable to make them as close as possible. An analogy to explain the above statement is "for a given budget b, to maximize the payoff, the spending needs to be kept as close to b as possible." In some embodiments, Nemhauser divergence can be used to represent the difference between f(X t+1 ) and f(X t+1 ), as shown in the following equation:

[0080]

[0081] where D represents divergence, e.g., the difference between f(X t+1 ) and f(X t+1 ).

[0082] With the above explanation, the task of constructing an upper bound approximation (e.g., data mapping) can be converted to the task of finding the minimum of Since θ = X t ∩ X t+1 (where X t is known and X t+1 is unknown), after obtaining θ, X t+1 (i.e., the subset for the next iteration t+1) can be determined.

[0083] As shown in FIG. 3B, f(x) at step 330 can refer to a block box service that takes a given rule set (as input) and provides a corresponding customer churn rate (as output). For example, f(x) can be learned based on server 240 in FIG. 2B, which receives an input query 260 including a rule set and generates a response 280 including unique transactions correctly identified by the rule set and unique transactions incorrectly identified by the rule set. The unique transactions incorrectly identified by the rule set can be used to determine the customer churn of the rule set. The upper bound approximation of f(x) can refer to Figure 3 the Figure 2B in equation (5) in the accompanying text. The upper bound approximation of f(x) is learned during each iteration and based on the input subset (e.g., the initial subset for the first iteration). In Figure 1 , θ can be understood as a parameter set for Once θ is estimated, it can be used to estimate a new subset for the next iteration.

[0084] At step 340, a parameter set 0 for the upper bound approximation of f(x) can be determined. An example way of determining 0 can include: for each risk detection rule in a plurality of risk detection rules: determining a first approximation coverage score based on the lower bound data mapping; determining a first disruption score increase associated with adding each risk detection rule to a first set of risk detection rules based on a number of unique historical transactions that have been incorrectly identified by each risk detection rule; and determining a first ratio of each risk detection rule, where the approximation coverage score is the numerator and the first disruption score increase is the denominator; generating a sequence by ordering the plurality of risk detection rules in descending order according to the determined first ratios of the plurality of risk detection rules; selecting a maximum number of risk detection rules for which a first total disruption score increase is not greater than a preset threshold, where the first total disruption score is a sum of the first disruption score increases associated with the selected risk detection rules; and determining the parameter set as an intersection of the first subset and the selected risk detection rules.

[0085] For example, at the tth iteration with known subset X t , 0 can be determined by the following steps. To simplify the description, Ej can be defined as the set that does not include j, i.e.,

[0086]

[0087] The above representation can be understood as: for a given j, if j is in the subset X t , then Ej is equal to X t but does not include j; if j is not in the subset X t , then Ej is equal to the subset X t .

[0088] For each of the plurality of rules in [n], an approximation coverage score can be determined by and a corresponding disruption score can be determined based on f(j|E\j) (e.g., by adding j to the disruption rate in the subset E\j). Then, a ratio can be determined for each rule as According to the ratios determined for all rules in [n], the rules can be ordered in descending order as {∈1,∈2,…∈ n} such that:

[0089]

[0090] where ∈ refers to a rule in the plurality of risk detection rules [n]. There are a such that where b is a preset threshold of customer disruption rate. Here refers to providing a total customer disruption rate increase (e.g., a maximum number of risk detection rules less than b.

[0091] a sorted sequence {e1, e2, … en} of the risk detection rules based on the approximation coverage scores. n and the value The parameter set θ in equation (5) can be estimated as:

[0092]

[0093] where ∩ refers to an intersection operation. refers to an estimate of the true value of θ at iteration t, and can be used to improve the current subset X t to obtain a new subset X t+1 at the next iteration (e.g., iteration t+1).

[0094] At step 350, a new subset for the next iteration can be determined. An exemplary way for determining the new subset can include: ranking the plurality of risk detection rules based on the first subset, the parameter set, and the approximation coverage scores generated by the lower bound data mapping through the plurality of risk detection rules; and selecting one or more consecutive risk detection rules from the beginning of the ranked plurality of risk detection rules as the third subset. In some embodiments, ranking the plurality of risk detection rules can include: for each risk detection rule in the plurality of risk detection rules: determining a second approximation coverage score based on the lower bound data mapping; determining a second risk detection rule set as the parameter set if each risk detection rule is not in the first subset; or as the first subset excluding each risk detection rule if each detection rule is in the first subset; determining a second disruption score increase associated with adding each risk detection rule to the second risk detection rule set; and determining a second ratio of each risk detection rule, where the second approximation coverage score is the numerator and the second disruption score increase is the denominator; and generating a sequence by ranking the plurality of risk detection rules in descending order according to the second ratio of each risk detection rule; and selecting a maximum number of risk detection rules for which a second total disruption score increase is not greater than a preset threshold, where the second total disruption score is a sum of the second disruption score increases associated with the selected risk detection rules.

[0095] For example, the optimization component 118 can learn the new subset X t+1 for the t+1th iteration by the following steps. Let the given subset X t be represented as:

[0096]

[0097] The above representation can be understood as: for a given j, if j is in the subset X t , then M\j is equal to X tbut does not include j; if j is not in the subset X t

[0098] For each of the plurality of risk detection rules, a ratio can be determined as where j refers to an individual rule. From the ratios determined for all rules in [n], the rules can be ordered in descending order as {μ1, μ2,... μ n}, such that:

[0099]

[0100] By letting be the largest index that satisfies the following inequality:

[0101]

[0102] where b is a pre-determined threshold of customer disruption rate. Here refers to the largest number of risk detection rules that provide a total customer disruption rate increase (e.g., ) that is less than b. Based on X t+1 for the next iteration can be determined as:

[0103]

[0104] At step 360, the new subset generated at step 350 can be compared to the old subset (e.g., the initial subset at the first iteration) to determine whether the iteration process can terminate. In some embodiments, the exit condition of step 360 can include whether the risk detection coverage of the new subset is equal to or greater than the risk detection coverage of the old subset. If yes, it means that the new subset is a better solution than the old subset, which means that the iteration process can continue to find a next better solution; if no, it means that a better solution can not be found, and thus the iteration process can terminate.

[0105] At step 370, once the exit condition of step 360 is satisfied, the final solution can be determined as the subset with the largest risk detection coverage from the risk detection coverage that has been explored by the iteration process.

[0106] During the iteration method 300, all subsets explored are guaranteed to have a customer disruption rate that is lower than a pre-determined threshold, due to the way the lower bound approximation and the upper bound approximation are constructed. Figure 4 The process is visualized for better understanding.

[0107] Figure 4 An exemplary illustration for maximizing risk detection coverage is shown, according to some embodiments.​Figure 4 The illustrations in the text involve Figure 3 The iterative process shown consists of two iterations: iteration t and iteration t+1. Figure 4 In the diagram, the X-axis at 402 represents the distinct subsets of the risk detection rule candidates. Assuming there are n candidates, the total number of distinct subsets is 2n. n Each point on the X-axis 402 can point to a customer with a destruction rate below a preset threshold of 2. n One of the subsets. Figure 4 In the diagram, Y-axis 404 represents the risk detection coverage corresponding to each point / point on X-axis 402. Figure 4 The solid curve 410 in the figure represents the entire set of risk detection coverage for all subsets (e.g., points on the X-axis) with customer breach rates below a preset threshold. In some embodiments, the number of subsets can become enormous when the number of rule candidates is large. Enumerating all valid subsets to determine the optimal subset may be impractical (this problem has been recognized as an NP-hard problem). This is why... Figure 3 The reason why iterative method 300 becomes necessary.

[0108] like Figure 4 As shown, X t It refers to the known subset at iteration t, and the goal is to search for a new subset X in the next iteration t+1. t+1 Understandably, during the first iteration, X... t It refers to Figure 3 The initial subset determined in step 310. For subset X t The corresponding risk detection coverage area can be referred to as point 422 on the solid line 410.

[0109] Dotted line 422 can refer to the lower bound approximation of solid line 410. Solid line 410 can refer to g(x) in formula (EE1) (e.g., the objective function), and dotted line 422 can refer to the lower bound approximation of g(X) in formula (3), expressed as like Figure 4 As shown, It is always below g(X), except at point X. t Place, Both g(X) and g(X) produce the same risk detection coverage (e.g., point 422).

[0110] based on and X t The set of parameters for the upper bound approximation of f(X) can be determined. After determining θ, X can be determined based on formula (9). t+1 (The detailed process is explained in the description attached to formula (9). For example... Figure 4 As shown, X t+1points 424 (solid line 410) that can be projected onto g(X) t+1 . Because points 424 corresponding to subset X t may have a higher risk detection coverage than points 422 corresponding to subset X t+1 , the iteration process can continue.

[0111] At iteration t+1, the new lower bound approximation constructed based on X t+1 may be represented as a new point line 430, i.e. As shown, new point line 430 is always below g(X) except for the X t+1 point on the X-axis, where both g(X t+1 ) yield a risk detection coverage represented as point 424. Subsequently, a new parameter set θ for the upper bound approximation of f(X) can be determined. After determining θ, X t+2 may be determined based on equation (9). The iteration process can continue until the newly found subset has a lower risk detection coverage than the previous subset.

[0112] The methods disclosed herein can be applicable to other use cases that require selecting a representative subset of multiple candidates to achieve a goal while satisfying constraints. For example, in the field of image set summarization, an image set (representative subset) can be identified from a very large number of image candidates (represented as a group set) to achieve a goal (e.g., representing desired features of images in the group set) while being subject to certain constraints (e.g., the number of images in the representative subset can not be larger than a pre-set number, or the total size of image files must be smaller than a pre-set size). In this regard, a general method is provided herein that includes obtaining a first subset of multiple candidates (e.g., an initial version of a representative subset, which can be iteratively improved using the disclosed methods), the first subset being associated with a first true positive score and a first false positive score. The first true positive score indicates a gain associated with the candidates in the first subset (e.g., in the case of image set summarization, the first true positive score can be determined based on distinct features it covers), and the first false positive score indicates a cost associated with the candidates in the first subset (e.g., in the case of image set summarization, the first false positive score can be determined based on unwanted features covered by images in the first subset). The method can further include constructing a lower bound data mapping based on the first subset, the lower bound data mapping outputting an approximate true positive score for an input subset. When the input subset is the first subset, the approximate true positive score is the same as the first true positive score, and when the input subset is a second subset different from the first subset, the approximate true positive score is not greater than a second true positive score associated with the second subset, the second true positive score indicating a gain associated with the candidates in the second subset. The method can further include constructing an upper bound data mapping comprising a set of parameters based on the first subset. The upper bound data mapping outputs an approximate false positive score for an input subset of multiple candidates. When the input subset is the first subset, the output approximate false positive score is the same as the first false positive score, and when the output subset is a second subset different from the first subset, the output approximate false positive score is not less than a second false positive score associated with the second subset, the second false positive score indicating a gain associated with the candidates in the second subset. The method can further include, from the first subset, the lower bound data mapping, and the upper bound data mapping, generating a third subset of the multiple candidates based on at least the approximate true positive score output by the lower bound data mapping corresponding to the multiple candidates as input and the set of parameters associated with the upper bound data mapping. The third subset is associated with a third true positive score indicating a gain associated with the multiple candidates in the third subset. The method can further include comparing the first true positive score with the third true positive score, and in response to the first true positive score exceeding the third true positive score, selecting the candidates in the first subset for a new transaction.

[0113] Figure 5An example method for risk detection is shown in accordance with various embodiments. The method 500 can be performed by a device, apparatus, or system for risk detection. The method 500 can be performed by the computing system 300 in Figures 1 to 4 the environment or system shown, such as Figure 3 the computing system 300 in FIG. 1. The operations of the method 500 presented below are intended to be illustrative. Depending on the implementation, the method 500 can include additional, fewer, or alternative steps performed in various orders or in parallel.

[0114] Block 510 includes obtaining a first subset of the plurality of risk detection rules, the first subset being associated with a first coverage score and a first break score. In some embodiments, the first coverage score indicates a number of unique historical transactions that have been correctly identified by the plurality of risk detection rules in the first subset, and the first break score indicates a number of unique historical transactions that have been incorrectly identified by the plurality of risk detection rules in the first subset.

[0115] Block 520 includes constructing, based on the first subset, an output lower bound data mapping for an approximate coverage score for an input subset. In some embodiments, when the input subset is the first subset, the approximate coverage score is the same as the first coverage score, and when the input subset is a second subset different from the first subset, the approximate coverage score is not greater than a second coverage score associated with the second subset, the second coverage score indicating a number of unique historical transactions that have been correctly identified by risk detection rules in the second subset. In some embodiments, the lower bound data mapping includes a submodular function and a monotonic function. In some embodiments, the first subset is empty. In some embodiments, constructing the lower bound data mapping includes: generating a sequence by reordering the plurality of risk detection rules, wherein the risk detection rules in the first subset are placed in the first position in the sequence; based on the generated sequence, constructing a list of temporal subsets S i , 0≤i≤n, where: n is a number of the plurality of risk detection rules, temporal subset S0is empty, and for a given i, where 1≤i≤n, temporal subset S i includes the i-th risk detection rule in the sequence and all risk detection rules in temporal subset S i-1 . Determining an approximate coverage score for each risk detection rule in the generated sequence. And determining a coverage score for a given subset of the plurality of risk detection rules as a sum of individual approximate coverage scores for each risk detection rule in the given subset. In some embodiments, determining the individual approximate coverage score for each risk detection rule in the sequence includes: for the i-th risk detection rule in the sequence, determining the individual approximate coverage score based on a difference between a coverage score of temporal subset S i and a coverage score of temporal subset S i-1 . In some embodiments, determining the individual approximate coverage score for each risk detection rule in the sequence includes: for the i-th risk detection rule in the sequence, determining the individual approximate coverage score based on a difference between a coverage score of temporal subset S i and a coverage score of temporal subset Si-1 The coverage score is learned by querying a database of historical transactions.

[0116] Block 530 includes constructing an upper bound data map comprising a set of parameters based on the first subset. In some embodiments, the upper bound data map outputs an approximate breakdown score for an input subset of a plurality of risk detection rules. When the input subset is the first subset, the output approximate breakdown score is the same as the first breakdown score; and when the input subset is a second subset different from the first subset, the output approximate breakdown score is not less than a second breakdown score associated with the second subset; the second breakdown score indicates a number of unique historical transactions that have been incorrectly identified as risky transactions by the risk detection rules in the second subset. In some embodiments, constructing the upper bound data map with the set of parameters includes determining the set of parameters by, for each risk detection rule in the plurality of risk detection rules: determining a first approximate coverage score based on the lower bound data map; determining a first breakdown score increase associated with adding each risk detection rule to the first set of risk detection rules based on a number of unique historical transactions that have been incorrectly identified by each risk detection rule; and determining a first ratio of each risk detection rule, where the approximate coverage score is the numerator and the first breakdown score increase is the denominator; generating a sequence by ordering the plurality of risk detection rules in descending order according to the determined first ratios of the plurality of risk detection rules; selecting a maximum number of risk detection rules, where a first total breakdown score increase is not greater than a preset threshold; where the first total breakdown score is a sum of the first breakdown score increases associated with the selected risk detection rules; and determining the set of parameters as an intersection of the first subset and the selected risk detection rules. In some embodiments, the first set is determined as the first subset if each risk detection rule is not in the first subset, or the first set is determined as the first subset excluding each risk detection rule if each risk detection rule is in the first subset.

[0117] Block 540 includes generating, according to the first subset, the lower bound data mapping, and the upper bound data mapping, a third subset of the plurality of risk detection rules based on at least: the approximate coverage scores output by the lower bound data mapping corresponding to the plurality of risk detection rules as input, and the set of parameters associated with the upper bound data mapping. In some embodiments, the third subset is associated with a third coverage score indicating a number of unique historical transactions that have been correctly identified by the plurality of risk detection rules in the third subset. In some embodiments, generating the third subset of the plurality of risk detection rules includes: ranking the plurality of risk detection rules based on the first subset, the set of parameters, and the approximate coverage scores generated by the lower bound data mapping of the plurality of risk detection rules; and selecting, from a beginning of the ranked plurality of risk detection rules, one or more consecutive risk detection rules as the third subset. In some embodiments, ranking the plurality of risk detection rules includes: for each risk detection rule in the plurality of risk detection rules: determining a second approximate coverage score based on the lower bound data mapping; determining a second set of risk detection rules based on the set of parameters if each risk detection rule is not in the first subset; or determining the second set of risk detection rules as the first subset excluding each risk detection rule if each detection rule is in the first subset; determining a second disruption score increase associated with adding each risk detection rule to the second set of risk detection rules; and determining a second ratio of each risk detection rule, where the second approximate coverage score is a numerator and the second disruption score increase is a denominator; generating a sequence by ranking the plurality of risk detection rules in descending order according to the second ratio of each risk detection rule; and selecting the one or more consecutive risk detection rules as the third subset includes selecting a largest number of risk detection rules for which a second total disruption score increase is not greater than a pre-set threshold, where the second total disruption score is a sum of the second disruption score increases associated with the selected risk detection rules.

[0118] Block 550 includes comparing the first coverage score to the third coverage score.

[0119] Block 560 includes selecting, in response to the first coverage score exceeding the third coverage score, the rules in the first subset for use in risk detection of new transactions.

[0120] In some embodiments, the method 500 can further include, in response to the first coverage score not exceeding the third coverage score, replacing the first subset with the third subset as an updated first subset, where the first coverage score is replaced by the third coverage score of the third subset accordingly; performing one or more iterations of a process including: updating the lower bound data mapping based on the updated first subset; generating an updated third subset associated with an updated third coverage score based on the updated first subset and the updated lower bound data mapping; and if the exit condition is not satisfied, replacing the updated first subset with the updated third subset and replacing the updated first coverage score with the updated third coverage score, based on the construction step and the generation step cyclically. In some embodiments, the exit condition includes at least one of: the updated first coverage score being greater than the updated third coverage score, and the number of iterations being greater than a preset number.

[0121] Figure 6 A block diagram of a computer system for risk detection is shown in accordance with some embodiments. The computer system 600 can be an example of Figure 1 the implementation of one or more modules in a computing system in Figures 1 to 5 one or more other components shown in Figure 5 The method 500 in can be implemented by the computer system 600. The computer system 600 can include one or more processors and one or more non-transitory computer-readable storage media (e.g., one or more memories) coupled to the one or more processors and configured with instructions executable by the one or more processors to cause a system or device (e.g., a processor) to perform the methods described above (e.g., the method 500). The computer system 600 can include various units / modules corresponding to the instructions (e.g., software instructions).

[0122] In some embodiments, the computer system 600 can be referred to as an apparatus for risk detection. The apparatus can include an obtaining module 620 for obtaining a first subset of a plurality of risk detection rules, the first subset being associated with a first coverage score and a first breakage score, where: the first coverage score indicates a number of unique historical transactions that have been correctly identified by the plurality of risk detection rules in the first subset; and the first breakage score indicates a number of unique historical transactions that have been incorrectly identified by the plurality of risk detection rules in the first subset; a first approximation module 640 for constructing, based on the first subset, a lower bound data mapping that outputs an approximation coverage score for an input subset; and a second approximation module 660 for constructing, based on the first subset, an upper bound data mapping including a set of parameters; and an exploration module 680 for generating, from the first subset, the lower bound data mapping, and the upper bound data mapping, a third subset of the plurality of risk detection rules.

[0123] The technology described herein can be implemented by one or more special-purpose computing devices. The special-purpose computing devices can be a desktop computer system, a server computer system, a portable computer system, a handheld device, a networked device, or any other device or combination of devices that implements the technology in conjunction with hard-wired and / or program logic. The special-purpose computing device can be implemented as a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of these. A computing device typically can be controlled by operating system software that includes computer programs, such as applications programs, firmware, and / or middleware, stored in memory that can provide file system, networking, I / O services, and user interface functionality such as a graphical user interface (“GUI”), among other things. The various systems, apparatuses, storage media, modules, and units described herein can be implemented in a special-purpose computing device or one or more computing chips of one or more special-purpose computing devices. In some embodiments, the instructions described herein can be implemented in a virtual machine on the special-purpose computing device. When executed, the instructions can cause the special-purpose computing device to perform the various methods described herein. The virtual machine can include software, hardware, or a combination thereof.

[0124] Figure 7 An example computing device that can implement any of the embodiments described herein is shown. The computing device can be used to implement one or more components of the systems and methods shown. Figures 1 to 5 The computing device 700 can include a bus 702 or other communication mechanism for communicating information, and a one or more hardware processors 704 coupled with bus 702 for processing information. The hardware processor(s) 704 can be one or more general purpose microprocessors.

[0125] The computing device 700 can also include a main memory 707, such as a random access memory (RAM), cache and / or other dynamic storage, coupled to bus 702 for storing information and instructions to be executed by processor 704. Main memory 707 also can be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 704. Such instructions can be stored and executed by computing device 700 in memory, as

[0126] The computing device 700 can implement the techniques described herein using customized hard-wired logic, one or more ASICs or FPGAs, firmware and / or program logic which in combination with the computing device can cause the computing device 700 to be or to program a special-purpose machine. According to one embodiment, the techniques herein are performed by computing device 700 in response to processor 704 executing one or more sequences of instructions contained in main memory 707. Such instructions can be read into main memory 707 from another storage medium, such as storage device 709. Execution of the sequences of instructions contained in main memory 707 can cause processor 704 to perform the process steps described herein. One or more sequences of instructions can be implemented using firmware, one or more of software modules, and / or utility / callable routines, all of which should be treated as examples. That is, examples of hardware logic (including programmable logic for use with a programmable logic device) can be used in some embodiments. In general, software routines can implement process steps for implementing the techniques described herein. During operation of computing device 700, processor 704

[0127] The computing device 700 also includes a communication interface 710 coupled to bus 702. Communication interface 710 can provide a two-way data communication coupling to one or more network links that are connected to one or more networks. As another example, communication interface 710 can be a local area network (LAN) card to provide a data communication connection to a compatible LAN (or WAN component to have communication across a WAN). Wireless links can also be implemented.

[0128] The performance of certain of the operations can be distributed among the processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the processors or processor- implemented engines can be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other example embodiments, the processors or processor-implemented engines can be distributed across a number of geographic locations.

[0129] Each of the processes, methods, and algorithms described in the preceding sections can be embodied in, and fully or partially automated by, code modules executed by one or more computer systems or computer processors comprising computer hardware. The processes and algorithms can be implemented partially or wholly in application-specific circuitry.

[0130] When the functions disclosed herein are implemented in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. The specific technical solutions disclosed herein (all or part) or aspects that facilitate the current technology can be implemented in the form of a software product. The software product can be stored in a storage medium, including a plurality of instructions to make a computing device (which can be a personal computer, a server, a network device, etc.) execute all or some of the steps of the methods of the embodiments of the present application. The storage medium can include a flash drive, a portable hard drive, a ROM, a RAM, a magnetic disk, an optical disk, another medium operable to store program codes, or any combination thereof.

[0131] Particular embodiments also provide a system that includes a processor and a non-transitory computer-readable storage medium storing instructions executable by the processor to cause the system to perform operations corresponding to the steps in any of the methods of the above-described embodiments. Particular embodiments also provide a non-transitory computer-readable storage medium configured with instructions executable by one or more processors to cause the one or more processors to perform operations corresponding to the steps in any of the methods of the above-described embodiments.

[0132] Embodiments disclosed herein can be implemented by a cloud platform, a server or a group of servers (hereinafter collectively referred to as "service system") interacting with a client. The client can be a terminal device or a client registered by a user on the platform, wherein the terminal device can be a mobile terminal, a personal computer (PC), and any device that can install a platform application.

[0133] The various features and processes described above can be used independently of one another or can be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of the present disclosure. Additionally, some of the methods or processes described herein can be omitted in some implementations. The methods and processes described herein are also not limited to any particular order or sequence, and the blocks or states relating thereto can be performed in other orders or sequences that are appropriate. For example, blocks or states described as being performed sequentially can be performed at the same time, or in a different order. Additionally, some blocks or states can be performed multiple times. The example blocks or states can be performed by hardware, software, firmware, or combinations thereof. Example blocks or states can be performed by a processing element, such as one or more processors, processor cores, or processing units. The example systems and components described herein can be configured differently than described. For example, elements can be added, removed, or rearranged.

[0134] Various operations of example methods described herein can be performed, at least partially, by an algorithm. An algorithm can be included in program code or instructions stored in a memory (e.g., a non-transitory computer readable storage medium as described above). Such an algorithm can include a machine learning algorithm. In some embodiments, a machine learning algorithm can not explicitly program a computer to perform a function, but can instead learn from training data to make a predictive model that performs the function.

[0135] Various operations of example methods described herein can be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute processor-implemented engines that operate to perform one or more operations or functions described herein.

[0136] Similarly, the methods described herein can be at least partially processor- implemented, with a particular one or more processors being an example of hardware. For example, at least some of the operations of a method can be performed by one or more processors or processor-implemented engines. Also, one or more processors can operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS). For example, at least some of the operations can be performed by a group of computers (as examples of machines including processors), with these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., an application program interface (API)).

[0137] The performance of certain of the operations can be distributed among the processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the processors or processor-implemented engines can be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other example embodiments, the processors or processor-implemented engines can be distributed across a number of geographic locations.

[0138] Throughout this specification, plural instances can implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations can be performed concurrently, and nothing requires that the operations be performed in the order illustrated. Structures and functionality presented as separate components in example configurations can be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component can be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.

[0139] Although the subject matter has been described in reference to specific examples, it is understood that various modifications and changes can be made to these examples without departing from the broader scope of the disclosure. The described subject matter can be implemented in a further embodiment without one or more of the specific details described herein. The described subject matter can also be implemented in a combination of one or more embodiments. The preceding description, as such, is provided by way of example only and is not intended to limit the scope of the disclosure unless otherwise specifically indicated.

[0140] The embodiments shown herein are described in sufficient detail to enable those skilled in the art to practice the disclosed teachings. Other embodiments can be utilized and derived without departing from the scope of the disclosure, and structural and logical substitutions and changes can be made without departing from the scope of the disclosure. The specific embodiments disclosed are not intended to limit the scope of the disclosure, which is defined solely by the claims, and any equivalents thereof. The Specific embodiments are presented for purposes of illustration and not limitation.

[0141] Any process descriptions, elements, or blocks in the flow diagrams described herein and / or depicted in the attached figures should be understood as potentially representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process. Alternate implementations are included within the scope of the embodiments described herein in which elements or functions can be deleted, executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved as would be understood by those skilled in the art.

[0142] As used herein, “or” is inclusive and not exclusive, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A, B, or C” means “A, B, A and B, A and C, B and C, or A, B, and C” unless otherwise expressly specified or indicated by context. Moreover, “and” is joint and several, unless otherwise expressly stated or indicated by context. Therefore, herein, “A and B” means that “A and B are joint and several” unless otherwise expressly specified or indicated by context. Moreover, multiple instances of a resource, operation, or structure described herein as a single instance can be provided. Additionally, the boundaries between various resources, operations, engines, and data stores are somewhat arbitrary, and particular operations are shown in a context of specific illustrative configurations. Other allocations of functionality are envisioned and can fall within the scope of various embodiments of the present disclosure. In general, structures and functionality presented as separate resources in the example configurations can be implemented as a combined structure or resource. Similarly, structures and functionality presented as a single resource can be implemented as separate resources. These and other variations, modifications, additions, and improvements fall within the scope of the embodiments of the present disclosure as represented by the appended claims. The specification and drawings are, accordingly, to be regarded as illustrative rather than restrictive. For the purposes of United States patent practice, the contents of any related U.S. Patent Application, including any provisional patent application(s), naming the inventor(s) hereof are hereby expressly incorporated by reference in their entirety.

[0143] The terms “comprises”, “comprising”, “includes”, “including” or “contains”, “containing” are used to indicate the presence of stated features, integers, steps or components but not to the exclusion of others. Unless expressly indicated otherwise, or understood by context to mean otherwise, conditional language used herein, such as “can”, “could”, “might”, “may”, “e.g.”, or the like, typically is intended to convey that certain embodiments include, while other embodiments do not include, certain features, elements, and / or steps. Thus, such conditional language is not generally intended to imply that one or more embodiments are required to include a described feature, element and / or step or that one or more embodiments necessarily include logic for deciding, with or without user input or prompting, whether a described feature, element and / or step is included or is to be performed in any particular embodiment.

Claims

1. A computer-implemented method for risk detection, comprising: obtaining a first subset of a plurality of risk detection rules, the first subset being associated with a first coverage score and a first break score, wherein: the first coverage score indicates a number of unique historical transactions that have been correctly identified by a plurality of risk detection rules in the first subset, and the first break score indicates a number of unique historical transactions that have been incorrectly identified by a plurality of risk detection rules in the first subset; based on the first subset, constructing a lower bound data mapping that outputs an approximate coverage score for an input subset, wherein: when the input subset is the first subset, the approximate coverage score is the same as the first coverage score, and when the input subset is a second subset that is different from the first subset, the approximate coverage score is not greater than a second coverage score associated with the second subset, the second coverage score indicating a number of unique historical transactions that have been correctly identified by risk detection rules in the second subset; and based on the first subset, constructing an upper bound data mapping that includes a set of parameters, wherein: the upper bound data mapping outputs an approximate break score for an input subset of the plurality of risk detection rules, when the input subset is the first subset, the approximate break score is the same as the first break score, and when the input subset is a second subset that is different from the first subset, the approximate break score is not less than a second break score associated with the second subset, the second break score indicating a number of unique historical transactions that have been incorrectly identified as risk transactions by risk detection rules in the second subset; and based on the first subset, the lower bound data mapping, and the upper bound data mapping, generating a third subset of the plurality of risk detection rules based on at least: the approximate coverage score output by the lower bound data mapping corresponding to the plurality of risk detection rules as input, and the set of parameters associated with the upper bound data mapping, wherein the third subset is associated with a third coverage score indicating a number of unique historical transactions that have been correctly identified by a plurality of risk detection rules in the third subset; comparing the first coverage score to the third coverage score; and in response to the first coverage score exceeding the third coverage score, selecting rules in the first subset for use in risk detection of new transactions.

2. The method of claim 1, further comprising: in response to the first coverage score not exceeding the third coverage score, replacing the first subset with the third subset as an updated first subset, wherein the first coverage score is replaced with the third coverage score of the third subset accordingly; performing one or more iterations of a process including: updating the lower bound data mapping based on the updated first subset, based on the steps of constructing the lower bound data mapping and generating the third subset, iteratively until an exit condition is satisfied. generating an updated third subset associated with an updated third coverage score based on the updated first subset and the updated lower bound data mapping; and if the exit condition is not satisfied, replacing the updated first subset with the updated third subset and replacing the updated first coverage score with the updated third coverage score.

3. The method of claim 2, wherein, the exit condition comprises at least one of: the updated first coverage score is greater than the updated third coverage score, and a number of the one or more iterations is greater than a preset number.

4. The method of claim 1, wherein, the lower bound data mapping comprises a submodular function.

5. The method of claim 1, wherein, the first subset is empty.

6. The method of claim 1, wherein, constructing the lower bound data mapping comprises: generating a sequence by reordering the plurality of risk detection rules, wherein a risk detection rule in the first subset is placed first in the sequence; Based on the generated sequence, construct the time subset S i of the list 0≤i≤n, where: n is a number of the plurality of risk detection rules, a time subset S0 is empty, and For a given i, where 1 < i < n, the time subset S i includes the ith risk detection rule in the sequence and all risk detection rules in the time subset S i-1 . determining an approximate coverage score for each risk detection rule in the generated sequence; and determining a coverage score for a given subset of the plurality of risk detection rules as a sum of individual approximate coverage scores for each risk detection rule in the given subset.

7. The method of claim 6, wherein, the determining an individual approximate coverage score for each risk detection rule in the sequence comprises: For the i-th risk detection rule in the sequence, an individual approximation coverage score is determined based on a difference between a coverage score of the time subset S i and a coverage score of the time subset S i-1 , wherein the coverage score of the time subset S i and the coverage score of the time subset S i-1 are learned by querying a database of historical transactions.

8. The method of claim 1, wherein, constructing an upper bound data mapping comprising a parameter set, including determining the parameter set by: for each risk detection rule in the plurality of risk detection rules: determining a first approximate coverage score based on the lower bound data mapping; determining a first disruption score increase associated with adding the each risk detection rule to a first set of risk detection rules based on a number of unique historical transactions that have been falsely identified by the each risk detection rule; and determining a first ratio of the each risk detection rule, wherein the approximate coverage score is a numerator and the first disruption score increase is a denominator; generating a sequence by ordering the plurality of risk detection rules in descending order according to the determined first ratios of the plurality of risk detection rules; selecting a maximum number of risk detection rules whose first total disruption score increase is not greater than a preset threshold, wherein the first total disruption score is a sum of the first disruption score increases associated with the selected risk detection rules; and determining the parameter set as an intersection of the first subset and the selected risk detection rules.

9. The method of claim 8, wherein, if the each risk detection rule is not in the first subset, the first set is determined as the first subset, or if the each risk detection rule is in the first subset, the first set is determined as the first subset excluding the each risk detection rule. the generating a third subset of the plurality of risk detection rules comprises:

10. The method of claim 1, wherein, ordering the plurality of risk detection rules based on the first subset, the parameter set, and the approximate coverage scores generated by the lower bound data mapping for the plurality of risk detection rules; and the generating a third subset of the plurality of risk detection rules comprises: ordering the plurality of risk detection rules based on the first subset, the parameter set, and the approximate coverage scores generated by the lower bound data mapping for the plurality of risk detection rules; and selecting, from a beginning of the ordered plurality of risk detection rules, one or more consecutive risk detection rules as the third subset.

11. The method of claim 10, wherein, ordering the plurality of risk detection rules comprises: for each risk detection rule in the plurality of risk detection rules: determining a second approximate coverage score based on the lower bound data mapping; determining a second set of risk detection rules as the parameter set if the each risk detection rule is not in the first subset, or determining the second set of risk detection rules as the first subset excluding the each risk detection rule if the each risk detection rule is in the first subset; determining a second disruption score increase associated with adding the each risk detection rule to the second set of risk detection rules; and determining a second ratio of the each risk detection rule, wherein the second approximate coverage score is a numerator and the second disruption score increase is a denominator; generating a sequence by ordering the plurality of risk detection rules in descending order according to the second ratio of the each risk detection rule; and wherein selecting one or more consecutive risk detection rules as the third subset comprises: selecting a maximum number of risk detection rules whose second total disruption score increase is not greater than a pre-set threshold, wherein the second total disruption score is a sum of the second disruption score increase associated with the selected risk detection rules.

12. A system for risk detection comprising one or more processors and one or more non-transitory computer-readable memories coupled to the one or more processors and configured with instructions executable by the one or more processors to cause the system to perform operations comprising: obtaining a first subset of a plurality of risk detection rules, the first subset being associated with a first coverage score and a first disruption score, wherein: the first coverage score indicates a number of unique historical transactions that have been correctly identified by the plurality of risk detection rules in the first subset, and the first disruption score indicates a number of unique historical transactions that have been incorrectly identified by the plurality of risk detection rules in the first subset; based on the first subset, constructing a lower bound data mapping that outputs an approximate coverage score for an input subset, wherein: when the input subset is the first subset, the approximate coverage score is the same as the first coverage score, and when the input subset is a second subset different from the first subset, the approximate coverage score is not greater than a second coverage score associated with the second subset, the second coverage score indicating a number of unique historical transactions that have been correctly identified by risk detection rules in the second subset; and based on the first subset, constructing an upper bound data mapping comprising a parameter set, wherein: the upper bound data mapping outputs an approximate disruption score for an input subset of the plurality of risk detection rules, when the input subset is the first subset, the approximate disruption score is the same as the first disruption score, and when the input subset is a second subset different from the first subset, the approximate disruption score is not greater than a second disruption score associated with the second subset, the second disruption score indicating a number of unique historical transactions that have been incorrectly identified by risk detection rules in the second subset. when the input subset is a second subset different from the first subset, the approximate breakdown score is not less than a second breakdown score associated with the second subset, the second breakdown score indicating a number of unique historical transactions that have been incorrectly identified as risky transactions by risk detection rules in the second subset; and based on the first subset, the lower bound data mapping, and the upper bound data mapping, generating a third subset of the plurality of risk detection rules based on at least: the approximate coverage score output by the lower bound data mapping corresponding to the plurality of risk detection rules as input, and the set of parameters associated with the upper bound data mapping, wherein the third subset is associated with a third coverage score indicating a number of unique historical transactions that have been correctly identified by the plurality of risk detection rules in the third subset; comparing the first coverage score with the third coverage score; and in response to the first coverage score exceeding the third coverage score, selecting rules in the first subset for risk detection of new transactions.

13. The system of claim 12, wherein, The operations further include: in response to the first coverage score not exceeding the third coverage score, replacing the first subset with the third subset as an updated first subset, wherein the first coverage score is correspondingly replaced with the third coverage score of the third subset; performing one or more iterations of a process including: updating the lower bound data mapping based on the updated first subset; generating an updated third subset associated with an updated third coverage score based on the updated first subset and the updated lower bound data mapping; and replacing the updated first subset with the updated third subset and replacing the updated first coverage score with the updated third coverage score if the exit condition is not satisfied.

14. The system of claim 13, wherein, The exit condition includes at least one of: the updated first coverage score being greater than the updated third coverage score, and a number of the one or more iterations being greater than a preset number.

15. The system of claim 12, wherein, The lower bound data mapping includes a submodular function.

16. The system of claim 12, wherein, The first subset is empty.

17. A method for selecting a subset from a set of candidates, comprising: obtaining a first subset of a plurality of candidates, the first subset being associated with a first true positive score and a first false positive score, wherein: the first true positive score indicates a gain associated with a candidate in the first subset, and the first false positive score indicates a cost associated with a candidate in the first subset; constructing a lower bound data mapping based on the first subset, the lower bound data mapping outputting an approximate true positive score for an input subset, wherein: when the input subset is the first subset, the approximate true positive score is the same as the first true positive score, and the approximate true positive score is not less than a second breakdown score associated with a second subset different from the first subset, the second breakdown score indicating a number of unique historical transactions that have been incorrectly identified as risky transactions by risk detection rules in the second subset; and based on the first subset, the lower bound data mapping, and the upper bound data mapping, generating a third subset of the plurality of risk detection rules based on at least: the approximate coverage score output by the lower bound data mapping corresponding to the plurality of risk detection rules as input, and the set of parameters associated with the upper bound data mapping, wherein the third subset is associated with a third coverage score indicating a number of unique historical transactions that have been correctly identified by the plurality of risk detection rules in the third subset; comparing the first coverage score with the third coverage score; and in response to the first coverage score exceeding the third coverage score, selecting rules in the first subset for risk detection of new transactions. The operations further include: in response to the first coverage score not exceeding the third coverage score, replacing the first subset with the third subset as an updated first subset, wherein the first coverage score is correspondingly replaced with the third coverage score of the third subset; performing one or more iterations of a process including: updating the lower bound data mapping based on the updated first subset; generating an updated third subset associated with an updated third coverage score based on the updated first subset and the updated lower bound data mapping; and replacing the updated first subset with the updated third subset and replacing the updated first coverage score with the updated third coverage score if the exit condition is not satisfied. The exit condition includes at least one of: the updated first coverage score being greater than the updated third coverage score, and a number of the one or more iterations being greater than a preset number. The lower bound data mapping includes a submodular function. The first subset is empty. when the input subset is a second subset different from the first subset, the approximate false positive score is not less than a second false positive score associated with the second subset, the second false positive score indicating a gain associated with a candidate in the second subset; and based on the first subset, constructing an upper bound data mapping comprising a set of parameters, wherein: the upper bound data mapping outputs, for an input subset of the plurality of candidates, an approximate false positive score, when the input subset is the first subset, the approximate false positive score is the same as the first false positive score, and when the input subset is the second subset different from the first subset, the approximate false positive score is not less than a second false positive score associated with the second subset, the second false positive score indicating a gain associated with a candidate in the second subset; and based on the first subset, the lower bound data mapping, and the upper bound data mapping, generating a third subset of the plurality of candidates based on at least: the approximate true positive scores output by the lower bound data mapping corresponding to the plurality of candidates as input, and the set of parameters associated with the upper bound data mapping, wherein the third subset is associated with a third true positive score, the third true positive score indicating a gain associated with a plurality of candidates in the third subset; comparing the first true positive score with the third true positive score; and in response to the first true positive score exceeding the third true positive score, selecting a candidate in the first subset for a new transaction.

18. The method of claim 17, further comprising: in response to the first true positive score not exceeding the third true positive score, replacing the first subset with the third subset as an updated first subset, wherein the first true positive score is correspondingly replaced with the third true positive score of the third subset; based on the steps of constructing the lower bound data mapping and generating the third subset, performing one or more iterations of a process cyclically until a exit condition is satisfied, the process comprising: updating the lower bound data mapping based on the updated first subset; generating an updated third subset associated with an updated third true positive score based on the updated first subset and the updated lower bound data mapping; and if the exit condition is not satisfied, replacing the updated first subset with the updated third subset and replacing the updated first true positive score with the updated third true positive score.

19. The method of claim 18, wherein, the exit condition comprises at least one of: the updated first true positive score is greater than the updated third true positive score, and a number of the one or more iterations is greater than a preset number.

20. The method of claim 17, wherein, the first subset is empty.

Images