A risk control strategy evaluation method, system, electronic device and storage medium

Abstract

The present application provides a risk control strategy evaluation method, system, electronic device and storage medium. The method includes: obtaining configuration information, which includes a risk control strategy to be evaluated and an evaluation time interval. According to the configuration information, the decision logs of the online risk control strategy within the evaluation time interval can be automatically obtained. The decision logs include the first execution results of the online risk control strategy. Then, based on the decision logs, the risk control strategy to be evaluated is executed to obtain the second execution results of the risk control strategy to be evaluated. According to the first execution results and the second execution results of the risk control strategy to be evaluated, the gain information of the risk control strategy to be evaluated relative to the online risk control strategy is determined. In this way, the user only needs to input the configuration information to obtain the gain information of the risk control strategy to be evaluated relative to the current online risk control strategy, realizing the evaluation of the risk control strategy to be evaluated. Compared with manual evaluation, the evaluation efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular, provides a risk control strategy evaluation method, system, electronic device and storage medium. Background Art

[0002] With the development of Internet technology, the network is applied to more and more extensive fields. However, some users of online black production will download some online black production tools from the websites where the online black production tools are released. They will use these online black production tools to attack network devices (such as servers, personal computers, etc.). Such illegal acts will threaten social security and stability.

[0003] In order to prevent users of online black production from using online black production tools to attack network devices, risk control operation personnel need to set corresponding risk control strategies to prevent users of online black production from using online black production tools to attack network devices. However, the confrontation between risk control operation personnel and online black production is not a one-time thing, but a continuous process. Online black production usually tries new attack methods frequently. Therefore, risk control strategies also need to be updated frequently.

[0004] A risk control strategy is a collection of risk control rules. At present, the update of a risk control strategy usually involves updating multiple risk control rules of the risk control strategy. Before a new risk control rule is put on the line, it needs to be fully tested and evaluated to avoid configuration errors disturbing normal users or a large number of black production accounts being let go. Before the updated risk control strategy is put into operation, it is necessary to evaluate the gain of the new risk control rule on the effect of intercepting black production online.

[0005] The current evaluation methods used are all relatively manual, time-consuming and laborious, with high difficulty and cost, resulting in low efficiency of risk control strategy evaluation. Summary of the Invention

[0006] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides a risk control strategy evaluation method, system, electronic device and storage medium.

[0007] In a first aspect, the present application provides a risk control strategy evaluation method, including:

[0008] Obtaining configuration information, where the configuration information includes a risk control strategy to be evaluated and an evaluation time interval;

[0009] Based on the configuration information, obtaining decision logs of the online risk control strategy within the evaluation time interval, where the decision logs include first execution results of the online risk control strategy;

[0010] Based on the decision logs, executing the risk control strategy to be evaluated to obtain second execution results of the risk control strategy to be evaluated;

[0011] Determine the evaluation result of the risk control strategy to be evaluated according to the first execution result and the second execution result, where the evaluation result includes at least the gain information of the risk control strategy to be evaluated relative to the online risk control strategy, and the gain information is information indicating the gain generated by the risk control strategy to be evaluated relative to the online risk control strategy.

[0012] As a possible implementation, obtaining the decision log of the online risk control strategy within the evaluation time interval based on the configuration information includes:

[0013] If the maximum value of the evaluation time interval is less than the current time, obtain the decision log of the online risk control strategy within the evaluation time interval from the data warehouse;

[0014] If the minimum value of the evaluation time interval is greater than or equal to the current time, obtain the decision log of the online risk control strategy within the evaluation time interval from the message queue corresponding to the online risk control strategy.

[0015] As a possible implementation, obtaining the decision log of the online risk control strategy within the evaluation time interval based on the configuration information includes:

[0016] If the minimum value of the evaluation time interval is less than the current time and the maximum value is greater than or equal to the current time, form a first time interval with the part of the evaluation time interval that is less than the current time, and form a second time interval with the part that is greater than or equal to the current time;

[0017] Obtain the decision log of the online risk control strategy within the first time interval from the data warehouse;

[0018] Obtain the decision log of the online risk control strategy within the second time interval from the message queue corresponding to the online risk control strategy.

[0019] As a possible implementation, executing the risk control strategy to be evaluated based on the decision log to obtain the second execution result of the risk control strategy to be evaluated includes:

[0020] Obtain the dependent data of the risk control strategy to be evaluated;

[0021] Execute the risk control strategy to be evaluated based on the dependent data and the decision log to obtain the second execution result of the risk control strategy to be evaluated.

[0022] As a possible implementation, executing the risk control strategy to be evaluated based on the decision log to obtain the second execution result of the risk control strategy to be evaluated includes:

[0023] Compare the risk control strategy to be evaluated with the online risk control strategy to determine the first rules added and the second rules deleted in the risk control strategy to be evaluated compared to the online risk control strategy;

[0024] Determine the dependent data of the first rules;

[0025] Execute the first rules based on the dependent data and the decision logs to obtain the third execution results corresponding to the first rules;

[0026] Search for the fourth execution results corresponding to the second rules from the first execution results;

[0027] Delete the fourth execution results in the first execution results and add the third execution results to obtain the second execution results of the risk control strategy to be evaluated.

[0028] As a possible implementation, the method further includes:

[0029] Generate an evaluation report for the risk control strategy to be evaluated according to the evaluation result of the risk control strategy to be evaluated;

[0030] Store the evaluation report and send the evaluation report to a specified object.

[0031] In a second aspect, an embodiment of the present application further provides a risk control strategy evaluation system, including:

[0032] A configuration module, configured to obtain configuration information, where the configuration information includes a risk control strategy to be evaluated and an evaluation time interval;

[0033] A traffic replication module, configured to obtain decision logs of the online risk control strategy within the evaluation time interval based on the configuration information, where the decision logs include the first execution results of the online risk control strategy;

[0034] An engine module, configured to execute the risk control strategy to be evaluated based on the decision logs to obtain the second execution results of the risk control strategy to be evaluated;

[0035] An evaluation module, configured to determine the evaluation result of the risk control strategy to be evaluated according to the first execution results and the second execution results, where the evaluation result at least includes the gain information of the risk control strategy to be evaluated relative to the online risk control strategy.

[0036] As a possible implementation, the evaluation module is further configured to generate an evaluation report for the risk control strategy to be evaluated according to the evaluation result of the risk control strategy to be evaluated;

[0037] The system further includes: an evaluation report management module, configured to store the evaluation report and send the evaluation report to a specified object.

[0038] In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor and a memory, where the processor is configured to execute a risk control strategy evaluation program stored in the memory to implement the risk control strategy evaluation method described in the first aspect.

[0039] In a fourth aspect, an embodiment of the present application further provides a storage medium, where the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the risk control strategy evaluation method described in the first aspect.

[0040] The above technical solutions provided by the embodiments of the present application have the following advantages compared with the prior art:

[0041] A risk control strategy evaluation method provided by an embodiment of the present application obtains configuration information, where the configuration information includes a risk control strategy to be evaluated and an evaluation time interval. According to the configuration information, the decision-making logs of the online risk control strategy within the evaluation time interval can be automatically obtained. The decision-making logs include the first execution result of the online risk control strategy. Then, based on the decision-making logs, the risk control strategy to be evaluated is executed to obtain the second execution result of the risk control strategy to be evaluated. According to the first execution result and the second execution result of the risk control strategy to be evaluated, the gain information of the risk control strategy to be evaluated relative to the online risk control strategy is determined. In this way, the user only needs to input the configuration information to obtain the gain information of the risk control strategy to be evaluated relative to the current online risk control strategy, realizing the evaluation of the risk control strategy to be evaluated. Compared with manual evaluation, the evaluation efficiency is improved.

[0042] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application.

[0044] Figure 1 is a schematic diagram of a risk control strategy evaluation system shown according to an exemplary embodiment.

[0045] Figure 2 is a flowchart of a risk control strategy evaluation method shown according to an exemplary embodiment.

[0046] Figure 3 is a comparison diagram of a first execution result and a second execution result shown according to an exemplary embodiment.

[0047] Figure 4 is a two-dimensional representation diagram of index jump shown according to an exemplary embodiment.

[0048] Figure 5 is a block diagram of an electronic device shown according to an exemplary embodiment. Detailed implementation manners

[0049] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0050] A risk control strategy refers to a collection of risk control rules formulated by risk control operation personnel to eliminate or reduce risk events (such as attacks by online black production). Currently, before a risk control strategy goes live, in order to understand what benefits the upcoming risk control strategy will bring compared with the previous version of the risk control strategy, the upcoming risk control strategy is usually evaluated by manual testing. This evaluation method requires a lot of manpower and time, and the difficulty and cost are also very high, resulting in a slow iteration speed of the risk control strategy. Moreover, due to the long time-consuming of this evaluation method, during the period of fierce attacks by black production, there may be a situation where the risk control strategy is directly launched without sufficient evaluation because the situation is too serious, resulting in the existence of some risks.

[0051] To solve the above problems, the embodiments of the present application provide a risk control strategy evaluation method and system.

[0052] See Figure 1 , which is a schematic diagram of a risk control strategy evaluation system provided by an embodiment of the present application. The underlying storage configuration of the risk control strategy evaluation system is based on a database, and a friendly interactive background configuration page is provided. As Figure 1 shown, the system includes: a configuration module 101, a traffic replication module 102, an engine module 103, an evaluation module 104, and an evaluation report management module 105.

[0053] Among them, the configuration module 101 is mainly responsible for obtaining configuration information, where the configuration information includes but is not limited to the risk control strategy to be evaluated and the evaluation time interval, etc.

[0054] As an embodiment, the risk control strategy evaluation system can simultaneously evaluate multiple risk control strategies in parallel. Therefore, the configuration information obtained by the configuration module 101 may include multiple risk control strategies to be evaluated and the evaluation time interval.

[0055] The traffic replication module 102 is mainly responsible for copying the decision logs of the online risk control strategy according to the configuration information and importing them into the engine module 103.

[0056] As an example, the traffic replication module 102 divides the import of decision logs into the engine module 103 into offline import and near-real-time import, as Figure 1 shown. The offline import is mainly responsible for batch obtaining historical decision logs from the data warehouse according to the evaluation time interval and then controlling the concurrent import. The near-real-time import is responsible for obtaining decision logs from the message queue of the online risk control policy in near real time (with a second-level delay) and importing them. Among them, the online risk control policy refers to the policy currently running online for risk control management of the target business, where the target business is the business managed by the risk control policy to be evaluated. The decision logs of the online risk control policy include the actual passed parameters and data query results of the online risk control policy, and also include the execution results of the online risk control policy.

[0057] In the embodiment of the present application, the risk control policy is a collection of risk control rules, and the execution of each rule requires corresponding dependent data. For example, if the risk control rule is "intercept if the number of user login devices exceeds 30", the dependent data of this risk control rule is the user login device data. The passed parameters and data query results are the dependent data of the risk control rules in the risk control policy. Specifically, the passed parameters are the parameters directly passed by the business (such as IP, user ID, etc.), and the data query results are the labeled data queried according to the passed parameters such as user ID (such as the last login time of the current user ID or which mobile devices have been logged in, etc.). Among them, the passed parameters do not need to be queried so there is no time consumption, while the data query results need to query the labeled data so there is a time cost.

[0058] As an example, the risk control policy evaluation system can simultaneously evaluate multiple risk control policies in parallel, and the decision logs required for the evaluation of different risk control policies may be different. Therefore, the traffic replication module 102 can simultaneously import the decision logs of multiple online risk control policies into the engine module. Specifically, the traffic replication module 102 can start a traffic replication task for each online risk control policy respectively. The tasks can run in parallel, but due to limited service resources, at most 10 tasks can be supported simultaneously, and the excess will enter the queuing waiting queue. During the running of the tasks, the risk control operation personnel can view the running status of the tasks and can also sample the data.

[0059] The engine module 103 is mainly responsible for executing the risk control policy to be evaluated according to the decision logs of the online risk control policy corresponding to the risk control policy to be evaluated, so as to obtain the execution result corresponding to the risk control policy to be evaluated, and store the execution results of the online risk control policy and the risk control policy to be evaluated in the data warehouse.

[0060] In the embodiment of the present application, the engine module 103 supports parallel execution of multiple risk control policies to be evaluated.

[0061] The evaluation module 104 is mainly responsible for obtaining the execution results of the online risk control strategy and the execution results of the risk control strategy to be evaluated from the data warehouse, and then evaluating the risk control strategy to be evaluated based on the execution results of the online risk control strategy and the execution results of the risk control strategy to be evaluated, obtaining an evaluation result, where the evaluation result at least includes the gain information of the risk control strategy to be evaluated relative to the online risk control strategy. Further, an evaluation report can be generated according to the evaluation result.

[0062] The evaluation report management module 105 is mainly responsible for archiving the evaluation report and sending a message to a specified object (such as a risk control operation personnel) for notification, so that the risk control operation personnel can decide whether to directly put the risk control strategy to be evaluated online or adjust the risk control strategy to be evaluated and then conduct a new round of evaluation according to the evaluation report.

[0063] Using the above risk control strategy evaluation system can realize the configuration-based pre-online evaluation of the risk control strategy, support one-key export of the evaluation report, thereby improving the efficiency of the risk control strategy evaluation. Moreover, it supports parallel evaluation of multiple risk control strategies, further improving the evaluation efficiency of the risk control strategy and also increasing the update speed of the risk control strategy, so as to more quickly counter new attack methods of online black production.

[0064] The following describes the risk control strategy evaluation method provided in the embodiments of the present application in combination with the above risk control strategy evaluation system.

[0065] See Figure 2 , which is a schematic flowchart of a risk control strategy evaluation method provided in the embodiments of the present application. This method can be applied to Figure 1 the risk control strategy evaluation system shown in Figure 2 As shown, the method may include the following steps:

[0066] S21. Obtain configuration information, where the configuration information includes the risk control strategy to be evaluated and the evaluation time interval.

[0067] In the embodiments of the present application, the risk control strategy evaluation system can simultaneously evaluate multiple risk control strategies in parallel. The evaluations of multiple risk control strategies are independent of each other and have the same evaluation method. Therefore, in the embodiments of the present application, an example of evaluating one risk control strategy (hereinafter referred to as the risk control strategy to be evaluated) is used for illustration.

[0068] In the embodiments of the present application, the evaluation time interval refers to the time corresponding to the data required for evaluating the risk control strategy to be evaluated. For example, if it is necessary to evaluate the operation of the risk control strategy to be evaluated from October 1, 2021 to November 1, 2021, then the corresponding data between October 1, 2021 and November 1, 2021 needs to be obtained. In this way, October 1, 2021 to November 1, 2021 is the evaluation time interval.

[0069] In practical applications, the configuration information can be obtained through Figure 1 the configuration module 101 shown. Specifically, the configuration module 101 can display an information configuration interface, and an object (such as a risk control operation personnel) can input the corresponding configuration information in the information configuration interface, and then obtain the configuration information.

[0070] As an embodiment, multiple versions of risk control strategies can be pre-stored. In this way, when the object inputs the configuration information, it can only input the version information of the risk control strategy to be evaluated, and then directly obtain the corresponding risk control strategy to be evaluated from the multiple pre-stored risk control strategies according to the version information input by the object. By this method, the operation of the object can be more simplified and the user experience can be improved.

[0071] Generally, risk control strategies correspond to services (such as login, registration, voting, etc.), and different services may correspond to different risk control strategies. Based on this, if Figure 1 the risk control strategy evaluation system shown is only used to evaluate the risk control strategy corresponding to one service, then the service corresponding to the risk control strategy to be evaluated can be determined according to the used risk control strategy evaluation system. Therefore, the configuration information obtained only includes the risk control strategy to be evaluated and the evaluation time interval. However, if Figure 1 the risk control strategy evaluation system shown can evaluate the risk control strategies of multiple different services, then in order to evaluate more accurately, the service corresponding to the risk control strategy to be evaluated can also be included in the obtained configuration information.

[0072] S22. Obtain the decision log of the online risk control strategy within the evaluation time interval based on the configuration information, and the decision log includes the first execution result of the online risk control strategy.

[0073] In the embodiment of the present application, after the configuration information is obtained, the configuration information can be verified, and S22 is executed after the verification passes. The verification can be to detect whether each item of information in the configuration information is empty. If any information is empty, it is determined that the verification fails, otherwise it is determined that the verification passes.

[0074] In the embodiment of the present application, the online risk control strategy can also be determined before executing S22. The online risk control strategy is the risk control strategy corresponding to the target service that is currently running, where the target service is the service corresponding to the risk control strategy to be evaluated. Therefore, the online risk control strategy can be determined according to the service corresponding to the risk control strategy to be evaluated. After the online risk control strategy is determined, the decision log of the online risk control strategy can be obtained according to the evaluation time interval in the configuration information. The time included in the evaluation time interval can be earlier than the current time, or equal to or later than the current time.

[0075] In practical applications, Figure 1The shown traffic replication module 102 obtains the decision logs of the online risk control policy within the evaluation time interval by starting a traffic replication task. During the operation of the traffic replication task, the traffic replication module 102 can display the task operation status, so that the risk control operation personnel can view the task operation status. At the same time, it also supports the risk control operation personnel to sample and view the data.

[0076] In the embodiment of the present application, for the decision logs earlier than the current time, they can be directly obtained from the data warehouse in an offline import manner. For the decision logs equal to or later than the current time, they can be obtained in a near-real-time import manner in real time (second-level delay) from the message queue corresponding to the online risk control policy.

[0077] Specifically, before obtaining the decision logs of the online risk control policy, it can be first determined whether the evaluation time interval is earlier than the current time. If the maximum value of the evaluation time interval is less than the current time, it is determined that the evaluation time interval is earlier than the current time, and then the decision logs of the online risk control policy within the evaluation time interval are obtained from the data warehouse. If the minimum value of the evaluation time interval is greater than or equal to the current time, it is determined that the evaluation time interval is not earlier than the current time, and then the decision logs of the online risk control policy within the evaluation time interval are obtained from the message queue corresponding to the online risk control policy.

[0078] Furthermore, it is possible that part of the evaluation time interval is earlier than the current time and part of it is equal to or greater than the current time, that is, the minimum value of the evaluation time interval is less than the current time and the maximum value is greater than or equal to the current time. In this case, the evaluation time interval can be split based on the current time. The part of the evaluation time interval that is less than the current time is formed into a first time interval, and the part that is greater than or equal to the current time is formed into a second time interval. Then, the decision logs of the online risk control policy within the first time interval are obtained from the data warehouse, and the decision logs of the online risk control policy within the second time interval are obtained from the message queue corresponding to the online risk control policy.

[0079] Through the above method, not only can the historical decision logs of the online risk control policy be obtained, but also the real-time decision logs of the online risk control policy can be obtained, so that the evaluation time interval can be flexibly set according to actual needs.

[0080] S23. Based on the decision logs, execute the risk control policy to be evaluated to obtain the second execution result of the risk control policy to be evaluated.

[0081] As an alternative implementation, the dependent data of the risk control policy to be evaluated can be obtained, and then based on the dependent data and the decision log, the risk control policy to be evaluated is executed to obtain the second execution result of the risk control policy to be evaluated. Among them, the dependent data of the risk control policy to be evaluated is the dependent data of all risk control rules in the risk control policy to be evaluated. Obtaining the dependent data of the risk control policy to be evaluated can be to query the corresponding tag data according to the parameters passed in the decision log. Through this method, an accurate execution result can be obtained.

[0082] As another alternative implementation, the risk control policy to be evaluated and the online risk control policy can be compared first to determine the first rules added and the second rules deleted in the risk control policy to be evaluated compared with the online risk control policy; determine the dependent data of the first rules; based on the dependent data and the decision log, execute the first rules to obtain the third execution result corresponding to the first rules; find the fourth execution result corresponding to the second rules from the first execution result; delete the fourth execution result in the first execution result and add the third execution result to obtain the second execution result of the risk control policy to be evaluated. Through this method, only the dependent data of the newly added first rules needs to be queried, that is, only the newly dependent data is queried, and the data contained in the decision log is no longer queried, avoiding re-querying data, reducing resource consumption and improving the evaluation speed. Usually, the business traffic undertaken by business risk control is particularly large, and less resource consumption can improve the resource utilization rate of risk control services. At the same time, the evaluation cost and time can be saved.

[0083] Further, after obtaining the second execution result, the first execution result and the second execution result can both be stored in the data warehouse for subsequent direct acquisition.

[0084] S24. Determine the evaluation result of the risk control policy to be evaluated according to the first execution result and the second execution result. The evaluation result at least includes the gain information of the risk control policy to be evaluated relative to the online risk control policy.

[0085] In the embodiment of the present application, after obtaining the second execution result, the first execution result and the second execution result can be compared to determine the differences between them, and then the gain information of the risk control policy to be evaluated relative to the online risk control policy can be determined according to the differences. Among them, the gain information is information indicating the gain generated by the risk control policy to be evaluated relative to the online risk control policy. For example, it can include the change information of the risk level and the change information of the risk interception rate when the risk control policy to be evaluated is compared with the online risk control policy.

[0086] In practical applications, the Figure 1 evaluation module 104 shown can be used to determine the evaluation result of the risk control policy to be evaluated and generate a corresponding evaluation report according to the evaluation result.

[0087] As an example, the generated evaluation report may include a comparison chart of the first execution result and the second execution result and a two-dimensional table of index jumps. The horizontal axis of the two-dimensional table of index jumps is each index in the first execution result, the vertical axis is each index in the second execution result, the values in the table are the jump amounts corresponding to each index, and the proportion of the jump amount is identified by the depth of color. For example, the darker the color, the larger the proportion, or the lighter the color, the larger the proportion.

[0088] An example:

[0089] Taking the execution results of the risk control strategy including indicators such as the number of requests, the number of requests included in each risk level, and the interception rate as an example,

[0090] As Figure 3 shown, it is a comparison chart of the first execution result and the second execution result. As Figure 3 shown, among them, the first row is the first execution result, the second row is the second execution result. Among them, the number of requests in both is the same, both are 27. The number of risk-free requests in the first execution result is 4, and the number of risk-free requests in the second execution result is 7, which is a 75% increase compared to the first execution result; the number of low-risk requests in the first execution result is 0, and the number of low-risk requests in the second execution result is 0; the number of medium-risk requests in the first execution result is 23, and the number of medium-risk requests in the second execution result is 20, which is a 13.043% decrease compared to the first execution result; the number of high-risk requests in the first execution result is 0, and the number of high-risk requests in the second execution result is 0; the interception rate in the first execution result is 85.185%, and the interception rate in the second execution result is 74.074%.

[0091] As Figure 4 shown, it is a two-dimensional table of risk level jumps. Among them, the horizontal axis is each risk level in the first execution result, the vertical axis is each risk level in the second execution result, the values in the table are the jump request amounts, and the proportion of the request number is identified by the depth of color. As Figure 4 shown, there are 20 requests that are both marked as risk level 2 (i.e., medium risk) in the first execution result and the second execution result, and there are 4 requests that are both marked as risk level 0 (i.e., risk-free) in the first execution result and the second execution result. There are 3 requests that are marked as risk level 0 in the pre-online strategy but marked as risk level 2 in the real-time strategy.

[0092] Furthermore, after obtaining the evaluation report, the evaluation report can also be stored and sent to a specified object, such as a risk control operator, so that the risk control operator can determine the gain information of the risk control strategy to be evaluated relative to the online risk control strategy according to the evaluation report.

[0093] In practical applications, it can be adopted Figure 1The evaluation report management module 105 shown stores the evaluation report in the data warehouse and sends it to the risk control operation staff, so that the risk control operation staff can decide whether to directly put the risk control strategy to be evaluated online or adjust the risk control strategy to be evaluated and then conduct a new round of evaluation according to the evaluation report. The adjustment can be to adjust the threshold value in the risk control rule. For example, the current risk control rule is "intercept if the user has logged in on n devices", and the value of n needs to go through multiple rounds of evaluation. If the results of each evaluation are not very different, generally the evaluation time interval will also be extended. Only after multiple rounds of evaluation will the value of n be relatively reasonable. In addition, the reasonable management of the evaluation report is conducive to the risk control operation to query and compare the evaluation results of several risk control strategies. The risk control operation staff can also summarize the evaluation methods at a lower cost and improve the evaluation efficiency.

[0094] A risk control strategy evaluation method provided by an embodiment of the present application obtains configuration information, which includes a risk control strategy to be evaluated and an evaluation time interval. According to the configuration information, the decision log of the online risk control strategy within the evaluation time interval can be automatically obtained. The decision log includes the first execution result of the online risk control strategy. Then, based on the decision log, the risk control strategy to be evaluated is executed to obtain the second execution result of the risk control strategy to be evaluated. According to the first execution result and the second execution result of the risk control strategy to be evaluated, the gain information of the risk control strategy to be evaluated relative to the online risk control strategy is determined. In this way, the user only needs to input the configuration information to obtain the gain information of the risk control strategy to be evaluated relative to the current online risk control strategy, realizing the evaluation of the risk control strategy to be evaluated. Compared with manual evaluation, the evaluation efficiency is improved.

[0095] Furthermore, the embodiment of the present application also supports one-key export of the evaluation report, which is convenient for comparing the risk jump situation and strategy gain according to the evaluation report, improving the evaluation efficiency of the new risk control strategy of the risk control operation. The underlying service resource utilization rate is high, parallel evaluation is supported, the risk control strategy update speed is improved, and new attack methods of black production can be countered more quickly.

[0096] See Figure 5 For the schematic diagram of an electronic device provided by this embodiment, as Figure 5 shown, the electronic device includes: at least one processor 501, a memory 502, at least one network interface 503, and other user interfaces 504. Each component in the electronic device 500 is coupled together through a bus system 505. It can be understood that the bus system 505 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 505 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in Figure 5 all kinds of buses are labeled as the bus system 505.

[0097] Among them, the user interface 504 may include a display, a keyboard, or a pointing device (such as a mouse, a trackball, a touchpad, or a touch screen, etc.).

[0098] It can be understood that the memory 502 in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synch link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM). The memory 502 described herein is intended to include but not be limited to these and any other suitable types of memory.

[0099] In some embodiments, the memory 502 stores the following elements, executable units, or data structures, or subsets thereof, or extended sets thereof: an operating system 5021 and a second application 5022.

[0100] Among them, the operating system 5021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The second application 5022 includes various second applications, such as a media player and a browser, etc., for implementing various application services. The program for implementing the method of the embodiments of the present invention may be included in the second application 5022.

[0101] In an embodiment of the present invention, by invoking a program or instruction stored in the memory 502, specifically, a program or instruction stored in the second application program 5022, the processor 501 is configured to execute the method steps provided in each method embodiment, for example, including:

[0102] Obtain configuration information, where the configuration information includes a risk control strategy to be evaluated and an evaluation time interval;

[0103] Based on the configuration information, obtain a decision log of the online risk control strategy within the evaluation time interval, where the decision log includes a first execution result of the online risk control strategy;

[0104] Based on the decision log, execute the risk control strategy to be evaluated to obtain a second execution result of the risk control strategy to be evaluated;

[0105] According to the first execution result and the second execution result, determine an evaluation result of the risk control strategy to be evaluated, where the evaluation result at least includes gain information of the risk control strategy to be evaluated relative to the online risk control strategy.

[0106] The method disclosed in the above embodiment of the present invention can be applied to or implemented by the processor 501. The processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor 501 or an instruction in the form of software. The above processor 501 may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present invention can be directly embodied as being executed by a hardware decoding processor or executed by a combination of hardware and software units in the decoding processor. The software unit may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502 and combines its hardware to complete the steps of the above method.

[0107] It will be appreciated that the embodiments described herein can be implemented using hardware, software, firmware, middleware, microcode, or any combination thereof. For a hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions of the present application, or any combination thereof.

[0108] For a software implementation, the techniques herein can be implemented by units that execute the functions herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented within the processor or externally to the processor.

[0109] Embodiments of the present invention also provide a storage medium (computer-readable storage medium). The storage medium stores one or more programs. Here, the storage medium can include volatile memory, such as random access memory; the memory can also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; the memory can also include a combination of the above types of memory.

[0110] When one or more programs in the storage medium can be executed by one or more processors to implement the risk control strategy evaluation method performed on the electronic device side as described above.

[0111] The processor is used to execute the risk control strategy evaluation program stored in the memory to implement the following steps of the risk control strategy evaluation method performed on the electronic device side:

[0112] Obtain configuration information, where the configuration information includes the risk control strategy to be evaluated and the evaluation time interval;

[0113] Based on the configuration information, obtain the decision log of the online risk control strategy within the evaluation time interval, where the decision log includes the first execution result of the online risk control strategy;

[0114] Based on the decision log, execute the risk control strategy to be evaluated to obtain the second execution result of the risk control strategy to be evaluated;

[0115] Determine the evaluation result of the risk control strategy to be evaluated according to the first execution result and the second execution result, where the evaluation result at least includes the gain information of the risk control strategy to be evaluated relative to the online risk control strategy.

[0116] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0117] It can be understood that the same or similar parts in the above embodiments can be referred to each other, and the content not described in detail in some embodiments can be seen in the same or similar content in other embodiments.

[0118] It should be noted that in the description of the present application, the terms "first", "second", etc. are only used for descriptive purposes and cannot be construed as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise specified, the meaning of "a plurality of" refers to at least two.

[0119] Any process or method description shown in the flowchart or described in other ways herein can be understood to represent a module, segment, or part of code including one or more executable instructions for implementing a specific logical function or process. The scope of the preferred embodiments of the present application includes additional implementations, where the functions may be executed in a substantially simultaneous manner or in a reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the technical field of the embodiments of the present application.

[0120] It should be understood that each part of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following well-known technologies in the art or a combination thereof can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0121] Those of ordinary skill in the technical field of the present application can understand that all or part of the steps carried by the method of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0122] In addition, each functional unit in various embodiments of the present application may be integrated into a processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0123] The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disc, or the like.

[0124] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0125] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A risk control strategy evaluation method, characterized in that, Including: Obtain configuration information, where the configuration information includes the risk control strategy to be evaluated and the evaluation time interval; Based on the configuration information, obtain the decision logs of the online risk control strategy within the evaluation time interval, including: if the maximum value of the evaluation time interval is less than the current time, obtain the decision logs of the online risk control strategy within the evaluation time interval from the data warehouse; if the minimum value of the evaluation time interval is greater than or equal to the current time, obtain the decision logs of the online risk control strategy within the evaluation time interval from the message queue corresponding to the online risk control strategy, and the decision logs include the first execution result of the online risk control strategy; Based on the decision logs, execute the risk control strategy to be evaluated to obtain the second execution result of the risk control strategy to be evaluated; According to the first execution result and the second execution result, determine the evaluation result of the risk control strategy to be evaluated, and the evaluation result at least includes the gain information of the risk control strategy to be evaluated relative to the online risk control strategy, where the gain information is information indicating the gain generated by the risk control strategy to be evaluated relative to the online risk control strategy.

2. The method according to claim 1, characterized in that The obtaining the decision logs of the online risk control strategy within the evaluation time interval based on the configuration information includes: If the minimum value of the evaluation time interval is less than the current time and the maximum value is greater than or equal to the current time, then form a first time interval with the part of the evaluation time interval that is less than the current time, and form a second time interval with the part that is greater than or equal to the current time; Obtain the decision logs of the online risk control strategy within the first time interval from the data warehouse; Obtain the decision logs of the online risk control strategy within the second time interval from the message queue corresponding to the online risk control strategy.

3. The method according to claim 1, wherein The executing the risk control strategy to be evaluated based on the decision logs to obtain the second execution result of the risk control strategy to be evaluated includes: Obtain the dependent data of the risk control strategy to be evaluated; Based on the dependent data and the decision logs, execute the risk control strategy to be evaluated to obtain the second execution result of the risk control strategy to be evaluated.

4. The method according to claim 1, characterized in that, The executing the risk control strategy to be evaluated based on the decision logs to obtain the second execution result of the risk control strategy to be evaluated includes: Compare the risk control strategy to be evaluated with the online risk control strategy to determine the first rules newly added and the second rules deleted by the risk control strategy to be evaluated compared to the online risk control strategy; Determine the dependent data of the first rules; Based on the dependent data and the decision logs, execute the first rules to obtain the third execution result corresponding to the first rules; Search for the fourth execution result corresponding to the second rules from the first execution result; Delete the fourth execution result in the first execution result and add the third execution result to obtain the second execution result of the risk control strategy to be evaluated.

5. The method according to claim 1, wherein The method further includes: Generate an evaluation report for the risk control strategy to be evaluated according to the evaluation result of the risk control strategy to be evaluated; Store the evaluation report and send the evaluation report to a specified object.

6. A risk control strategy evaluation system, characterized in that, Including: Configuration module, configured to obtain configuration information, where the configuration information includes the risk control strategy to be evaluated and the evaluation time interval; Traffic replication module, configured to obtain the decision logs of the online risk control strategy within the evaluation time interval based on the configuration information, including: if the maximum value of the evaluation time interval is less than the current time, obtain the decision logs of the online risk control strategy within the evaluation time interval from the data warehouse; if the minimum value of the evaluation time interval is greater than or equal to the current time, obtain the decision logs of the online risk control strategy within the evaluation time interval from the message queue corresponding to the online risk control strategy, and the decision logs include the first execution result of the online risk control strategy; Engine module, configured to execute the risk control strategy to be evaluated based on the decision logs to obtain the second execution result of the risk control strategy to be evaluated; Evaluation module, configured to determine the evaluation result of the risk control strategy to be evaluated according to the first execution result and the second execution result, and the evaluation result at least includes the gain information of the risk control strategy to be evaluated relative to the online risk control strategy.

7. The system according to claim 6, characterized in that, The evaluation module is further configured to generate an evaluation report of the risk control strategy to be evaluated according to the evaluation result of the risk control strategy to be evaluated; The system further includes: an evaluation report management module, configured to store the evaluation report and send the evaluation report to a specified object.

8. An electronic device, characterized in that, Including: A processor and a memory, where the processor is configured to execute a risk control strategy evaluation program stored in the memory to implement the risk control strategy evaluation method according to any one of claims 1-5.

9. A storage medium, characterized in that, The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the risk control strategy evaluation method according to any one of claims 1-5.

Images