Automated verification method and apparatus

Abstract

The application discloses an automatic verification method and device, and relates to the technical field of artificial intelligence. A specific embodiment of the method comprises the following steps: obtaining detail information of a plurality of risk control test cases input in a pre-created case template; determining a plurality of target risk control test cases to be sent in the case template, generating a risk control test request, and sending the target risk control test cases to a pre-established decision engine in batches based on the generated risk control test request; wherein the decision engine comprises a plurality of pre-configured risk control rules; receiving a risk control decision result returned by the decision engine for the risk control test request, and comparing the detail information of the target risk control test cases with the risk control decision result to obtain a verification result of the risk control rules. The embodiment can improve the verification efficiency of the risk control rules in a batched and automatic test manner.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an automated verification method and apparatus. Background Technology

[0002] Business systems often require real-time risk control for specific business processes. The server executes corresponding risk control strategies by setting numerous risk control rules, thus necessitating testing the effectiveness and accuracy of these rules. Currently, risk control rules are primarily verified manually, one by one. However, due to the numerous actual risk control scenarios, each involving a large number of rules, manual verification is inefficient and fails to meet business needs. Furthermore, when testing involves third-party data, testers need to manually input the necessary data, further reducing testing efficiency. Additionally, triggering certain risk control rules involving alerts may require accumulated thresholds for metrics such as amount and number of transactions. Current technology can only achieve this by executing a large number of interactions within a short period, potentially leading to communication congestion and further reducing testing efficiency. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide an automated verification method and apparatus, which can improve the testing efficiency of risk control rules through batch automated testing.

[0004] To achieve the above objectives, according to one aspect of the present invention, an automated verification method is provided.

[0005] The automated verification method of this invention includes: obtaining detailed information of multiple risk control test cases input in a pre-created case template; determining multiple target risk control test cases to be sent in the case template, generating risk control test requests, and sending the target risk control test cases in batches to a pre-established decision engine based on the generated risk control test requests; wherein the decision engine contains multiple pre-configured risk control rules; receiving the risk control decision results returned by the decision engine for the risk control test requests, and comparing the detailed information of the target risk control test cases with the risk control decision results to obtain the verification results of the risk control rules.

[0006] Optionally, the details of each risk control test case include: case identifier, test subject data, expected result data, and internal interface data; and the generation of the risk control test request includes: generating the risk control test request using the internal interface data in the target risk control test case; wherein the risk control test request carries the case identifier and test subject data of each target risk control test case.

[0007] Optionally, the risk control decision result includes the target risk control rule identifier triggered by the test subject data in any target risk control test case; and the comparison of the details of the target risk control test case with the risk control decision result to obtain the verification result of the risk control rule includes: comparing the expected result data in the target risk control test case with the target risk control rule identifier to obtain the verification result of the risk control rule.

[0008] Optionally, the details of each risk control test case further include a risk control scenario identifier, the decision engine includes a decision platform corresponding to each risk control scenario, each decision platform contains multiple risk control rules corresponding to the same risk control scenario; and after the target risk control test case enters the decision engine, it is sent by the decision engine to the decision platform corresponding to the same risk control scenario to execute the triggering of the risk control rules.

[0009] Optionally, the risk control test case includes an early warning case, wherein the test subject data of the early warning case is associated with the numerical requirements of a preset specific indicator, and the early warning case further includes historical indicator accumulation logic for real-time statistical analysis of the specific indicator value; and the method further includes: using the historical indicator accumulation logic in any early warning case to statistically analyze the current specific indicator value; and in response to the statistically analyzed specific indicator value meeting the specific indicator value requirements corresponding to the early warning case, sending the early warning case to the decision engine along with the risk control test request.

[0010] Optionally, the details of the risk control test case further include external decision data to be filled and an external service call path for obtaining the external decision data; and the method further includes: before generating a risk control test request using the internal interface data in the target risk control test case, generating an external data acquisition request based on the external service call path and sending it to the corresponding external service to obtain the external decision data; the risk control test request further carries the external decision data that has been filled in the target risk control test case.

[0011] Optionally, the expected result data in the risk control test case includes the identifier of the risk control rule to be triggered; and the step of comparing the expected result data in the target risk control test case with the target risk control rule identifier to obtain the verification result of the risk control rule includes: determining the expected hit case identifier of each risk control rule based on the expected trigger risk control rule identifier in the target risk control test case, obtaining the actual hit case identifier of each risk control rule from the target risk control rule identifier; determining whether the expected hit case identifier of the same risk control rule is consistent with the actual hit case identifier: if so, determining that the verification of the risk control rule has passed; otherwise, determining that the verification of the risk control rule has failed.

[0012] Optionally, the expected result data in the risk control test case further includes the expected decision result, and the risk control decision result further includes the actual decision result corresponding to any target risk control test case and the target risk control rule triggered by the target risk control test case; and the method further includes: determining whether the expected decision result and the actual decision result of the same risk control rule corresponding to each target risk control test case are consistent: if so, determining that the verification of the risk control rule has passed; otherwise, determining that the verification of the risk control rule has failed.

[0013] Optionally, determining the multiple target risk control test cases to be sent in the case template includes: determining all risk control test cases in the case template as the target risk control test cases; or, determining risk control test cases in the case template whose case identifiers are within the input identifier range as the target risk control test cases; or, determining risk control test cases under the input specific risk control scenario as the target risk control test cases.

[0014] Optionally, the risk control decision result further includes: detailed data of the decision reason for any target risk control test case triggering any target risk control rule; and the detailed information of each risk control test case further includes: index data for querying the risk control test case.

[0015] To achieve the above objectives, according to another aspect of the present invention, an automated verification device is provided.

[0016] The automated verification device of this invention may include: a case import unit, configured to acquire detailed information of multiple risk control test cases input in a pre-created case template; a test request unit, configured to determine multiple target risk control test cases to be sent in the case template, generate a risk control test request, and send the target risk control test cases in batches to a pre-established decision engine based on the generated risk control test request; wherein the decision engine contains multiple pre-configured risk control rules; and a comparison unit, configured to receive the risk control decision result returned by the decision engine for the risk control test request, compare the detailed information of the target risk control test cases with the risk control decision result to obtain the verification result of the risk control rules.

[0017] Optionally, the details of each risk control test case include: case identifier, test subject data, expected result data, and internal interface data. The risk control test request carries the case identifier and test subject data of each target risk control test case. The risk control decision result includes the target risk control rule identifier triggered by the test subject data in any target risk control test case. Furthermore, the test request unit is further configured to: generate a risk control test request using the internal interface data in the target risk control test case; and the comparison unit is further configured to: compare the expected result data in the target risk control test case with the target risk control rule identifier to obtain the verification result of the risk control rule.

[0018] To achieve the above objectives, according to another aspect of the present invention, an electronic device is provided.

[0019] An electronic device according to the present invention includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the automated verification method provided by the present invention.

[0020] To achieve the above objectives, according to another aspect of the present invention, a computer-readable storage medium is provided.

[0021] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the automated verification method provided by the present invention.

[0022] To achieve the above objectives, according to another aspect of the present invention, a computer program product is provided.

[0023] A computer program product according to an embodiment of the present invention includes a computer program that, when executed by a processor, implements the automated verification method provided in the embodiment of the present invention.

[0024] According to the technical solution of the present invention, the embodiments described above have the following advantages or beneficial effects:

[0025] The testing tool's batch processing and automatic request generation functions enabled automated batch testing of multiple risk control rules, improving testing efficiency. When testing involves third-party data, the tool automatically retrieves the necessary external decision data through pre-defined external service call paths in the test cases, avoiding manual data collection in existing technologies and further enhancing efficiency. For testing alert-related rules, historical metric accumulation logic is pre-deployed in the corresponding alert cases, and the accumulated metric status is statistically analyzed in real time. When the accumulated metric reaches a threshold, the testing tool automatically sends the corresponding alert case to execute the test, thus avoiding the communication congestion caused by a large number of interactions executed in a short period of time in existing technologies, further improving testing efficiency without affecting test accuracy.

[0026] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0027] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0028] Figure 1 This is a schematic diagram of the main steps of the automated verification method in this embodiment of the invention;

[0029] Figure 2 This is a schematic diagram illustrating the specific execution steps of the automated verification method in this embodiment of the invention;

[0030] Figure 3 This is a schematic diagram of the components of the automated verification device in an embodiment of the present invention;

[0031] Figure 4 This is an exemplary system architecture diagram that can be applied thereto according to embodiments of the present invention;

[0032] Figure 5 This is a schematic diagram of the electronic device structure used to implement the automated verification method in the embodiments of the present invention. Detailed Implementation

[0033] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0034] It should be noted that, unless otherwise specified, the embodiments of the present invention and the technical features thereof can be combined with each other.

[0035] Figure 1 This is a schematic diagram of the main steps of the automated verification method according to an embodiment of the present invention.

[0036] like Figure 1 As shown, the automated verification method of this invention can be executed by a pre-established testing tool. In specific applications, risk managers define risk control rules to control risks in different situations, thereby reducing the probability of risk events or minimizing losses when they occur. For example, in a program's registration and login scenario, the following risk control rule can be defined: if the same mobile number obtains more than 3 verification codes within 1 minute (trigger condition), then a certain restriction operation (execution strategy) is executed. The specific execution steps of the automated verification method of this invention are as follows:

[0037] Step S101: Obtain detailed information of multiple risk control test cases entered in the pre-created case template.

[0038] In this step, the case template is a pre-written template file in a specific format used to batch import risk control test cases. Its format can be any applicable format such as Excel or XML. The details of each risk control test case can include: case identifier, test subject data, expected result data, and internal interface data. The case identifier is used to uniquely distinguish different risk control test cases. Test subject data refers to the test input data entered into the decision engine to trigger risk control rules. For example, in the previous registration / login scenario, test subject data could be the number of verification codes obtained by a mobile phone number within one minute or the time of each verification code obtained within a recent period. Expected result data refers to the data used to compare with the risk control decision results returned by the decision engine in response to the risk control test request to arrive at the final risk control rule verification conclusion. Internal interface data refers to the interface data such as the Internet Protocol IP address and domain name used by the testing tool to access the decision engine. In this embodiment of the invention, the decision engine refers to a system used to execute risk control processing according to risk control rules. It includes decision platforms corresponding to various risk control scenarios, and each decision platform contains multiple risk control rules corresponding to the same risk control scenario. In practical applications, the details of a risk control test case may also include index data for querying that test case to facilitate the retrieval of test cases.

[0039] Step S102: Determine multiple target risk control test cases to be sent in the case template, generate risk control test requests, and send the target risk control test cases in batches to the pre-established decision engine based on the generated risk control test requests.

[0040] In practical applications, the testing tool can first use the internal interface data in the target risk control test case to generate a risk control test request, and then send the target risk control test cases to the decision engine in batches based on the generated risk control test request.

[0041] In this step, multiple test cases to be tested, i.e., target risk control test cases, are first identified from the case template, and these test cases are sent in batches to the decision engine for verification. Specifically, testers can identify all risk control test cases in the case template as target risk control test cases, risk control test cases whose case identifiers fall within the input identifier range in the case template as target risk control test cases, or risk control test cases under a specific input risk control scenario as target risk control test cases. Subsequently, the testing tool automatically generates risk control test requests using the internal interface data in the target risk control test cases. The risk control test requests can carry the case identifier and test subject data of each target risk control test case, and based on the generated risk control test requests, the target risk control test cases are sent in batches to the decision engine.

[0042] In practical applications, the details of each risk control test case can further include a risk control scenario identifier. After entering the decision engine, each risk control test case can be matched with risk control rules according to the corresponding risk control scenario. That is, after the target risk control test case enters the decision engine, it is sent by the decision engine to the decision platform corresponding to the same risk control scenario to trigger the risk control rules. This allows the matching of risk control test cases and risk control rules to be controlled through risk control scenarios, thereby ensuring that the rule verification process is consistent with the risk control scenario, while reducing the complexity of rule matching.

[0043] In practical applications, risk control test cases may include early warning cases. Early warning cases are used to verify risk control rules related to early warning operations. The test subject data is generally correlated with the numerical requirements of preset specific indicators. For example, these specific indicators may be cumulative amount, cumulative number of transactions, etc. Preferably, in this embodiment of the invention, the detailed data of the early warning cases stored in the case template may further include historical indicator accumulation logic for real-time statistical analysis of specific indicator values. In this way, the testing tool can use the historical indicator accumulation logic in any early warning case to calculate the current specific indicator value in real time. When it finds that the calculated specific indicator value meets the specific indicator value requirements corresponding to the early warning case, it sends the early warning case along with the risk control test request to the decision engine, thereby executing the corresponding early warning risk control rule verification. Through the above steps, the large number of interactive operations executed in a short period of time and the resulting communication congestion are avoided in the prior art, further improving testing efficiency without affecting testing accuracy.

[0044] As a preferred solution, the details of the risk control test case can further include the external decision data to be populated and the external service call path used to obtain the external decision data. Before generating the risk control test request, the testing tool can automatically generate an external data acquisition request based on the above external service call path and send it to the corresponding external service to obtain the external decision data. The risk control test request can then further include the external decision data already populated in the target risk control test case to complete the verification of the risk control rules. Through these steps, the manual data insertion operations of existing technologies can be avoided, thereby significantly improving the testing efficiency of risk control rules.

[0045] Step S103: Receive the risk control decision results returned by the decision engine for the risk control test request, and compare the details of the target risk control test case with the risk control decision results to obtain the verification results of the risk control rules.

[0046] In specific scenarios, the risk control decision result may include the target risk control rule identifier triggered by the test subject data in any target risk control test case. As a preferred solution, after obtaining the risk control decision result, the testing tool compares the expected result data in the target risk control test case with the above target risk control rule identifier to obtain the verification result of the risk control rule.

[0047] In practical applications, the expected result data in the risk control test case may include the identifier of the risk control rule to be triggered. In the comparison and verification process in this step, the testing tool can determine the expected hit case identifier of each risk control rule based on the expected trigger risk control rule identifier in the target risk control test case, and obtain the actual hit case identifier of each risk control rule from the target risk control rule identifier. After that, the testing tool determines whether the expected hit case identifier of the same risk control rule is consistent with the actual hit case identifier: if so, it determines that the verification of the risk control rule has passed; otherwise, it determines that the verification of the risk control rule has failed.

[0048] In one embodiment, the expected result data in the risk control test case may further include the expected decision result, that is, the relevant decision result given by the expected decision engine that is not identified by a risk control rule, such as the final approved loan amount. This decision result can be the result obtained by the decision engine after no risk control rules are triggered, or it can be the result obtained after some risk control rules are triggered. Correspondingly, the risk control decision result returned by the decision engine may further include the actual decision result corresponding to any target risk control test case and the target risk control rule triggered by that target risk control test case. After obtaining the risk control decision result, the testing tool can determine whether the expected decision result and the actual decision result for each target risk control test case corresponding to the same risk control rule are consistent: if yes, the verification of the risk control rule is determined to be passed; otherwise, the verification of the risk control rule is determined to be failed.

[0049] Preferably, the above risk control decision results may further include detailed data on the decision reasons for any target risk control test case triggering any target risk control rule, that is, the reason-related data used by the decision engine to trigger the corresponding risk control rule. These data are usually related to the triggering conditions of the risk control rule and may include the judgment thresholds of various parameters used in the triggering conditions and the actual values ​​of the risk control test case.

[0050] The following describes a specific embodiment of the present invention; see [link to specific embodiment]. Figure 2 .

[0051] In the financial technology field, there are numerous risk control scenarios, such as registration and login, credit approval, and anti-fraud. To minimize the probability of risk events, verifying risk control rules is an essential step. Risk control rule verification involves obtaining verification results from the decision engine based on test cases and comparing them with expected results to verify whether a match is successful. Current technologies typically perform this verification manually, which is time-consuming. Furthermore, in scenarios like registration and login and credit approval, test cases not only include internal interface data but also rely on external decision data. Current technologies require manually embedding data in third-party systems to obtain external decision data, which is labor-intensive. Therefore, it is difficult to verify a large number of risk control rules using existing manual methods.

[0052] To address the shortcomings of the manual verification methods described above, this embodiment proposes a tool for batch verification of risk control rules. Regarding data input, test cases can be imported in batches to execute an automated verification process. As for external data dependencies, relevant service call data can be deployed in the test cases to automatically acquire third-party data, as detailed below.

[0053] Step S201: Template Import. Import the test case template in Excel format containing the test case details into the validation tool. Each test case includes a case number (i.e., case identifier), risk control scenario identifier, test subject data, expected result data, internal interface data, and index data. Some test cases may also include historical indicator accumulation logic, external decision data to be filled, and external service call paths for obtaining external decision data. Some test cases may also include one positive case and multiple negative cases. The test subject data and expected result data of the positive and negative cases may be mutually exclusive.

[0054] Step S202: Test method selection. The test methods of this invention include: inputting case number for testing, selecting a range of case numbers for testing, sending all cases, and sending cases according to the risk control scenario. All of these methods can achieve batch processing and verification of risk control rules.

[0055] Step S203: The testing tool directly generates a risk control test request based on the internal interface data in the test case. If external decision data is required, the testing tool will call the external service in advance through the external service call path to obtain it and carry it in the risk control test request. If the case to be tested is an early warning case, the testing tool needs to monitor the indicator to the corresponding threshold through the historical indicator accumulation logic.

[0056] Step S204: Invoke the decision engine. The testing tool sends the risk control test request to the decision engine, which matches the request data with the risk control rules within it. The decision engine includes decision platforms corresponding to various risk control scenarios, and each decision platform includes multiple risk control rules.

[0057] Step S205: Receive risk control decision results. The decision engine determines whether a risk control rule has been hit based on the request content, and then returns the risk control decision results. These results may include the identifier of the target risk control rule triggered by the test subject data in any target risk control test case, details of the decision reason for triggering any target risk control rule in any target risk control test case, and the actual decision result corresponding to the target risk control test case and the target risk control rule triggered by that target risk control test case. It may also include the URL (Uniform Resource Locator) of the risk control test request so that the testing tool can verify the correctness of the risk control scenario identifier determined by the decision engine. The testing tool can provide a file download function to export the risk control decision results as a file.

[0058] Step S206: Result Comparison and Verification. The testing tool compares the risk control decision results with the expected results in the case template to obtain the verification results for the risk control rules.

[0059] In this way, based on case templates and technologies such as automatic request generation, automatic acquisition of external data, and real-time accumulation of historical indicators, the risk control rules can be automatically verified in batches, simplifying the testing process and improving testing efficiency.

[0060] The following describes another specific embodiment of the present invention.

[0061] This embodiment enables automated batch verification of risk control rules. In practical applications, risk managers define risk control rules to manage risks in different situations, thereby reducing the probability of risk events or minimizing losses when they do occur. The specific execution steps of this embodiment are as follows:

[0062] First, the testing tool obtains detailed information about multiple risk control test cases entered by staff in the case template. As mentioned earlier, the detailed information of the risk control test cases includes: case identifier, test subject data, expected result data, internal interface data, index data, and risk control scenario identifier. The meaning and specific function of each data have been explained above and will not be repeated here.

[0063] Next, the testing tool identifies multiple target risk control test cases to be sent from the case template. Using the internal interface data mentioned above, it automatically generates risk control test requests and sends these target risk control test cases to the decision engine in batches via these requests. During this process, testers can select all risk control test cases in the case template as target risk control test cases, or they can select risk control test cases whose case identifiers fall within the input identifier range. For example, all risk control test cases with case identifiers from 001 to 100 can be selected as target risk control test cases. Testers can also select risk control test cases under a specific input risk control scenario as target risk control test cases. For example, if a login scenario is selected, all risk control test cases under the login scenario can be selected as target risk control test cases.

[0064] For early warning cases in risk control test cases, the test can only be triggered when the historical indicator accumulation logic in the early warning case details detects that the data requirements are met. For example, the execution condition for an early warning case is that the user's cumulative transaction amount reaches 1000 yuan. The testing tool uses the historical indicator accumulation logic in the early warning case to calculate the current user's cumulative transaction amount in real time. When it finds that the user's cumulative transaction amount has reached 1000 yuan, it sends the early warning case along with the risk control test request to the decision engine, thereby executing the corresponding early warning risk control rule verification. Through the above steps, the large number of interactive operations executed in a short period of time and the resulting communication congestion are avoided in existing technologies, further improving testing efficiency without affecting testing accuracy.

[0065] For some special cases in risk control test cases, the details include external decision data to be populated. This data plays a role in the risk control rule matching process. For example, if a risk control test case requires the latest interest rate and exchange rate for a certain currency, the testing tool will automatically generate an external data acquisition request based on the external service call path in the details of the risk control test case and send it to the corresponding external service before generating the corresponding risk control test request. This will obtain the latest interest rate, exchange rate, and other external decision data, and the completed external decision data will be included in the risk control test request to complete the verification of the risk control rules. Through these steps, the manual data insertion operations in existing technologies can be avoided, thereby significantly improving the testing efficiency of risk control rules.

[0066] Finally, the testing tool receives the risk control decision results returned by the decision engine and compares the expected result data in the target risk control test case with the target risk control rule identifier in the risk control decision results to obtain the verification result of the risk control rule. This comparison has two aspects. First, the testing tool aggregates the expected hit case identifiers for each risk control rule using the risk control rule identifier as the aggregation dimension, and then aggregates the actual hit case identifiers for each risk control rule from the target risk control rule identifiers. This allows it to determine whether the expected hit case identifiers for the same risk control rule are consistent with the actual hit case identifiers. Second, the testing tool compares the expected decision result and the actual decision result for the same risk control rule. For example, if risk control rule A expects an approved loan amount (expected decision result) of 100,000 yuan for risk control test case B, but the actual approved loan amount (actual decision result) returned by the decision engine is 150,000 yuan, then the verification of risk control rule B is considered to have failed, and the staff needs to be alerted to pinpoint the reason for the failure.

[0067] It should be noted that the collection, analysis, use, transmission, and storage of user personal information involved in the technical solution of this invention all comply with the provisions of relevant laws and regulations, are used for legal and reasonable purposes, and are not shared, disclosed, or sold outside of these legal uses, and are subject to supervision and management by regulatory authorities. Necessary measures should be taken to prevent unauthorized access to such personal information data, ensure that personnel authorized to access personal information data comply with the provisions of relevant laws and regulations, and ensure the security of user personal information. Once this user personal information data is no longer needed, the risk should be minimized by restricting or even prohibiting data collection and / or deleting the data. When used, including in certain related applications, user privacy is protected by de-identifying the data, for example, by removing specific identifiers (e.g., date of birth), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than the specific address level), controlling how the data is stored, and / or other de-identification methods.

[0068] For the foregoing method embodiments, they are described as a series of actions for ease of description. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, and some steps may actually be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential for implementing the present invention.

[0069] To facilitate better implementation of the above-described solutions of the embodiments of the present invention, related apparatus for implementing the above-described solutions is also provided below.

[0070] Please see Figure 3As shown, the automated verification device 300 provided in this embodiment of the invention may include: a case import unit 301, a test request unit 302, and a comparison unit 303.

[0071] The case import unit 301 can be used to obtain detailed information of multiple risk control test cases input in a pre-created case template; the test request unit 302 can be used to determine multiple target risk control test cases to be sent in the case template, generate risk control test requests, and send the target risk control test cases in batches to a pre-established decision engine based on the generated risk control test requests; wherein the decision engine contains multiple pre-configured risk control rules; the comparison unit 303 can be used to receive the risk control decision results returned by the decision engine for the risk control test requests, compare the detailed information of the target risk control test cases with the risk control decision results to obtain the verification results of the risk control rules.

[0072] In one optional technical solution, the detailed information of each risk control test case includes: case identifier, test subject data, expected result data, and internal interface data. The risk control test request carries the case identifier and test subject data of each target risk control test case. The risk control decision result includes the target risk control rule identifier triggered by the test subject data in any target risk control test case. Furthermore, the test request unit 302 can be further used to: generate a risk control test request using the internal interface data in the target risk control test case. The comparison unit 303 can be further used to: compare the expected result data in the target risk control test case with the target risk control rule identifier to obtain the verification result of the risk control rule.

[0073] In this embodiment of the invention, the details of each risk control test case further include a risk control scenario identifier. The decision engine includes a decision platform corresponding to each risk control scenario, and each decision platform contains multiple risk control rules corresponding to the same risk control scenario. After the target risk control test case enters the decision engine, it is sent by the decision engine to the decision platform corresponding to the same risk control scenario to trigger the risk control rules. The test case includes an early warning case, and the test subject data of the early warning case is associated with the numerical requirements of a preset specific indicator. The early warning case further includes historical indicator accumulation logic for real-time statistics of the specific indicator value. The test request unit 302 can be further used to: use the historical indicator accumulation logic in any early warning case to count the current specific indicator value; in response to the counted specific indicator value meeting the specific indicator value requirements corresponding to the early warning case, send the early warning case to the decision engine along with the risk control test request.

[0074] In practical applications, the details of the risk control test case further include external decision data to be filled and an external service call path for obtaining the external decision data; and the test request unit 302 can be further configured to: before generating a risk control test request using the internal interface data in the target risk control test case, generate an external data acquisition request based on the external service call path and send it to the corresponding external service to obtain the external decision data; the risk control test request further carries the external decision data that has been filled in the target risk control test case.

[0075] Preferably, the expected result data in the risk control test case includes the expected risk control rule identifier and the expected decision result, wherein the risk control decision result further includes the actual decision result corresponding to any target risk control test case and the target risk control rule triggered by the target risk control test case; and the comparison unit 303 may be further configured to: determine the expected hit case identifier of each risk control rule based on the expected risk control rule identifier in the target risk control test case, obtain the actual hit case identifier of each risk control rule from the target risk control rule identifier; determine whether the expected hit case identifier of the same risk control rule is consistent with the actual hit case identifier: if yes, determine that the verification of the risk control rule has passed; otherwise, determine that the verification of the risk control rule has failed; determine whether the expected decision result and the actual decision result of the same risk control rule corresponding to each target risk control test case are consistent: if yes, determine that the verification of the risk control rule has passed; otherwise, determine that the verification of the risk control rule has failed.

[0076] Furthermore, in this embodiment of the invention, the case import unit 301 can be further used to: determine all risk control test cases in the case template as the target risk control test cases; or, determine risk control test cases in the case template whose case identifiers are within the input identifier range as the target risk control test cases; or, determine risk control test cases under the input specific risk control scenario as the target risk control test cases; the risk control decision result further includes: detailed data of the decision reason for any target risk control test case triggering any target risk control rule; and the detailed information of each risk control test case further includes: index data for querying the risk control test case.

[0077] According to the technical solution of this invention, a testing tool with batch processing and automated testing functions has been developed for risk control rule verification. Based on this testing tool, automated verification of risk control rules is achieved by importing case template files, thereby effectively simplifying the traditional manual verification process and improving the security, accuracy, efficiency, and comprehensiveness of risk control rule verification. Furthermore, the technical solution of this invention can proactively send messages to external systems to obtain external decision data in a message-decoupled manner, thereby simplifying the external data filling operation during the testing process, avoiding the tedious data embedding process, and facilitating subsequent regression testing.

[0078] Figure 4 An exemplary system architecture 400 is shown that can be applied to the automated verification method or automated verification device of the present invention.

[0079] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, and 403, network 404, and server 405 (this architecture is merely an example; the components included in a specific architecture may be adjusted according to the specific application). Network 404 serves as the medium for providing a communication link between terminal devices 401, 402, and 403 and server 405. Network 404 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0080] Users can use terminal devices 401, 402, and 403 to interact with server 405 via network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, and 403, such as the test tool mentioned above (for example only).

[0081] Terminal devices 401, 402, and 403 can be various electronic devices with displays that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0082] Server 405 can be a server that provides various services, such as a decision engine server (for example only) that supports the testing tools operated by users using terminal devices 401, 402, and 403. The decision engine server can process received risk control test requests and feed back the processing results (such as risk control decision results - for example only) to terminal devices 401, 402, and 403.

[0083] It should be noted that the automated verification method provided in this embodiment of the invention is generally executed by server 405, and correspondingly, the automated verification device is generally set in server 405.

[0084] It should be understood that Figure 4The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0085] The present invention also provides an electronic device. The electronic device according to an embodiment of the present invention includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the automated verification method provided by the present invention.

[0086] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system 500 suitable for implementing an electronic device according to embodiments of the present invention. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0087] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the computer system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0088] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0089] In particular, according to the embodiments disclosed in this invention, the processes described in the above main step diagrams can be implemented as computer software programs. For example, embodiments of this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the main step diagrams. In the above embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit 501, it performs the functions defined in the system of this invention.

[0090] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0091] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0092] The units described in the embodiments of the present invention can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor can be described as including a case import unit, a test request unit, and a comparison unit. The names of these units do not necessarily limit the specific unit; for example, the case import unit can also be described as "a unit for providing risk control test cases to the test request unit."

[0093] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to perform the following steps: obtaining detailed information of multiple risk control test cases input in a pre-created case template; determining multiple target risk control test cases to be sent from the case template, generating risk control test requests, and sending the target risk control test cases in batches to a pre-established decision engine based on the generated risk control test requests; wherein the decision engine contains multiple pre-configured risk control rules; receiving risk control decision results returned by the decision engine for the risk control test requests, and comparing the detailed information of the target risk control test cases with the risk control decision results to obtain the verification results of the risk control rules.

[0094] A computer program product according to an embodiment of the present invention includes a computer program that, when executed by a processor, implements the automated verification method according to an embodiment of the present invention.

[0095] In the technical solution of this invention, the batch processing and automatic request generation functions of the testing tool enable automated batch testing of multiple risk control rules, improving the testing efficiency of risk control rules. When the testing process involves third-party data, the testing tool automatically obtains the external decision data required for the test through the pre-defined external service call paths in the test cases to complete the automated testing process, avoiding manual data collection in existing technologies and further improving testing efficiency. For testing of early warning-related rules, the corresponding early warning cases pre-deploy historical indicator accumulation logic and statistically analyze the indicator accumulation in real time. When the indicator accumulates to the threshold, the testing tool automatically sends the corresponding early warning case to execute the testing process, thereby avoiding the concentrated execution of a large number of interactive operations in a short period of time and the resulting communication congestion in existing technologies, further improving testing efficiency without affecting testing accuracy.

[0096] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An automated verification method, characterized in that, include: Retrieve detailed information about multiple risk control test cases entered in a pre-created case template; Multiple target risk control test cases to be sent are identified in the case template, a risk control test request is generated, and the target risk control test cases are sent in batches to a pre-established decision engine based on the generated risk control test request; wherein, the decision engine contains multiple pre-configured risk control rules; The system receives the risk control decision result returned by the decision engine in response to the risk control test request, compares the details of the target risk control test case with the risk control decision result to obtain the verification result of the risk control rule; the verification result of the risk control rule includes the URL of the risk control test request, so that the test tool can verify whether the risk control scenario identifier determined by the decision engine is correct; The details of each risk control test case include a risk control scenario identifier. The decision engine includes a decision platform corresponding to each risk control scenario. Each decision platform contains multiple risk control rules corresponding to the same risk control scenario. After the target risk control test case enters the decision engine, it is sent by the decision engine to the decision platform corresponding to the same risk control scenario to trigger the risk control rules.

2. The method according to claim 1, characterized in that, The detailed information for each risk control test case includes: case identifier, test subject data, expected result data, and internal interface data; and the generation of the risk control test request includes: A risk control test request is generated using the internal interface data in the target risk control test case; wherein the risk control test request carries the case identifier and test subject data of each target risk control test case.

3. The method according to claim 2, characterized in that, The risk control decision result includes the identifier of the target risk control rule triggered by the test subject data in any target risk control test case; and the comparison of the details of the target risk control test case with the risk control decision result to obtain the verification result of the risk control rule includes: The expected result data in the target risk control test case is compared with the target risk control rule identifier to obtain the verification result of the risk control rule.

4. The method according to claim 3, characterized in that, The risk control test cases include early warning cases, wherein the test subject data of the early warning cases is associated with the numerical requirements of preset specific indicators, and the early warning cases further include historical indicator accumulation logic for real-time statistical analysis of the values ​​of the specific indicators; and the method further includes: The current value of a specific indicator is calculated by accumulating historical indicators from any warning case. In response to the statistically obtained value of a specific indicator meeting the specific indicator value requirement corresponding to the warning case, the warning case is sent to the decision engine along with the risk control test request.

5. The method according to claim 3, characterized in that, The details of the risk control test case further include the external decision data to be filled and the external service call path used to obtain the external decision data; And, the method further includes: Before generating a risk control test request using the internal interface data in the target risk control test case, an external data acquisition request is generated based on the external service call path and sent to the corresponding external service to obtain the external decision data. The risk control test request further includes the external decision data that has been filled in for the target risk control test case.

6. The method according to claim 3, characterized in that, The expected result data in the risk control test case includes the identifier of the risk control rule to be triggered; and the comparison of the expected result data in the target risk control test case with the target risk control rule identifier to obtain the verification result of the risk control rule includes: Based on the risk control rule identifier expected to be triggered in the target risk control test case, determine the expected hit case identifier of each risk control rule, and obtain the actual hit case identifier of each risk control rule from the target risk control rule identifier; Determine whether the expected hit case identifier of the same risk control rule is consistent with the actual hit case identifier: if yes, determine that the verification of the risk control rule has passed; otherwise, determine that the verification of the risk control rule has failed.

7. The method according to claim 6, characterized in that, The expected result data in the risk control test case further includes the expected decision result, and the risk control decision result further includes the actual decision result corresponding to any target risk control test case and the target risk control rule triggered by the target risk control test case; and the method further includes: Determine whether the expected decision result and the actual decision result of the same risk control rule for each target risk control test case are consistent: if yes, determine that the verification of the risk control rule has passed; otherwise, determine that the verification of the risk control rule has failed.

8. The method according to claim 3, characterized in that, The process of determining multiple target risk control test cases to be sent in the case template includes: Select all risk control test cases in the aforementioned case template as the target risk control test cases; or... The risk control test cases whose case identifiers fall within the input identifier range in the case template are identified as the target risk control test cases; or... The risk control test cases under the specific risk control scenario input are determined as the target risk control test cases.

9. The method according to claim 3, characterized in that, The risk control decision results further include: detailed data on the decision reasons why any target risk control test case triggers any target risk control rule; and... The details of each risk control test case further include: index data used to query that risk control test case.

10. An automated verification device, characterized in that, include: The case import unit is used to retrieve detailed information about multiple risk control test cases entered in a pre-created case template; The test request unit is used to determine multiple target risk control test cases to be sent from the case template, generate risk control test requests, and send the target risk control test cases in batches to a pre-established decision engine based on the generated risk control test requests; wherein, the decision engine contains multiple pre-configured risk control rules; The comparison unit is used to receive the risk control decision result returned by the decision engine in response to the risk control test request, and compare the details of the target risk control test case with the risk control decision result to obtain the verification result of the risk control rule; the verification result of the risk control rule includes the URL of the risk control test request, so that the test tool can verify whether the risk control scenario identifier determined by the decision engine is correct; The details of each risk control test case include a risk control scenario identifier. The decision engine includes a decision platform corresponding to each risk control scenario. Each decision platform contains multiple risk control rules corresponding to the same risk control scenario. After the target risk control test case enters the decision engine, it is sent by the decision engine to the decision platform corresponding to the same risk control scenario to trigger the risk control rules.

11. The apparatus according to claim 10, characterized in that, The detailed information for each risk control test case includes: case identifier, test subject data, expected result data, and internal interface data. The risk control test request carries the case identifier and test subject data for each target risk control test case. The risk control decision result includes the target risk control rule identifier triggered by the test subject data in any target risk control test case; and... The test request unit is further configured to: generate a risk control test request using the internal interface data in the target risk control test case; The comparison unit is further used to: compare the expected result data in the target risk control test case with the target risk control rule identifier to obtain the verification result of the risk control rule.

12. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-9.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-9.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-9.

Images