Strategy testing method and device, equipment, storage medium and program product

Abstract

The invention provides a strategy testing method and device, equipment, a storage medium and a program product, and can be applied to the technical field of computers. The method comprises the steps that a strategy component, a reference component and a variable of a to-be-tested strategy module are determined from a to-be-tested strategy, the to-be-tested strategy comprises a plurality of strategy modules, the to-be-tested strategy module is at least one strategy module in the to-be-tested strategy, the strategy component is used for operating the to-be-tested strategy module, and the reference component is used for evaluating the to-be-tested strategy module; under the condition that the configuration information of the variable represents that the variable is a fixed variable, determining a variable value of the variable from a fixed value set; respectively operating the strategy component and the reference component based on the variable value to obtain an execution result aiming at the strategy component and a reference result aiming at the reference component; and based on the reference result, evaluating the execution result to obtain a test result.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a strategy testing method, apparatus, device, storage medium, and program product. Background Technology

[0002] A strategy is typically a set of solutions consisting of rules and decisions, which make decisions based on a series of rules according to different scenarios. As business needs are constantly updated, strategies also iterate accordingly. After strategy iteration, testing is usually required to ensure the quality of the strategy.

[0003] However, during strategy testing, since a strategy consists of multiple strategy modules and the variables of these modules are related to each other, it is usually necessary to test the entire strategy. This results in repeated testing of unrelated strategy modules and a lot of redundant variables, wasting testing time and running costs. Summary of the Invention

[0004] In view of this, the present disclosure provides a strategy testing method, apparatus, device, storage medium, and program product.

[0005] One aspect of this disclosure provides a strategy testing method, comprising: determining a strategy component, a reference component, and variables for a strategy module to be tested from a strategy to be tested, wherein the strategy to be tested includes multiple strategy modules, and the strategy module to be tested is at least one of the strategy modules to be tested; the strategy component is used to run the strategy module to be tested, and the reference component is used to evaluate the strategy module to be tested; determining the variable value of the variable from a set of fixed values ​​when the configuration information of the variable indicates that the variable is a fixed variable; running the strategy component and the reference component respectively based on the variable value to obtain an execution result for the strategy component and a reference result for the reference component; and evaluating the execution result based on the reference result to obtain a test result.

[0006] According to embodiments of this disclosure, the method further includes: when it is determined that the strategy module has been updated, parsing the strategy component of the strategy module to determine the variables used to run the strategy component; determining the identification result based on the variable attributes of the variables; and updating the configuration information of the variables to indicate that the variables are fixed variables when the identification result indicates that the configuration information of the variables has been updated.

[0007] According to embodiments of this disclosure, determining the identification result based on the variable attributes of the aforementioned variables includes: assessing the complexity of the aforementioned variables based on the variable association relationships in the aforementioned variable attributes, and determining the complexity identification result of the aforementioned variables, wherein the variable association relationships characterize the relationship between the aforementioned variables and variables in other strategy modules of the aforementioned strategy to be tested; determining the type identification result of the aforementioned variables based on the variable types in the aforementioned variable attributes, wherein the variable types include discrete and continuous types; and determining the aforementioned identification result based on the aforementioned complexity identification result and the aforementioned type identification result.

[0008] According to embodiments of this disclosure, the method further includes: determining the variable value from a dynamic value set when the configuration information of the variable indicates that the variable is a dynamic variable.

[0009] According to embodiments of this disclosure, after executing the strategy component based on the aforementioned variable values, the method further includes: determining, from the variable set of the strategy to be tested, an associated variable related to the strategy module to be tested, wherein the variable value of the associated variable is associated with the variable value of the variable of the strategy module to be tested; determining the variable value of the associated variable based on the variable value of the variable of the strategy module to be tested; and updating the initial variable value of the associated variable in the dynamic value set based on the variable value of the associated variable.

[0010] According to embodiments of this disclosure, the method further includes: determining reference logic for evaluation based on the policy logic of the policy module to be tested and the configuration information of the variables; and generating the reference component based on the reference logic.

[0011] According to embodiments of this disclosure, the above-mentioned evaluation of the execution result based on the reference result to obtain the test result includes: if the execution result and the reference result match, determining that the test result of the strategy module to be tested indicates that the strategy module to be tested has passed the test; if the execution result and the reference result do not match, determining that the test result of the strategy module to be tested indicates that the strategy module to be tested has failed the test.

[0012] According to embodiments of this disclosure, the method further includes: obtaining a first strategy test result of the strategy to be tested based on the test results of the strategy module to be tested; obtaining a second strategy test result of the strategy to be tested based on the strategy execution result of the strategy to be tested, wherein the strategy execution result represents the result obtained after executing all strategy components of the strategy to be tested; and obtaining a target test result of the strategy to be tested based on the first strategy test result and the second strategy test result.

[0013] Another aspect of this disclosure provides a strategy testing apparatus, comprising: a strategy determination module, configured to determine a strategy component, a reference component, and variables of a strategy module to be tested from a strategy to be tested, wherein the strategy to be tested includes multiple strategy modules, and the strategy module to be tested is at least one of the strategy modules to be tested; the strategy component is used to run the strategy module to be tested, and the reference component is used to evaluate the strategy module to be tested; a variable determination module, configured to determine the variable value of the variable from a fixed value set when the configuration information of the variable indicates that the variable is a fixed variable; a strategy execution module, configured to run the strategy component and the reference component respectively based on the variable value to obtain an execution result for the strategy component and a reference result for the reference component; and a result evaluation module, configured to evaluate the execution result based on the reference result to obtain a test result.

[0014] Another aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the methods described above.

[0015] Another aspect of this disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the methods described above.

[0016] Another aspect of this disclosure provides a computer program product including computer-executable instructions that, when executed, implement the methods described above.

[0017] According to embodiments of this disclosure, by introducing fixed variables that are not affected by other strategy modules during the strategy testing process, the test of the strategy module to be tested can be carried out directly based on the variable values ​​of the fixed variables, without the need to perform full testing of the entire strategy to be tested, thereby reducing the complexity of redundant variables and reference components and improving testing efficiency. Attached Figure Description

[0018] The above and other objects, features and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0019] Figure 1 An exemplary system architecture for applying the strategy testing methods disclosed herein is illustrated schematically;

[0020] Figure 2 A flowchart illustrating a strategy testing method according to an embodiment of the present disclosure is shown schematically.

[0021] Figure 3A schematic diagram of a variable configuration page according to an embodiment of the present disclosure is shown;

[0022] Figure 4 A schematic diagram illustrating a strategy to be tested according to an embodiment of this disclosure is shown.

[0023] Figure 5 A block diagram of a strategy testing apparatus according to an embodiment of the present disclosure is schematically shown; and

[0024] Figure 6 A block diagram of an electronic device suitable for implementing a strategy testing method according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0025] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0026] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0028] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0029] In the embodiments disclosed herein, the collection, updating, analysis, processing, use, transmission, provision, disclosure, and storage of data (e.g., including but not limited to user personal information) comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures have been taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0030] In the embodiments disclosed herein, user authorization or consent is obtained before acquiring or collecting user personal information.

[0031] A strategy is typically a set of solutions consisting of rules and decisions. Depending on the scenario and the specific characteristics of the target object, it uses a series of rules to make decisions about these objects to achieve an evaluation objective. For example, in risk assessment, the evaluation result of the assessed object is mainly obtained by executing risk strategies. After strategy iteration, it is usually necessary to test the strategy to ensure its correctness.

[0032] In the strategy under test, the variables of multiple strategy modules are usually related. During the strategy testing process, full testing is usually required, which leads to repeated testing of many irrelevant strategies and redundant variables, wasting testing time and running costs.

[0033] Since most strategy iterations only change a portion of the strategy modules, this disclosure provides a strategy testing method to improve strategy testing efficiency. The method includes: determining a strategy component, a reference component, and variables for a strategy module to be tested from the strategy to be tested. The strategy to be tested includes multiple strategy modules, and the strategy module to be tested is at least one of the strategy modules. The strategy component is used to run the strategy module to be tested, and the reference component is used to evaluate the strategy module to be tested. When the configuration information of the variables indicates that the variables are fixed, the variable values ​​are determined from a set of fixed values. Based on the variable values, the strategy component and the reference component are run respectively to obtain execution results for the strategy component and reference results for the reference component. Finally, based on the reference results, the execution results are evaluated to obtain test results.

[0034] The embodiments of this disclosure introduce fixed variables whose values ​​are not affected by the strategy modules. When conducting strategy testing, the strategy module to be tested can be directly tested locally based on the fixed variables without considering other strategy modules in the strategy to be tested. This allows for the direct design of the strategy to be tested for the strategy module to be tested, reducing redundant variables and improving testing efficiency.

[0035] Figure 1An exemplary system architecture 100 to which strategy testing methods can be applied according to embodiments of this disclosure is illustrated schematically. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0036] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0037] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (for example only).

[0038] Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0039] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using terminal devices 101, 102, and 103 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0040] It should be noted that the strategy testing method provided in this embodiment can generally be executed by server 105. Correspondingly, the strategy testing device provided in this embodiment can generally be located in server 105. The strategy testing method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the strategy testing device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Alternatively, the strategy testing method provided in this embodiment can also be executed by terminal devices 101, 102, or 103, or by other terminal devices different from terminal devices 101, 102, or 103. Correspondingly, the strategy testing device provided in this embodiment can also be located in terminal devices 101, 102, or 103, or in other terminal devices different from terminal devices 101, 102, or 103.

[0041] For example, the strategy to be tested may originally be stored in any one of terminal devices 101, 102, or 103 (e.g., terminal device 101, but not limited thereto), or it may be stored on an external storage device and imported into terminal device 101. Then, terminal device 101 may execute the strategy testing method provided in the embodiments of this disclosure locally, or send the image to be processed to other terminal devices, servers, or server clusters, and have the image processing method provided in the embodiments of this disclosure executed by the other terminal devices, servers, or server clusters that receive the image to be processed.

[0042] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0043] Figure 2 A flowchart illustrating a strategy testing method according to an embodiment of this disclosure is shown schematically.

[0044] like Figure 2 As shown, the method includes operations S210~S240.

[0045] In operation S210, the policy components, reference components, and variables of the policy module to be tested are determined from the policy to be tested.

[0046] In operation S220, when the configuration information of the variable indicates that the variable is a fixed variable, the variable value is determined from the set of fixed values.

[0047] In operation S230, the policy component and the reference component are run based on the variable values ​​respectively, and the execution results for the policy component and the reference results for the reference component are obtained.

[0048] In operation S240, the execution results are evaluated based on the reference results to obtain the test results.

[0049] The strategy to be tested comprises multiple strategy modules, which can be, for example, rules within the strategy. When iterating over the strategy, typically only a subset of strategy modules is updated. Therefore, when testing the iterated strategy, at least one updated strategy module can be identified as the strategy module to be tested, and this module can be tested to ensure that no errors are introduced during the strategy iteration process, thereby guaranteeing the accuracy of the strategy under test.

[0050] Strategies typically make decisions by processing the values ​​of variables according to rules. When testing a strategy module, its variables need to be determined first. This strategy module can transmit data with other strategy modules through an interface, and its variables can be the input parameters of that interface.

[0051] Since variables in multiple strategy modules within the strategy under test may be related to each other, when testing a strategy module, variables that are related to other strategy modules can be set as fixed variables. The variable values ​​in the fixed value set can be used to replace the variable values ​​after they are affected by other strategy modules, thus eliminating the need to run other strategy modules and improving testing efficiency.

[0052] When the configuration information of a variable indicates that the variable is a fixed variable, the variable value can be obtained from a fixed value set. For example, the fixed value set can store the fixed values ​​of different variables in the form of key-value pairs. When it is necessary to determine the variable value, the variable value corresponding to the variable is obtained from the fixed value set according to the variable identifier. The variable values ​​stored in the fixed value set do not change as the rule is executed during the decision to be executed.

[0053] In some embodiments, multiple variable values ​​can be set for a fixed variable. When conducting strategy testing, any one of these multiple variable values ​​can be used for testing, or all variable values ​​can be used for testing, so as to simulate a variety of different real-world scenarios using different variable values ​​and achieve comprehensive testing of the strategy module under test.

[0054] When the configuration information of variables represents variables that are not fixed, testers can set multiple variable values ​​for the variables to simulate various real-world scenarios and achieve comprehensive testing of the modules to be tested.

[0055] When testing a strategy, a strategy component is typically used to run the strategy module under test, and a reference component is used to evaluate the strategy module under test. In some embodiments, the reference component can simulate the correct operating logic of the strategy component and generate correct reference results based on variable values. At the same time, the strategy component generates corresponding execution results based on variable values. By comparing the reference results with the execution results, the strategy component can be tested.

[0056] When evaluating the execution results, it can be determined whether the reference result and the execution result are the same. If the reference result and the execution result are the same, the test result can be determined as the strategy module under test passing the test; otherwise, the test result can be determined as the strategy module under test failing the test.

[0057] According to embodiments of this disclosure, by introducing fixed variables that are not affected by other strategy modules during the strategy testing process, the test of the strategy module to be tested can be carried out directly based on the variable values ​​of the fixed variables, without the need to perform full testing of the entire strategy to be tested, thereby reducing the complexity of redundant variables and reference components and improving testing efficiency.

[0058] According to embodiments of this disclosure, the strategy testing method further includes: if it is determined that the strategy module has been updated, parsing the strategy component of the strategy module to determine the variables used to run the strategy component; determining the identification result based on the variable attributes; and updating the configuration information of the variable to indicate that the variable is a fixed variable if the identification result indicates that the variable's configuration information has been updated.

[0059] When determining fixed variables, since strategy testing is usually aimed at updated strategy modules, if the strategy module has been updated, the variables of the strategy components of the updated strategy module can be filtered to determine suitable variables as fixed variables.

[0060] When selecting fixed variables from a pool of variables, one can identify the variable attributes and determine suitable variables to be fixed based on these attributes. Variable attribute identification can include, for example, the variable's complexity and type.

[0061] When determining a fixed variable based on its attributes, it can be determined whether the variable attribute is a preset variable attribute type. If the variable attribute is a preset variable attribute type, it can be determined that the variable will be set as a fixed variable.

[0062] Since the configuration information of a variable can characterize whether the variable is a fixed variable, when a variable is set as a fixed variable, the identification result of the variable can be determined to indicate that the configuration information of the updated variable is a fixed variable, so that the configuration information of the variable can be updated according to the identification result.

[0063] According to embodiments of this disclosure, by determining fixed variables from the variables of a strategy component based on variable attributes, testing efficiency can be effectively improved when using fixed variables for testing.

[0064] In another embodiment of this disclosure, strategy testers can also set the configuration information of variables through the variable configuration page. For example, strategy testers can set variables as fixed variables through the variable configuration page.

[0065] Figure 3 A schematic diagram of a variable configuration page according to an embodiment of the present disclosure is shown.

[0066] like Figure 3 As shown, the variable configuration page displays information such as the variable name, variable value, and whether it is a fixed variable for variables i1, i2, and i3. In addition, you can delete and copy variables using the "Delete Variable" and "Copy Variable" buttons.

[0067] like Figure 3 As shown, strategy testers can edit the variable values ​​in the variable value text box. For example, they can set the variable value of i1 to include 100 and 0, the variable value of i2 to include 1000 and 0, and the variable value of i3 to include 100 and null. Furthermore, strategy testers can configure variables as fixed variables in the "Whether to be a fixed variable" selection box, for example, configuring i1 and i2 as fixed variables and i3 as a dynamic variable.

[0068] According to embodiments of this disclosure, determining the identification result based on the variable attributes of a variable includes: assessing the complexity of a variable based on the variable relationships in the variable attributes to determine the complexity identification result of the variable; determining the type identification result of the variable based on the variable type in the variable attributes; and determining the identification result based on the complexity identification result and the type identification result.

[0069] When filtering fixed variables based on variable attributes, you can filter them based on the variable relationships within those attributes. Variable relationships characterize the relationship between a variable and variables in other strategy modules within the strategy being tested. For example, you can determine variable relationships based on the number of variables in other strategy modules that are related to that variable.

[0070] In the embodiments of this disclosure, the complexity of a variable can be determined based on the variable relationship, and the complexity identification result can be further determined based on the complexity. Specifically, it can be determined that a variable with a complex variable relationship has higher complexity, and the complexity identification result of the variable is determined to be high complexity; conversely, the complexity identification result of the variable can be determined to be low complexity.

[0071] When identifying the complexity of a variable, its complexity can be quantified. Specifically, a preset threshold can be set. If the number of other variables related to the variable exceeds the preset threshold, the variable identified by the complexity identification result is determined to be a complex variable; otherwise, it can be determined to be a simple variable.

[0072] When filtering fixed variables based on variable attributes, you can also filter them based on the variable type within those attributes. Variable types include discrete and continuous, and the type identification result can characterize whether a variable is discrete or continuous.

[0073] To reduce the complexity of the strategy testing process, highly complex variables can be selected as fixed variables. Furthermore, since continuous variables cannot be configured with fixed values, discrete variables can be chosen as fixed variables.

[0074] The initial configuration information of variables can default to dynamic variables. When determining the recognition result based on the complexity recognition result and the type recognition result, the configuration information of the variable to be updated can be determined if the variable represented by the complexity recognition result is a complex variable and the variable represented by the type recognition result is discrete. If the variable represented by the complexity recognition result is a simple variable or the variable represented by the type recognition result is continuous, the configuration information of the variable not to be updated can be determined.

[0075] According to embodiments of this disclosure, by determining the identification results for variables based on their variable relationships and types, selecting highly complex and discrete variables as fixed variables can improve the accuracy of the test and reduce redundant information.

[0076] According to embodiments of this disclosure, the strategy testing method further includes: determining the variable value from a dynamic value set when the configuration information of the variable indicates that the variable is a dynamic variable.

[0077] The dynamic value set can store the actual values ​​of variables as they change during the execution of the rules in the process of the decision being tested. When the variable is not an intermediate variable, the strategy component and the reference component can be executed based on the variable values ​​stored in the dynamic value set.

[0078] For example, the first strategy module, the second strategy module, and the third strategy module all include a dynamic variable i. After executing the rule represented by the first strategy module, the dynamic variable i is assigned the value 1. After executing the rule represented by the second strategy module, the dynamic variable i is assigned the value 2. When the strategy module to be tested is the third strategy module, the value of the dynamic variable can be determined to be 2, and the third strategy module can be tested using the value 2.

[0079] According to embodiments of this disclosure, when the variable is a dynamic variable, the variable value of the dynamic variable is directly determined from the dynamic value set. Therefore, even if the variable is affected by other strategy modules, there is no need to execute other strategy modules during testing, thereby improving testing efficiency.

[0080] According to embodiments of this disclosure, after executing the strategy component based on variable values, the strategy testing method further includes: determining associated variables related to the strategy module under test from the variable set of the strategy under test; determining the variable values ​​of the associated variables based on the variable values ​​of the variables of the strategy module under test; and updating the initial variable values ​​of the associated variables in the dynamic value set based on the variable values ​​of the associated variables.

[0081] Since the strategy module under test has been updated, and the values ​​of the associated variables are related to the values ​​of the variables in the strategy module under test, the relationship may change, which may cause the initial values ​​of the associated variables to not match the strategy under test after the strategy module under test is updated. Therefore, it is necessary to update the initial values ​​of the associated variables.

[0082] When updating the initial values ​​of associated variables, the values ​​of the associated variables can be determined based on the values ​​of the variables in the strategy module under test after running the strategy component based on the variable values. For example, the association relationship can be a functional relationship, and the values ​​of the variables can be substituted into the functional relationship to obtain the values ​​of the associated variables.

[0083] After determining the variable values, the initial variable values ​​of the associated variables in the dynamic value set can be updated to the variable values ​​of the associated variables to ensure that the variable values ​​of the associated variables in the dynamic value set match the strategy to be tested.

[0084] Alternatively, the dynamic value set can be updated only after the test results indicate that the strategy module under test has passed the test, thereby ensuring the accuracy of the variable values ​​in the dynamic value set.

[0085] According to embodiments of this disclosure, by updating the variable values ​​of associated variables using the variable values ​​of the strategy module under test, the accuracy of the dynamic value set can be improved, thereby improving the accuracy of the strategy testing process.

[0086] Figure 4 A schematic diagram of a strategy to be tested according to an embodiment of this disclosure is shown.

[0087] like Figure 4As shown, the strategy to be tested includes a first strategy module 410, a second strategy module 420, a third strategy module 430, and a fourth strategy module 440. The first strategy module 410 takes variable i1 as input, processes variable i1, and outputs variable i1. The second strategy module 420 takes variables i1 and i2 as input, processes variables i1 and i2, and outputs variable i3. The third strategy module 430 takes variables i3 and i4 as input, processes variables i3 and i4, and outputs variable i5. The fourth strategy module 440 takes variable i5 as input, processes variable i5, and outputs variable i6.

[0088] Taking the third strategy module 430 as the strategy module to be tested as an example, it can determine whether variables i3 and i4 are fixed variables based on the configuration information of variables i3 and i4, and obtain the variable values ​​of variables i3 and i4 from the fixed value set and the dynamic value set, without executing the strategy components of the first strategy module 410 and the second strategy module 420.

[0089] Taking variable i3 as an example, when variable i3 is a fixed variable, the variable value of variable i3 can be obtained from the set of fixed values. When variable i3 is a dynamic variable, the variable value of variable i3 can be obtained from the set of dynamic values.

[0090] Since variable i5 is obtained by the third strategy module 430 after processing variables i3 and i4, variable i5 is an associated variable of variables i3 and i4. After executing the strategy component of the third strategy module 430 based on variables i3 and i4, the variable value of variable i5 can be determined based on the variable values ​​of variables i3 and i4, and the initial variable value of variable i5 in the dynamic value set can be updated based on the variable value of variable i5.

[0091] According to embodiments of this disclosure, the strategy testing method further includes: determining reference logic for evaluation based on the strategy logic and variable configuration information of the strategy module to be tested; and generating a reference component based on the reference logic.

[0092] Since the reference component is used to evaluate the strategy module to be tested, the reference logic of the reference component needs to reflect the correct logic of the strategy module to be tested. Therefore, the reference logic needs to be determined first based on the strategy logic of the strategy module to be tested, and then the reference component is generated based on the reference logic.

[0093] In the embodiments of this disclosure, the reference logic is determined based on the policy logic of the policy module to be tested and the configuration information of the variables. When determining the reference logic, it can be first determined whether the variable is a fixed variable based on the configuration information. If the configuration information indicates that the variable is a fixed variable, the variable value is obtained from the fixed value set; otherwise, the variable value is obtained from the dynamic value set.

[0094] After determining the variable values, the correct execution result corresponding to the variable values ​​can be determined according to the strategy logic. Then, based on the variable values ​​and the correct execution result, the reference logic can be determined. For example, the reference logic can be expressed in the form of a structured query language, where each sub-logic can correspond to a structured query statement.

[0095] Once the reference logic is determined, it needs to be transformed into an executable reference component. A reference component can be, for example, a script file containing multiple structured query statements. These structured query statements can process variable values ​​and generate reference results, which can then be used to evaluate the module under test.

[0096] According to embodiments of this disclosure, the generation efficiency and accuracy of reference components are improved by utilizing policy logic and configuration information to generate reference components.

[0097] According to embodiments of this disclosure, the execution result is evaluated based on the reference result to obtain the test result, including: if the execution result and the reference result match, determining that the test result of the strategy module to be tested indicates that the strategy module to be tested has passed the test; if the execution result and the reference result do not match, determining that the test result of the strategy module to be tested indicates that the strategy module to be tested has failed the test.

[0098] When evaluating the execution results, since the reference logic of the reference component corresponds to the correct strategy logic of the strategy module under test, the reference result output by the reference component should also be the correct result. Therefore, the execution results can be evaluated using the reference result, thereby verifying the accuracy of the strategy module under test.

[0099] If the execution result matches the reference result, for example, if the execution result and the reference result are the same, it can be determined that the execution result is also correct, the strategy logic of the strategy module under test is correct, and thus it can be determined that the strategy module under test passes the test.

[0100] If the execution result and the reference result do not match, for example, if the execution result and the reference result are different, it can be determined that the execution result is incorrect, the strategy logic of the strategy module under test is wrong, and thus it can be determined that the strategy module under test fails the test.

[0101] According to embodiments of this disclosure, by using reference results to evaluate the execution results, the evaluation of the strategy module to be tested is achieved, thereby improving the accuracy of the test results.

[0102] According to embodiments of this disclosure, the strategy testing method further includes: obtaining a first strategy test result of the strategy to be tested based on the test results of the strategy module to be tested; obtaining a second strategy test result of the strategy to be tested based on the strategy execution results of the strategy to be tested; and obtaining a target test result of the strategy to be tested based on the first strategy test result and the second strategy test result.

[0103] To avoid the impact of updates to the strategy module under test on other strategy modules, the strategy under test can be tested after the strategy module under test is updated to ensure that the overall logic of the strategy under test is correct.

[0104] When determining the test results for the strategy under test, the strategy under test can be evaluated from two aspects. On the one hand, the test results can be based on the test results of local testing of the strategy module under test to determine the first strategy test result for the strategy under test. On the other hand, the strategy execution results can be based on the strategy execution results obtained after global testing of the entire strategy under test to determine the second strategy test result for the strategy under test.

[0105] When determining the first strategy test result for a strategy under test, the test result of the strategy module under test can be considered the first strategy test result. For example, if the test result indicates that the strategy module under test has passed the test, then the first strategy test result also indicates that the strategy module under test has passed the test; conversely, if the test result indicates that the strategy module under test has failed the test, then the first strategy test result also indicates that the strategy module under test has failed the test.

[0106] When determining the test results of the second strategy against the strategy under test, a global test of the strategy under test is required. Specifically, the strategy execution results obtained after executing all reference components of the strategy under test can be used to evaluate the strategy execution results obtained after executing all strategy components of the strategy under test, thus obtaining the test results of the second strategy.

[0107] According to embodiments of this disclosure, the strategy execution result obtained after executing all reference components of the strategy to be tested is evaluated using the reference execution result obtained after executing all reference components of the strategy to be tested. This includes: if the reference execution result matches the strategy execution result, determining that the second strategy test result indicates that the strategy to be tested has passed the test; if the reference execution result does not match the strategy execution result, determining that the second strategy test result indicates that the strategy to be tested has failed the test.

[0108] After determining the test results of the first strategy and the second strategy, the target test result for the entire strategy under test can be determined based on the test results of the first strategy and the second strategy.

[0109] When determining the target test result, if both the first strategy test result and the second strategy test result indicate that the strategy under test has passed the test, the target test result is determined to indicate that the strategy under test has passed the test. Conversely, if either the first strategy test result or the second strategy test result indicates that the strategy under test has failed the test, the target test result is determined to indicate that the strategy under test has failed the test.

[0110] According to embodiments of this disclosure, by performing local testing on the strategy module to be tested and then performing global testing on the strategy to be tested, a more comprehensive verification of the strategy module to be tested can be achieved, thereby improving the accuracy of the strategy to be tested.

[0111] Figure 5 A block diagram of a strategy testing apparatus according to an embodiment of the present disclosure is shown schematically.

[0112] like Figure 5 As shown, the strategy testing device 500 includes a strategy determination module 510, a variable determination module 520, a strategy execution module 530, and a result evaluation module 540.

[0113] The strategy determination module 510 is used to determine the strategy components, reference components, and variables of the strategy module to be tested from the strategy to be tested. The strategy to be tested includes multiple strategy modules, and the strategy module to be tested is at least one of the strategy modules in the strategy to be tested. The strategy components are used to run the strategy module to be tested, and the reference components are used to evaluate the strategy module to be tested. In some embodiments, the strategy determination module 510 can be used to perform the operation S210 described above, which will not be repeated here.

[0114] The variable determination module 520 is used to determine the variable value from a fixed value set when the configuration information of the variable indicates that the variable is a fixed variable. In some embodiments, the variable determination module 520 can be used to perform the operation S220 described above, which will not be repeated here.

[0115] The strategy execution module 530 is used to run the strategy component and the reference component respectively based on the variable values, and obtain the execution result for the strategy component and the reference result for the reference component. In some embodiments, the strategy execution module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0116] The result evaluation module 540 is used to evaluate the execution results based on the reference results to obtain the test results. In some embodiments, the result evaluation module 540 can be used to perform the operation S240 described above, which will not be repeated here.

[0117] According to embodiments of this disclosure, the strategy testing apparatus 500 further includes a variable determination module, an identification module, and a fixed update module.

[0118] The variable determination module is used to parse the policy components of the policy module and determine the variables used to run the policy components, provided that the policy module has been updated.

[0119] The recognition module is used to determine the recognition result based on the variable attributes of the variable.

[0120] The fixed update module is used to update the configuration information of variables to indicate that the variables are fixed variables, provided that the identification result represents the configuration information of the updated variables.

[0121] According to embodiments of this disclosure, the identification module includes a complex identification submodule, a type identification submodule, and a result determination submodule.

[0122] The complexity identification submodule is used to evaluate the complexity of variables based on the variable relationships in the variable attributes, and determine the complexity identification result of the variables. The variable relationships characterize the relationship between the variable and the variables in other strategy modules in the strategy under test.

[0123] The type identification submodule is used to determine the type identification result of a variable based on the variable type in the variable attributes. The variable type includes discrete and continuous types.

[0124] The result determination submodule is used to determine the recognition result based on the complexity recognition result and the type recognition result.

[0125] According to embodiments of this disclosure, the strategy testing apparatus 500 further includes a variable value determination module.

[0126] The variable value determination module is used to determine the variable value from the dynamic value set when the configuration information of the variable indicates that the variable is a dynamic variable.

[0127] According to embodiments of this disclosure, the strategy testing apparatus 500 further includes an association determination module, an association value determination module, and an association value update module.

[0128] The association determination module is used to determine the associated variables related to the strategy module under test from the variable set of the strategy under test. The variable values ​​of the associated variables are related to the variable values ​​of the strategy module under test.

[0129] The correlation value determination module is used to determine the value of the correlation variable based on the variable value of the variable in the strategy module under test.

[0130] The associated value update module is used to update the initial variable values ​​of associated variables in the dynamic value set based on the variable values ​​of associated variables.

[0131] According to embodiments of this disclosure, the strategy testing apparatus 500 further includes a logic determination module and a component generation module.

[0132] The logic determination module is used to determine the reference logic for evaluation based on the policy logic and variable configuration information of the policy module to be tested.

[0133] The component generation module is used to generate reference components based on reference logic.

[0134] According to embodiments of this disclosure, the result evaluation module includes a first determination submodule and a second determination submodule.

[0135] The first determining submodule is used to determine, when the execution result matches the reference result, the test result characterization of the strategy module to be tested, indicating that the strategy module to be tested has passed the test.

[0136] The second determination submodule is used to determine the test result of the strategy module to be tested as failing the test when the execution result and the reference result do not match.

[0137] According to embodiments of this disclosure, the result evaluation module further includes a test determination submodule, an execution determination submodule, and a target determination submodule.

[0138] The test determination submodule is used to obtain the first strategy test result of the strategy to be tested based on the test results of the strategy module to be tested.

[0139] The execution determination submodule is used to obtain the second strategy test result of the strategy under test based on the strategy execution result of the strategy under test. The strategy execution result represents the result obtained after executing all strategy components of the strategy under test.

[0140] The target determination submodule is used to obtain the target test results of the strategy to be tested based on the test results of the first strategy and the test results of the second strategy.

[0141] Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be implemented by dividing them into multiple modules. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as hardware circuitry, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a System-on-Chip, a System-on-a-Substrate, a System-on-Package, an Application-Specific Integrated Circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0142] For example, any plurality of the strategy determination module 510, variable determination module 520, strategy execution module 530, and result evaluation module 550 can be combined into one module / unit / subunit, or any one of these modules / units / subunits can be split into multiple modules / units / subunits. Alternatively, at least part of the functionality of one or more of these modules / units / subunits can be combined with at least part of the functionality of other modules / units / subunits and implemented in one module / unit / subunit. According to embodiments of this disclosure, at least one of the strategy determination module 510, variable determination module 520, strategy execution module 530, and result evaluation module 550 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the strategy determination module 510, variable determination module 520, strategy execution module 530, and result evaluation module 550 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0143] It should be noted that the strategy testing device part in the embodiments of this disclosure corresponds to the strategy testing method part in the embodiments of this disclosure. For a detailed description of the strategy testing device part, please refer to the strategy testing method part, which will not be repeated here.

[0144] Figure 6 A block diagram of an electronic device suitable for implementing the methods described above, according to embodiments of the present disclosure, is illustrated schematically. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0145] like Figure 6 As shown, an electronic device 600 according to an embodiment of this disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this disclosure.

[0146] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Processor 601 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 602 and / or RAM 603. It should be noted that programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.

[0147] According to embodiments of this disclosure, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to a bus 604. The system 600 may also include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0148] According to embodiments of this disclosure, the method flow according to embodiments of this disclosure can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by processor 601, it performs the functions defined in the system of embodiments of this disclosure. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0149] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0150] According to embodiments of this disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0151] For example, according to embodiments of this disclosure, a computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0152] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of this disclosure. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the methods provided in the embodiments of this disclosure.

[0153] When the computer program is executed by the processor 601, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0154] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0155] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0156] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features recited in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not expressly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure may be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0157] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A strategy testing method, comprising: From the strategy to be tested, determine the strategy components, reference components and variables of the strategy module to be tested. The strategy to be tested includes multiple strategy modules. The strategy module to be tested is at least one of the strategy modules in the strategy to be tested. The strategy components are used to run the strategy module to be tested. The reference components are used to evaluate the strategy module to be tested. When the configuration information of the variable indicates that the variable is a fixed variable, the variable value of the variable is determined from the set of fixed values; Based on the variable values, the strategy component and the reference component are run respectively to obtain the execution result for the strategy component and the reference result for the reference component; as well as Based on the reference results, the execution results are evaluated to obtain the test results.

2. The method according to claim 1, further comprising: If it is determined that the strategy module has been updated, the strategy component of the strategy module is parsed to determine the variables used to run the strategy component; The identification result is determined based on the variable attributes of the aforementioned variables; as well as If the identification result indicates that the configuration information of the variable needs to be updated, the configuration information of the variable will be updated to indicate that the variable is a fixed variable.

3. The method according to claim 2, wherein, The determination of the recognition result based on the variable attributes of the variable includes: Based on the variable relationships in the variable attributes, the complexity of the variable is evaluated, and the complexity identification result of the variable is determined. The variable relationships characterize the relationship between the variable and the variables in other strategy modules in the strategy to be tested. Based on the variable type in the variable attributes, the type identification result of the variable is determined, and the variable type includes discrete and continuous types; and Based on the complexity recognition result and the type recognition result, the recognition result is determined.

4. The method according to claim 1, further comprising: If the configuration information of the variable indicates that the variable is a dynamic variable, the variable value of the variable is determined from the set of dynamic values.

5. The method according to claim 4, wherein, After executing the strategy component based on the variable value, the method further includes: From the set of variables of the strategy to be tested, determine the associated variables related to the module of the strategy to be tested, and the variable values ​​of the associated variables are related to the variable values ​​of the module of the strategy to be tested; Based on the variable values ​​of the variables in the strategy module under test, determine the variable values ​​of the associated variables; and The initial value of the associated variable in the dynamic value set is updated based on the value of the associated variable.

6. The method according to claim 1, further comprising: Based on the strategy logic of the strategy module to be tested and the configuration information of the variables, a reference logic for evaluation is determined; as well as The reference component is generated based on the reference logic.

7. The method according to claim 1, wherein, The evaluation of the execution results based on the reference results to obtain test results includes: If the execution result matches the reference result, the test result of the strategy module to be tested is determined to indicate that the strategy module to be tested has passed the test; If the execution result and the reference result do not match, the test result of the strategy module to be tested indicates that the strategy module to be tested has failed the test.

8. The method according to claim 7, further comprising: Based on the test results of the strategy module to be tested, the first strategy test result of the strategy to be tested is obtained; Based on the strategy execution result of the strategy to be tested, a second strategy test result of the strategy to be tested is obtained, wherein the strategy execution result represents the result obtained after executing all strategy components of the strategy to be tested; as well as Based on the test results of the first strategy and the test results of the second strategy, the target test result of the strategy to be tested is obtained.

9. A strategy testing apparatus, comprising: A strategy determination module is used to determine the strategy components, reference components, and variables of the strategy module to be tested from the strategy to be tested. The strategy to be tested includes multiple strategy modules, and the strategy module to be tested is at least one of the strategy modules in the strategy to be tested. The strategy components are used to run the strategy module to be tested, and the reference components are used to evaluate the strategy module to be tested. A variable determination module is used to determine the variable value from a set of fixed values ​​when the configuration information of the variable indicates that the variable is a fixed variable. The strategy execution module is used to run the strategy component and the reference component respectively based on the variable values, and obtain the execution result for the strategy component and the reference result for the reference component; as well as The result evaluation module is used to evaluate the execution result based on the reference result to obtain the test result.

10. An electronic device, comprising: One or more processors; Memory, used to store one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 8.

11. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 8.

12. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images