Test method, device and frame

Abstract

The embodiment of the invention discloses a test method, device and framework, after a test demand is determined in a graph scheduling test task, values of test parameters are determined according to the test demand, and the test parameters comprise a test case calling path and at least one public parameter extracted from a test task scheduling framework and a test case framework; transmitting values of test parameters to a test case framework through a test task scheduling framework, calling corresponding test cases based on the values of the test parameters through the test case framework, determining a to-be-executed test case set, and executing the test cases in the test case set based on the values of the test parameters through the test case framework. And generating a test report. Therefore, the configuration of test parameters can be simplified, and the test efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of software testing, and in particular to a testing method, device and framework. Background Art

[0002] With the development of science and technology, software testing is being used more and more widely, and various testing method frameworks are emerging one after another. However, most testing methods have complex configurations of test parameters, which affects testing efficiency, especially for graph scheduling testing tools. Summary of the invention

[0003] In view of this, embodiments of the present invention provide a testing method, device and framework to simplify test parameter configuration and improve test efficiency.

[0004] In a first aspect, a method for testing graph scheduling is provided, the method comprising:

[0005] Determine test requirements based on the graph scheduling test tasks;

[0006] Determining a value of a test parameter according to the test requirement, the test parameter including a test case calling path and at least one common parameter extracted from a test task scheduling framework and a test case framework;

[0007] Passing the value of the test parameter to the test case framework through the test task scheduling framework;

[0008] Calling corresponding test cases based on the values ​​of the test parameters through the test case framework to determine a set of test cases to be executed;

[0009] The test cases in the test case set are executed through the test case framework based on the values ​​of the test parameters to generate a test report.

[0010] In some embodiments, determining a set of test cases to be executed includes:

[0011] Select test cases according to the test case screening parameters to determine the test case set to be executed.

[0012] In some embodiments, the test case screening parameters include preconfigured selection parameters and screening parameters, and selecting test cases according to the test case screening parameters and determining the test case set to be executed includes:

[0013] Determining a first test case set based on the selection parameters;

[0014] Determining a second test case set based on the screening parameters;

[0015] An intersection of the first test case set and the second test case set is determined as a test case set to be executed.

[0016] In some embodiments, determining a set of test cases to be executed includes:

[0017] In response to the test requirement being a problem reproduction test requirement, the value of the test parameter includes a time node when the problem occurred, and the executed test cases are automatically selected based on the time node when the problem occurred to determine a set of test cases to be executed.

[0018] In some embodiments, determining a set of test cases to be executed includes:

[0019] In response to the test requirement being a problem reproduction test requirement, executed test cases are selected according to pre-set selection parameters and screening parameters to determine a set of test cases to be executed.

[0020] In some embodiments, the method further comprises:

[0021] A test environment is configured according to the test parameters to execute the test case.

[0022] In some embodiments, configuring the test environment includes:

[0023] In response to the existence of the official release package, installing the official release package to configure the test environment;

[0024] In response to the absence of an official release package, a test environment is configured using a source code compilation and installation method.

[0025] In some embodiments, generating a test report includes:

[0026] Scheduling and executing the test cases in the test case set through the test case framework to generate test results;

[0027] Comparing the test result with the expected result in the configuration file of the model under test through the test case framework to determine the comparison result;

[0028] The comparison result is sent to the test task scheduling framework through the test case framework to generate a test report, which includes the test result, the expected result and the comparison result between the test result and the expected result.

[0029] In some embodiments, the test case framework adds a test case by calling a test case adding interface.

[0030] In a second aspect, a graph scheduling test framework is provided, the test framework comprising:

[0031] The common parameter extraction module is used to extract common parameters according to the graph scheduling test task scheduling framework and the test case framework;

[0032] A test task scheduling framework, used to receive a test requirement, determine a value of a test parameter according to the test requirement, and pass the value of the test parameter to the test case framework, wherein the test parameter includes a test case call path and at least one common parameter extracted from the test task scheduling framework and the test case framework;

[0033] The test case framework is used to receive the value of the test parameter transmitted by the test task scheduling framework to determine the value of the call path, and call and execute the corresponding test case based on the value of the call path to generate a test result.

[0034] In some embodiments, the test case framework includes a test case adding interface and a test case result comparison interface, wherein the test case adding interface is used to add a test case, and the test case result comparison interface is used to compare the test result with the expected result.

[0035] In a third aspect, a graph scheduling test device is provided, the test device comprising:

[0036] A test requirement determination module, configured to determine test requirements according to the graph scheduling test tasks;

[0037] A test parameter determination module is configured to determine the value of a test parameter according to the test requirement, wherein the test parameter includes a test case call path and at least one common parameter extracted from a test task scheduling framework and a test case framework;

[0038] A test parameter transfer module, configured to transfer the value of the test parameter to the test case framework through the test task scheduling framework;

[0039] A test case calling module is configured to call corresponding test cases based on the values ​​of the test parameters through the test case framework to determine a set of test cases to be executed;

[0040] The test case execution module is configured to execute the test cases in the test case set based on the values ​​of the test parameters through the test case framework to generate a test report.

[0041] According to a fourth aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in the first aspect.

[0042] According to a fifth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0043] After the embodiment of the present invention determines the test requirements through graph scheduling test tasks, the values ​​of test parameters are determined according to the test requirements, the test parameters include the test case calling path and at least one common parameter extracted from the test task scheduling framework and the test case framework, and then the values ​​of the test parameters are transferred to the test case framework through the test task scheduling framework, the corresponding test cases are called based on the values ​​of the test parameters through the test case framework, the test case set to be executed is determined, the test cases in the test case set are executed based on the values ​​of the test parameters through the test case framework, and a test report is generated. Therefore, this embodiment can simplify the configuration of test parameters and improve test efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0045] Figure 1 is a flow chart of a test method for graph scheduling according to an embodiment of the present invention;

[0046] Figure 2 is a flow chart of determining a set of test cases to be executed according to an embodiment of the present invention;

[0047] Figure 3 is a flow chart of generating a test report for graph scheduling according to an embodiment of the present invention;

[0048] Figure 4 is a flowchart of a test task execution of a graph scheduling according to an embodiment of the present invention;

[0049] Figure 5 is a schematic diagram of a test framework for graph scheduling according to an embodiment of the present invention;

[0050] Figure 6 is a schematic diagram of another graph scheduling test framework according to an embodiment of the present invention;

[0051] Figure 7 is a schematic diagram of a test device for graph scheduling according to an embodiment of the present invention;

[0052] Figure 8 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The present application is described below based on embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, some specific details are described in detail. It is possible for those skilled in the art to fully understand the present application without the description of these details. In order to avoid confusing the essence of the present application, known methods, processes, flows, components and circuits are not described in detail.

[0054] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0055] Unless the context clearly requires otherwise, the words "include", "comprising" and similar words throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, the meaning is "including but not limited to".

[0056] In the description of this application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.

[0057] The graph scheduling test tool is used to test the graph scheduling scheme. That is, when designing a graph scheduling scheme, the intermediate (relay) file of the model is analyzed to check whether the running results are consistent with expectations, so as to ensure that users can observe the scheduling results during the design of the graph scheduling scheme and improve development efficiency. The test method of the existing graph scheduling tool is relatively complex to configure the test parameters and generally requires manual configuration, which will seriously affect the test efficiency.

[0058] Figure 1 FIG. 1 is a flow chart of a test method for graph scheduling according to an embodiment of the present invention. Figure 1 As shown, the graph scheduling testing method of this embodiment includes the following steps:

[0059] Step S110, determining the test requirements according to the graph scheduling test tasks.

[0060] The graph scheduling test task is used to test whether the operation result of the graph scheduling solution is consistent with expectations. The test requirements include information related to the graph scheduling test task, such as the model under test and the test content. In some implementations, the types of test requirements include release test requirements, problem reproduction test requirements, special scenario test requirements, etc.

[0061] Step S120, determining the value of the test parameter according to the test requirement, wherein the test parameter includes the test case calling path and at least one common parameter extracted from the test task scheduling framework and the test case framework.

[0062] Among them, the test case framework is used to add, manage and schedule test cases. The test task scheduling framework is used to receive and manage test tasks, and based on the corresponding test task scheduling test case framework, execute the corresponding test case to obtain the test result. For the graph scheduling test task, the test task scheduling framework and the test case framework both need test case scheduling path, log path and other parameters. For each graph scheduling test task, if the test task scheduling framework and the test case framework both need to parse the graph scheduling test task to obtain relevant parameters, it obviously causes a waste of computing resources. Therefore, this embodiment extracts the common parameters of the test task scheduling framework and the test case framework in advance, and for each graph scheduling test task, receives the test requirements through the test task scheduling framework, determines the value of the test parameter according to the test requirement, and passes the value of the test parameter to the test case framework. The test case framework can receive the value of the test parameter passed by the test task scheduling framework, and determine the specific call path of the corresponding test case according to the value of the test parameter, so as to call and execute the corresponding test case based on the call path to generate the test result. The test case framework can also pass the test result to the test task scheduling framework to generate a test report. Thus, the test task scheduling framework and the test case framework are integrated, and test parameter values, test results and other data related to the test task can be transmitted to each other, which reduces computing resources, simplifies parameter configuration and improves test efficiency.

[0063] Based on the test parameters, the corresponding test case can be called, and then the test case can be executed to complete the corresponding graph scheduling test task. Specifically, the common parameters include parameters for configuring the path, log path, compilation mode, etc. for test case calls. The value of the test parameter is the corresponding test case call path, log path, compilation mode, etc. determined according to specific test requirements. It should be understood that the value of the test parameter changes accordingly according to changes in test requirements.

[0064] For example, when the test requirement is a problem reproduction test requirement, the calling path of the test case for the problem reproduction test is determined according to the problem reproduction test requirement. Through the test case calling path, the corresponding test case to be executed can be obtained, and then the problem reproduction test is completed by executing the test case to be executed.

[0065] Step S130: passing the value of the test parameter to the test case framework through the test task scheduling framework.

[0066] In some implementations, since the test parameters are common parameters between the test task scheduling framework and the test task framework, and the path, log path, compilation mode, etc. (i.e., the value of the test parameter) called by the test case are unchanged in a graph scheduling test task. Therefore, after determining the value of the corresponding test parameter according to a graph scheduling test task, the value of the corresponding test parameter can be passed to the test case framework through the test task scheduling framework, thereby simplifying parameter configuration and improving test efficiency.

[0067] Step S140: Calling corresponding test cases based on the values ​​of the test parameters through the test case framework to determine a set of test cases to be executed.

[0068] Specifically, the specific calling path of the corresponding test case is determined based on the value of the test parameter, and the corresponding test case is called based on the calling path, and the test case set to be executed is a test case set composed of test cases obtained based on the calling path. In some implementations, the calling path is the calling path of the test case set to be executed, and the test case set to be executed corresponding to the corresponding test requirement can be directly called through the calling path, thereby completing the corresponding graph scheduling test task. Optionally, after calling the test case set to be executed, all test cases in the test case set can be executed in parallel to obtain test results. Alternatively, after calling the test case set, the test cases in the test case set can be cached in the memory, and then the test cases are executed one by one. In other implementations, the calling path is the calling path of the test case to be executed, and the test cases to be executed corresponding to the corresponding test requirements are called one by one through the calling path, thereby completing the corresponding graph scheduling test task. Optionally, each time a test case is called, the test case can be directly executed, or the test cases can be called one by one and stored in the memory until all the test cases to be executed corresponding to the current graph scheduling test task are called and completed, and then all the test cases are executed in parallel. Further, optionally, some of the test cases stored in the memory are executed in parallel.

[0069] In some implementations, test cases may be selected according to test case screening parameters to determine the set of test cases to be executed. In some implementations, the test case screening parameters include pre-configured selection parameters and drop parameters. Specifically, the steps of determining the set of test cases to be executed are as follows: Figure 2 As shown, including:

[0070] Step S210: determining a first test case set based on the selection parameters.

[0071] The selection parameters are pre-configured according to the test requirements, and are used to select corresponding test cases according to the test requirements, and the selected test cases are used as the first test case set.

[0072] Step S220: determining a second test case set based on the screening parameters.

[0073] The screening parameters are pre-configured according to the test requirements, and are used to delete the corresponding test cases according to the test requirements, and then use the undeleted test cases as the second test case set.

[0074] Step S230: determine the intersection of the first test case set and the second test case set as the test case set to be executed.

[0075] Therefore, the test cases are screened by selecting and filtering parameters, and then the test case set to be executed is determined, which simplifies the selection of test cases and improves the test efficiency.

[0076] In one embodiment, when the test requirement is a problem reproduction test requirement, the value of the test parameter includes the time node when the problem occurred, and determining the set of test cases to be executed includes automatically working back according to the time node when the problem occurred to select executed test cases to determine the set of test cases to be executed.

[0077] Since when the test requirement is a test requirement for reproducing the problem, the test cases that have been executed before the time node when the problem occurs are determined, the problem can be reproduced by back-testing the test case set based on the executed test cases. Specifically, the test case executed at the time node when the problem occurs is the starting test case for back-testing, and the executed test cases are selected by automatically back-testing according to the back-testing step length. For example, when the back-testing step length is set to 1, the test case executed at the time node when the problem occurs is the starting test case for back-testing, and 1 back-testing case is added to the starting test case, and the starting test case and the back-test case are automatically formed into a test case set, and the back-test case set is executed a fixed number of times, and the number of problem recurrences is recorded to calculate the problem recurrence probability. To ensure the accuracy of the recurrence probability, the fixed number of times is not 1, and it can be set to a value as large as possible according to the actual test requirements. When a test case set that reproduces the problem 100% appears, the problem can be directly located according to the test case set. If there is no test case set that reproduces the problem 100%, the test case set is increased in steps until all executed test cases are added to the test case set, thereby improving the efficiency of problem location.

[0078] In another embodiment, when the test requirement is a problem reproduction test requirement, determining the set of test cases to be executed includes selecting executed test cases according to pre-set selection parameters and screening parameters to determine the set of test cases to be executed.

[0079] Specifically, the values ​​of the selection parameters and the screening parameters are determined according to the problem reproduction test requirements, and the test cases that have been executed before the time node when the problem occurs are screened to determine the test case set to be executed. The specific steps for determining the test case set to be executed can be referred to steps S210 to S230, which will not be repeated here.

[0080] Step S150: executing the test cases in the test case set based on the values ​​of the test parameters through the test case framework to generate a test report.

[0081] Specifically, the steps of generating a test report are as follows: Figure 3 As shown, including:

[0082] Step S310, scheduling and executing the test cases in the test case set through the test case framework to generate test results.

[0083] In some implementations, the test case framework schedules and executes the test cases in the test case set through a test case addition interface, thereby generating a test result. The test cases can be executed one by one, or multiple test cases can be executed in parallel. Specifically, the number of test cases executed in parallel can be configured through a test task scheduling framework.

[0084] In some implementations, the test case framework can also add test cases by calling the test case addition interface. The added test cases can be test cases designed and written according to the test requirements, or they can be reused test cases of other models. In this way, the code reuse rate can be improved, and when problems occur, test cases can be quickly added through the test case addition interface to locate the problem.

[0085] In some embodiments, the test framework also includes interfaces such as a test result comparison interface and a log call interface. The test result comparison interface is used to compare the test results with the expected results, and the log call interface is used to call the log based on the log path to store the log information in the graph scheduling test task.

[0086] Step S320, comparing the test result with the expected result in the configuration file of the model under test through the test case framework to determine the comparison result.

[0087] The configuration file of the model under test is configured according to the specific graph scheduling test task before the execution of the graph scheduling test task, which includes the pre-set expected results that can be obtained by this graph scheduling test task. In some implementations, the file format of the test results can be converted into a JSON file format through a format conversion tool, and the test results and expected results are automatically compared by calling the test result comparison interface to determine the comparison results. For example, when the model under test is a graph scheduling model, and the test requirement is the accuracy of the grouping and segmentation of the graph scheduling model operators, the expected operator grouping and segmentation results are set in the graph scheduling model configuration file in advance, and the graph scheduling model is tested by reusing the corresponding test case set in the test case library, and the test case execution results are compared with the number of operator groups and the number of operator groupings in the configuration file of the graph scheduling model. The number of segmentation, and then determine whether the comparison results are consistent. Step S330, send the comparison results to the test task scheduling framework through the test case framework to generate a test report.

[0088] The test report includes the test results, expected results, and the comparison results between the test results and the expected results. Based on the test report, it is possible to visually observe whether the execution test results of the test case are consistent with the expected results. When the two are consistent, it means that there is no problem with the tested model. When the two are inconsistent, it means that there is a problem with the tested model. Users can perform tests such as problem reproduction according to actual conditions.

[0089] In some embodiments, the test method further includes configuring a test environment for the graph scheduling test task according to the test parameters to execute the test case to generate a test result. If there is an official release package in the current operating system, the official release package is installed to configure the test environment, which can facilitate the release test under the above test environment. If there is no official release package in the current operating system, the test environment is configured using the source code compilation and installation method, which is convenient for debugging the development source code and tracing the problem under this test environment. Thus, single case testing or use case set testing can be performed under different operating systems and in different test environments.

[0090] The embodiment of the present invention determines the test requirements by scheduling the test task in a graph, and then determines the value of the test parameter according to the test requirements, wherein the test parameter includes the test case calling path and at least one common parameter extracted from the test task scheduling framework and the test case framework, and then transmits the value of the test parameter to the test case framework through the test task scheduling framework, and calls the corresponding test case based on the value of the test parameter through the test case framework, determines the test case set to be executed, and executes the test cases in the test case set based on the value of the test parameter through the test case framework, and generates a test report. Therefore, the present embodiment can simplify the configuration of the test parameters and improve the test efficiency.

[0091] Figure 4 is a flowchart of a test task execution of a graph scheduling in an embodiment of the present invention, such as Figure 4 As shown, the specific test steps include:

[0092] Step S401, obtaining a graph scheduling test task.

[0093] Step S402, determine the test requirements according to the graph scheduling test tasks.

[0094] Step S403: Determine the value of the test parameter according to the test requirement.

[0095] The test parameters include a test case calling path and at least one common parameter extracted from a test task scheduling framework and a test case framework.

[0096] Specifically, the common parameters include parameters for configuring the test case call path, log path, compilation mode, etc. The value of the test parameter is the corresponding test case call path, log path, compilation mode, etc. determined according to the specific test requirements. For example, when the test requirement is a problem reproduction test requirement, the value of the test parameter also includes the time node when the problem occurs. According to the problem reproduction test requirement, the call path of the test case for reproducing the problem test can be determined, and through the test case call path, the corresponding test case to be executed can be obtained, and then the problem reproduction test is completed by executing the test case to be executed.

[0097] Step S404, determining whether there is an official release package.

[0098] If there is an official release package in the current operating system, execute step S405, otherwise execute step S406.

[0099] Step S405: Install the official release package to configure the test environment.

[0100] If there is an official release package in the current operating system, the official release package is installed to configure the test environment, and the release test can be easily performed under the above test environment.

[0101] Step S406, configuring the test environment using source code compilation and installation.

[0102] If there is no official release package in the current operating system, the test environment is configured using the source code compilation and installation method, which facilitates debugging the development source code and tracing the problem. Thus, through steps S404-S406, single case test or case set test can be implemented in different operating systems and different test environments, improving the versatility of the test framework.

[0103] Step S407, determining whether it is necessary to select a test case according to the screening parameters.

[0104] If it is necessary to select test cases according to the screening parameters, execute step S408, otherwise execute step S409.

[0105] Step S408, selecting test cases according to the screening parameters, and determining a set of test cases to be executed.

[0106] The test case screening parameters include pre-configured selection parameters and screening parameters. According to the selection parameters and screening parameters and based on the test requirements, the test cases in the corresponding test case set in the test case library can be screened to determine the test case set to be executed. The specific process of determining the test case can refer to steps S210-S230, which will not be repeated here. Among them, the test case library stores test case sets for different test requirements, and the test cases in the above test case set are pre-written and reusable test cases. According to different test requirements, corresponding test cases or test case sets can be selected from the test case library for testing.

[0107] In some implementations, when the test requirement is a problem reproduction requirement, the test cases can be automatically reversed according to the time node when the problem occurred to determine the test case set to be executed.

[0108] Specifically, the test case that has been executed can be automatically reversed according to the time node when the problem occurs to select the test case that has been executed and determine the test case set. Since when the test requirement is a test requirement for reproducing the problem, the test case that has been executed before the time node when the problem occurs is determined, the test case set can be reversed based on the executed test case to reproduce the problem. Specifically, the test case executed at the time node when the problem occurs is the starting test case for reversed, and the executed test case is selected according to the reversed step length. For example, when the reversed case step length is set to 1, the test case executed at the time node when the problem occurs is the starting test case for reversed, and 1 reversed case is added to the starting test case, the starting test case and the reversed test case are automatically formed into a test case set, and the reversed test case set is executed a fixed number of times, and the number of times the problem recurs is recorded to calculate the probability of problem recurrence. To ensure the accuracy of the probability of recurrence, the fixed number of times is not 1, and it can be set to a value as large as possible according to the actual test requirements. When a test case set that reproduces the problem 100% appears, the test case set is determined as the test case set to be executed. If there is no test case set that can reproduce the problem 100%, the test case set will be increased in steps until all executed test cases have been added to the test case set, and the test case set consisting of all executed test cases will be determined as the test case set to be executed.

[0109] Step S409: determine the corresponding test case set in the test case library as the test case set to be executed.

[0110] Specifically, the test case set corresponding to the test requirement in the test case library is directly determined as the test case set to be executed. There is no fixed execution order relationship between step S404 and step S407. In an optional implementation, step S407 is executed first and then step S404. In an optional implementation, step S404 and step S407 are executed simultaneously.

[0111] Step S410, executing the test cases in the test case set to generate test results.

[0112] In some implementations, the test case framework can schedule and execute the test cases in the test case set to generate test results. Specifically, the test case framework schedules and executes the test cases in the test case set through a case adding interface to generate test results.

[0113] In some implementations, the test case framework can also add test cases by calling the test case addition interface. The added test cases can be test cases designed and written according to the test requirements, or they can be reused test cases of other models. In this way, the code reuse rate can be improved, and when problems occur, test cases can be quickly added through the test case addition interface to locate the problem.

[0114] In some embodiments, the test framework also includes interfaces such as a test result comparison interface and a log call interface. The test result comparison interface is used to compare the test result with the expected result, and the log call interface is used to call the log based on the log path to store the log information in the test task. The expected result is pre-configured in the configuration file of the model under test, which includes the expected result that can be obtained in this test task.

[0115] Step S411, comparing the test result with the expected result.

[0116] In some implementations, the file format of the test result can be converted into the JSON file format through a format conversion tool, and the test result comparison interface can be called to automatically compare the data contained in the test result and the expected result, thereby determining the comparison result. For example, the number of groups in the two is compared to see if they are consistent, and the number of groups after segmentation is consistent, thereby determining whether the comparison results are consistent.

[0117] Step S412: Generate a test report.

[0118] Specifically, the comparison result is sent to the test task scheduling framework through the test case framework to generate a test report. The test report includes the test result, the expected result, and the comparison result between the test result and the expected result. Based on the test report, it can be intuitively observed whether the execution test result of the test case is consistent with the expected result. When the two are consistent, it means that there is no problem with the model under test. When the two are inconsistent, it indicates that there is a problem with the model under test. Users can perform tests such as problem reproduction according to actual conditions.

[0119] The embodiment of the present invention determines the test requirements by scheduling the test task in a graph, and then determines the value of the test parameter according to the test requirements, wherein the test parameter includes the test case calling path and at least one common parameter extracted from the test task scheduling framework and the test case framework, and then transmits the value of the test parameter to the test case framework through the test task scheduling framework, and calls the corresponding test case based on the value of the test parameter through the test case framework, determines the test case set to be executed, and executes the test cases in the test case set based on the value of the test parameter through the test case framework, and generates a test report. Therefore, the present embodiment can simplify the configuration of the test parameters and improve the test efficiency.

[0120] Figure 5 Schematic diagram of a test framework for graph scheduling according to an embodiment of the present invention. Figure 5 As shown, the test framework includes a common parameter extraction module 51 , a test task scheduling framework 52 and a test case framework 53 .

[0121] Among them, the common parameter extraction module 51 is used to extract common parameters according to the graph scheduling test task scheduling framework and the test case framework. The test task scheduling framework 52 is used to receive the test requirements, determine the value of the test parameter according to the test requirements, and pass the value of the test parameter to the test case framework, and the test parameter includes the test case call path and at least one common parameter extracted from the test task scheduling framework and the test case framework. The test case framework 53 is used to receive the value of the test parameter passed by the test task scheduling framework to determine the value of the call path, and call and execute the corresponding test case based on the value of the call path to generate a test result.

[0122] In some embodiments, Figure 6As shown, the test framework A also includes an environment configuration module 54, which is used to configure the test environment for executing the test case according to the test parameter configuration. If there is a formal release package in the current operating system, the formal release package is installed to configure the test environment, which can facilitate the release test under the above test environment. If there is no formal release package in the current operating system, the test environment is configured using the source code compilation and installation method, which is convenient for debugging the development source code and tracing the problem under this test environment. Thus, single case test or use case set test can be performed under different operating systems and different test environments.

[0123] In some implementations, the common parameters extracted by the common parameter extraction module 51 from the test task scheduling framework and the test case framework include test case call path configuration parameters 511, log path configuration parameters 512, compilation mode configuration parameters 513 and other common parameters 514. When the test task scheduling framework 52 receives the test requirements, it determines the values ​​of the test parameters according to the test requirements, that is, determines the values ​​of the corresponding test case call path configuration parameters 511, log path configuration parameters 512 and compilation mode configuration parameters 513. The test task scheduling framework 52 passes the values ​​of the above-determined test case call path configuration parameters 511, log path configuration parameters 512 and compilation mode configuration parameters 513 to the test case framework 53 layer by layer, so that the test case general interface extraction module 57 in the test case framework 53 can call, execute and compare the results of the test case, thereby completing the corresponding graph scheduling test task.

[0124] In some implementations, the test case general interface extraction module 57 includes a case addition interface 571, a test result comparison interface 572, and a log call interface 573. The case addition interface 571 is used to schedule and execute the test cases in the test case set and the newly added test cases. The newly added test cases can be test cases designed and written according to the test requirements, or they can be test cases of other models that are reused. The test result comparison interface 572 is used to compare the test results with the expected results. The log call interface 573 is used to call the log based on the log path to store the log information in the graph scheduling test task.

[0125] Furthermore, the test cases can be called and executed one by one or multiple test cases can be called and executed in parallel through the test case adding interface 571. Specifically, the number of test cases executed in parallel can be configured through the task scheduler 55.

[0126] The test case framework 53 sends the test result to the test task scheduling framework 52 to generate a corresponding test report, which includes the test result, the expected result, and the comparison result between the test result and the expected result.

[0127] For example, when the graph scheduling test task requirement is a problem reproduction test requirement, the test case framework 53 can call the corresponding test case through the test case addition interface 571 based on the test case call path configuration parameter 511 transmitted by the test task scheduling framework 52 to obtain the test case set.

[0128] Among them, the process of selecting test cases to determine the test case set can be implemented in at least the following two ways. In one implementation, the test case selection and screening unit 56 in the test task scheduling framework 52 can select test cases based on the test case screening parameters according to the test requirements, so as to determine the test case set to be executed. The test case screening parameters include pre-configured selection parameters and screening parameters. The specific process of determining the test case can refer to steps S210-S230. In another implementation, since when the test requirement is a problem reproduction test requirement, the value of the test parameter includes the time node when the problem occurs, it is also possible to automatically reverse the time node when the problem occurs to select the executed test case and determine the test case set to be executed.

[0129] Thus, the test case framework 53 calls and executes the test cases in the above-mentioned test case set to be executed through the case addition interface 571, compares the test case execution results with the expected results in the configuration file of the model under test, and determines the comparison results. The test case framework 53 sends the test results to the test task scheduling framework 52 to generate a corresponding test report and complete the problem reproduction test.

[0130] The embodiment of the present invention determines the test requirements by scheduling the test task in a graph, and then determines the value of the test parameter according to the test requirements, wherein the test parameter includes the test case calling path and at least one common parameter extracted from the test task scheduling framework and the test case framework, and then transmits the value of the test parameter to the test case framework through the test task scheduling framework, and calls the corresponding test case based on the value of the test parameter through the test case framework, determines the test case set to be executed, and executes the test cases in the test case set based on the value of the test parameter through the test case framework, and generates a test report. Therefore, the present embodiment can simplify the configuration of the test parameters and improve the test efficiency.

[0131] Figure 7 Schematic diagram of a test device for graph scheduling according to an embodiment of the present invention. Figure 7As shown, the device includes a test requirement determination module 71, a test parameter determination module 72, a test parameter transfer module 73, a test case calling module 74 and a test case execution module 75.

[0132] Among them, the test requirement determination module 71 is configured to determine the test requirements according to the graph scheduling test task. The test parameter determination module 72 is configured to determine the value of the test parameter according to the test requirement, and the test parameter includes a test case call path and at least one common parameter extracted from the test task scheduling framework and the test case framework. The test parameter transfer module 73 is configured to transfer the value of the test parameter to the test case framework through the test task scheduling framework. The test case calling module 74 is configured to call the corresponding test case based on the value of the test parameter through the test case framework to determine the test case set to be executed. The test case execution module 75 is configured to execute the test cases in the test case set based on the value of the test parameter through the test case framework to generate a test report.

[0133] Further, the test case calling module 74 also includes a first determination unit and a second determination unit, the first determination unit is configured to select test cases according to the test case screening parameters to determine the test case set to be executed, and the second determination unit is configured to automatically reverse the time node of the problem to select the executed test cases in response to the test requirement being a test requirement for reproducing the problem, and the value of the test parameter includes the time node of the problem occurrence, and determine the test case set to be executed. The test case execution module 75 also includes a first generation module, a comparison module, and a second generation module, the first generation module is configured to schedule and execute the test cases in the test case set through the test case framework to generate test results, the comparison module is configured to compare the test results with the expected results in the configuration file of the model under test through the test case framework to determine the comparison results, and the second generation module is configured to send the comparison results to the test task scheduling framework through the test case framework to generate a test report, and the test report includes the test results, the expected results, and the comparison results of the test results and the expected results.

[0134] In some optional implementations, the test case screening parameters include preconfigured selection parameters and screening parameters, the first determination unit includes a first test case set determination unit, a second test case set determination unit and a test case set to be executed determination unit, the first test case set determination unit is configured to determine the first test case set based on the selection parameters, the second test case set determination unit is configured to determine the second test case set based on the screening parameters, and the test case set determination unit to be executed is configured to determine the intersection of the first test case set and the second test case set as the test case set to be executed.

[0135] The embodiment of the present invention determines the test requirements by scheduling the test task in a graph, and then determines the value of the test parameter according to the test requirements, wherein the test parameter includes the test case calling path and at least one common parameter extracted from the test task scheduling framework and the test case framework, and then transmits the value of the test parameter to the test case framework through the test task scheduling framework, and calls the corresponding test case based on the value of the test parameter through the test case framework, determines the test case set to be executed, and executes the test cases in the test case set based on the value of the test parameter through the test case framework, and generates a test report. Therefore, the present embodiment can simplify the configuration of the test parameters and improve the test efficiency.

[0136] An embodiment of the present invention further provides an electronic device. Figure 8 Schematic diagram of an electronic device according to an embodiment of the present invention. Figure 8 As shown, Figure 8 The electronic device shown is a general address query device, which includes a general computer hardware structure, which at least includes a processor 81 and a memory 82. The processor 81 and the memory 82 are connected via a bus 83. The memory 82 is suitable for storing instructions or programs executable by the processor 81. The processor 81 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 81 executes the instructions stored in the memory 82, thereby executing the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 83 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 84 and the display device and the input / output (I / O) device 85. The input / output (I / O) device 85 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 85 is connected to the system via an input / output (I / O) controller 86.

[0137] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices (equipment) or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0138] The present application is described with reference to flowcharts of methods, apparatuses (devices) and computer program products according to embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions.

[0139] These computer program instructions may be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device that implements the process Figure 1 A function specified in a process or multiple processes.

[0140] These computer program instructions may also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the instructions for implementing the process Figure 1 A device that specifies functions in a process or multiple processes.

[0141] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used for a computer to execute part or all of the above method embodiments.

[0142] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by specifying relevant hardware through a program, and the program is stored in a storage medium, including several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0143] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A graph scheduling testing method, characterized in that: The method comprises: Determine test requirements based on the graph scheduling test tasks; Determining the value of a test parameter according to the test requirement, the test parameter including: a test case calling path, and at least one common parameter extracted from a test task scheduling framework and a test case framework; Passing the value of the test parameter to the test case framework through the test task scheduling framework; Calling corresponding test cases based on the values ​​of the test parameters through the test case framework to determine a set of test cases to be executed; The test cases in the test case set are executed through the test case framework based on the values ​​of the test parameters to generate a test report.

2. The method according to claim 1, characterized in that Determining the set of test cases to be executed includes: Select test cases according to the test case screening parameters to determine the test case set to be executed.

3. The method according to claim 2, characterized in that The test case screening parameters include pre-configured selection parameters and screening parameters, and the test case is selected according to the test case screening parameters to determine the test case set to be executed includes: Determining a first test case set based on the selection parameters; Determining a second test case set based on the screening parameters; An intersection of the first test case set and the second test case set is determined as a test case set to be executed.

4. The method according to claim 1, characterized in that: Determining the set of test cases to be executed includes: In response to the test requirement being a problem reproduction test requirement, the value of the test parameter includes a time node when the problem occurred, and the executed test cases are automatically selected based on the time node when the problem occurred to determine a set of test cases to be executed.

5. The method according to claim 1, characterized in that: Determining the set of test cases to be executed includes: In response to the test requirement being a problem reproduction test requirement, executed test cases are selected according to pre-set selection parameters and screening parameters to determine a set of test cases to be executed.

6. The method according to claim 1, characterized in that The method further comprises: A test environment is configured according to the test parameters to execute the test case.

7. The method according to claim 6, characterized in that The configuration test environment includes: In response to the existence of the official release package, installing the official release package to configure the test environment; In response to the absence of an official release package, a test environment is configured using a source code compilation and installation method.

8. The method according to claim 1, characterized in that: The test case framework adds test cases by calling a test case adding interface.

9. A graph scheduling test framework, characterized in that: The test framework includes: The common parameter extraction module is used to extract common parameters according to the graph scheduling test task scheduling framework and the test case framework; A test task scheduling framework, used to receive a test requirement, determine a value of a test parameter according to the test requirement, and pass the value of the test parameter to the test case framework, wherein the test parameter includes a test case call path and at least one common parameter extracted from the test task scheduling framework and the test case framework; The test case framework is used to receive the value of the test parameter transmitted by the test task scheduling framework to determine the value of the call path, and call and execute the corresponding test case based on the value of the call path to generate a test result.

10. The test framework according to claim 9, characterized in that: The test case framework includes a test case adding interface and a test case result comparison interface. The test case adding interface is used to add a test case, and the test case result comparison interface is used to compare the test result with the expected result.

Images