Object testing method and device

Abstract

The embodiments of this specification provide an object testing method and device, wherein the method is applied to a test platform, and the test platform stores a test suite corresponding to at least one test type, including determining a test task for a target object; determining a target test suite corresponding to the test task in the test suite corresponding to the at least one test type according to the test task, wherein the target test suite includes a test configuration and a test script, and the test configuration is obtained by converting the target variable parameters of the target test suite; and the test task is executed according to the test script according to the test configuration to obtain a test result of the target object. By integrating test suites of different test types into the test platform, the test configuration and test scripts of the test suite are managed in an integrated and standardized manner, which facilitates the storage of the test suite, and tests are performed according to the test configuration and the test script, thereby simplifying the test process for the target object and improving the test efficiency.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of software testing technology, and more particularly to an object testing method and device. Background Art

[0002] With the continuous development of information technology, in order to improve the testing efficiency in the software testing process, test cases can be used for testing. In the existing unit testing, testers usually write a fixed test suite, which contains a series of test cases for multi-faceted testing of software or products. In actual testing, multiple types of tests are often conducted independently, such as functional testing, performance testing, compatibility testing, security testing, etc. Since different types of tests are conducted in a decentralized manner, the testing efficiency is low and the management is complex, which increases the probability of errors in the testing process. Therefore, how to uniformly manage the test execution process and improve the test efficiency is an urgent problem to be solved. Summary of the invention

[0003] In view of this, an embodiment of the present specification provides an object testing method. One or more embodiments of the present specification also relate to an object testing device, a computing device, a computer-readable storage medium and a computer program product to solve the technical defects existing in the prior art.

[0004] According to a first aspect of an embodiment of this specification, there is provided an object testing method, which is applied to a testing platform, wherein the testing platform stores a test suite corresponding to at least one test type, including: Determine the test tasks for the target objects; Determine, according to the test task, a target test suite corresponding to the test task in the test suite corresponding to the at least one test type, wherein the target test suite includes a test configuration and a test script, and the test configuration is obtained by converting a target variable parameter of the target test suite; The test task is executed according to the test configuration through the test script to obtain the test result of the target object.

[0005] According to a second aspect of an embodiment of this specification, there is provided an object testing device, which is applied to a test platform, wherein the test platform stores a test suite corresponding to at least one test type, including: A first determination module is configured to determine a test task for a target object; A second determination module is configured to determine, according to the test task, a target test suite corresponding to the test task in the test suite corresponding to the at least one test type, wherein the target test suite includes a test configuration and a test script, and the test configuration is obtained by converting a target variable parameter of the target test suite; The execution module is configured to execute the test task according to the test configuration through the test script to obtain the test result of the target object.

[0006] According to a third aspect of an embodiment of this specification, a computing device is provided, including: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above object testing method are implemented.

[0007] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned object testing method are implemented.

[0008] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions, which implement the steps of the above-mentioned object testing method when executed by a processor.

[0009] An embodiment of the present specification realizes the integration of test suites corresponding to at least one test type through a test platform. When testing a target object, the test task of the target object is determined, and the target test suite corresponding to the test task is determined in the test suite corresponding to at least one test type according to the test task, thereby reducing the difficulty of using the test suite. And when the test task is subsequently executed, the test script included in the target test suite is executed according to the test configuration to obtain the test result of the target object. By integrating test suites of different test types into the test platform, the test process for the target object is simplified and the test suite is easy to store. After the test results are obtained in the subsequent test, the test report can also be quickly analyzed to improve the test efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram of a scenario of an object testing method provided according to an embodiment of this specification is shown; Figure 2 A flowchart of an object testing method provided according to an embodiment of this specification is shown; Figure 3A A schematic diagram of variable parameters of a test suite in an object testing method provided in an embodiment of this specification is shown; Figure 3B A schematic diagram of file conversion of a test configuration in an object testing method provided in an embodiment of this specification is shown; Figure 3CA schematic diagram of a test suite management page in an object testing method provided by an embodiment of the present invention is shown; Figure 4 A flowchart of a processing process of an object testing method provided by an embodiment of this specification is shown; Figure 5 A schematic diagram of the structure of an object testing device provided by an embodiment of this specification is shown; Figure 6 A structural block diagram of a computing device 600 provided according to an embodiment of the present specification is shown. DETAILED DESCRIPTION

[0011] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0012] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0013] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0014] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0015] First, the terms involved in one or more embodiments of this specification are explained.

[0016] Functional testing: Functional testing is to test the characteristics and operable behaviors of a product to determine whether they meet the design requirements based on product characteristics, operation descriptions and user methods. In one embodiment of this specification, functional testing is used to verify the functions of the product. According to the functional test cases, each item is tested to check whether the product meets the functions required by the user.

[0017] Performance testing: Performance testing usually verifies whether the performance of the software can still meet the performance indicators when used repeatedly under normal environment and system conditions. In one embodiment of this specification, performance testing is used to determine the stability, speed, scalability and responsiveness of the application under a specific workload.

[0018] Compatibility testing: Compatibility testing is to ensure that software or hardware products can work properly in different environments, including operating systems, browsers, device types, etc. In one embodiment of this specification, compatibility testing is used to check whether the software can run normally on different hardware platforms and software platforms.

[0019] Security testing: Security testing refers to the process of verifying the security level of an application and identifying potential security flaws. It mainly tests the security of the application layer, including the security of the application itself and the data security of the application. In one embodiment of this specification, security testing is used to verify whether the protection mechanism integrated in the software can actually protect the system from illegal intrusion.

[0020] Test suite: A test suite is a group of test users that are executed together to test specific functions or features of the software. The test suite not only provides a way to organize test cases, but also can execute multiple test users at one time, thereby improving test efficiency. In an embodiment of this specification, the test suite is a test suite obtained by standardizing an open source test suite, which mainly includes two parts: test configuration and test script.

[0021] In the current testing field, software functional testing, performance testing, compatibility testing, security testing, etc. are often performed separately in multiple testing phases, that is, different types of tests are performed independently. This decentralized testing method is prone to test errors, low test efficiency, and complex management. Subsequent test results will also be stored in a decentralized manner, making it difficult to analyze the software test results as a whole, making subsequent optimization work more difficult.

[0022] Based on this, an object testing method is provided in this specification. This specification also relates to an object testing apparatus, a computing device, a computer-readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0023] See also Figure 1 , Figure 1 A scenario schematic diagram of an object testing method provided according to an embodiment of the present specification is shown, wherein, in the case of testing a target object, since the open source test suite is written by developers according to their respective development habits, including different writing languages, code styles, etc. In order to facilitate testers to use the test suite for testing, the tester can obtain the initial test suite through the test platform, and the initial test suite is the various existing test suites obtained. The initial test suite is converted by the test platform to standardize and transform, and a test suite containing test configurations and test scripts is obtained, so that the test platform can directly use the transformed test suite. The test platform will store the standardized test suite in the test suite database under the platform. When the target object is tested later, the target test suite of the corresponding test type can be selected from the test suite database, and the target object can be tested using the target test suite, thereby providing a fast and convenient testing method for the tester. Through the test platform, a variety of different types of test suites can be integrated, which is convenient for testers to uniformly manage and maintain various types of test suites. When conducting tests, the test platform uses the target test suite to automatically execute test tasks, thereby improving test efficiency. After obtaining the test results, the test platform can also automatically analyze the test results and generate corresponding test reports for testers, making it easier for testers to analyze problems encountered during the test based on the test reports and improve subsequent optimization operations on the target objects.

[0024] See also Figure 2 , Figure 2 A flow chart of an object testing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0025] Step 202: Determine a test task for the target object.

[0026] The target object can be understood as the object to be tested, and the target object can be the developed software, product, such as application, operating system, etc. The test task can be understood as the task of testing the target object, and the test type, test suite, and the execution order of the test suite during the task execution can be specified in the test task.

[0027] In actual applications, when testing the target object, it is necessary to create corresponding test tasks, and the test of the target object can be completed by executing the test tasks. The test tasks may include but are not limited to the test types of the target object, such as functional testing, compatibility testing, and other types; the test suites used for each test type, such as the ltp (Linux Test Project) test suite under the functional test type, the iperf (network performance test) test suite under the performance test type, the browserstack (cloud-based browser test) test suite under the compatibility test type, and the snort (network intrusion detection) test suite under the security test type; at the same time, the execution information such as the running version, priority, and dependency order of the test suite can be specified; the running environment resources of each test suite, such as stand-alone operation, distributed operation, etc. By correctly creating test tasks, the target object can be tested more comprehensively.

[0028] Furthermore, in order to be able to more accurately determine the test task of the target object, it is necessary to determine it according to the test requirements. Specifically, the test task for the target object is determined, including: determining the test requirements of the target object, and selecting an initial task template in the task template library according to the test requirements, wherein the task template library includes at least one task template; obtaining task configuration information of the associated target object, updating the initial task template based on the task configuration information to obtain the target task template; and creating the test task of the target object according to the target task template.

[0029] The test requirements of the target object can be understood as the requirements that the tester needs to test the target object, such as compatibility testing of the newly developed operating system and functional testing of the newly developed software product. According to the test requirements, an initial task template can be selected from the task template library, and then the initial task template is updated by associating the task configuration information of the target object to obtain the target task template, and the test task of the target object can be created based on the target task template.

[0030] In actual applications, the test platform will be configured with a task template library, which stores preset task templates, namely initial task templates. The initial task template can be understood as a task template created in advance for test tasks of different test types. The basic configuration corresponding to the test type, such as test items, test environment, etc., can be set in the initial task template. The tester only needs to simply fill in other task configuration information to obtain the target task template, and then generate the test task through the target task template.

[0031] In specific implementation, the initial task template can also be a task template generated after the tester configures the task template and selects the template to store. For example, when performing functional testing on the target object for the first time, the tester can choose to perform relevant configurations on the initial task template under the functional test. After the configuration is completed, you can choose to save the current template. When performing the same type of functional testing in the future, you can directly call the template, thereby saving template configuration time and improving testing efficiency.

[0032] Correspondingly, when the initial task template is a new task template, the tester can determine the task configuration information, and through the task configuration information, the environment information, test suite information and more task execution information of this test task can be determined. The environment information can include the configuration information of the test machine, such as kernel reinstallation, machine restart, global variables and other parameter settings; the test suite information can include the selection of the test suite, the test case configuration of the test suite and other settings; the configuration of the task execution information can include the notification setting after the task is executed, the task priority setting, the task timeout setting, etc. The task configuration information can be determined by the tester on the editing page of the initial task template provided by the test platform. The tester can also choose to generate the task configuration information by writing code, such as selecting the yaml (yaml is a highly readable format used to express data serialization) mode to create the task. After obtaining the task configuration information submitted by the tester, the initial task template can be updated based on the task configuration information to obtain the target task template. After choosing to submit the test or confirming the template based on the target task template, the corresponding test task can be created. After obtaining the target task template, you can also choose the option to store the template, so that the target task template can be stored in the task template library and used as the initial task template in subsequent other test tasks.

[0033] In a specific embodiment of this specification, the target object is an operating system, and the test requirement is to test the network performance of the operating system. An initial task template is selected in the task template library according to the test requirement. The initial task template can be a template in a performance test. The tester fills in the task configuration information according to the content of the initial task template, including the task name, test suite, notification information, etc., and updates the initial task template based on the task configuration information to generate a target task template. The tester can first save the target task template so that when performing the same test later, the target task template can be used directly, and then the test task can be submitted based on the target task template, thereby creating a test task for the target object.

[0034] Based on this, the test platform stores initial task templates through the task template library, so that testers can directly use the templates to create test tasks during the test process, reducing the complexity of the test process and improving test efficiency.

[0035] Furthermore, in order to avoid testers from repeatedly submitting corresponding configuration settings in the same type of test tasks, the target task template of the test task can be selected for storage. Specifically, after obtaining the target task template, the method also includes: determining the storage information of the target task template in response to a storage instruction for the target task template; and storing the target task template in the task template library according to the storage information.

[0036] Among them, the storage instruction of the target task template is used to store the target task template. After generating the target task template, the tester can issue a storage instruction through the template storage function provided by the test platform. At this time, the test platform will determine the storage information of the target task template. The storage information may include a template name used to indicate the template after storage, a template description used to indicate the purpose of the template, etc. By associating the storage information with the target task template, it is convenient for subsequent testers to select the corresponding template from the task template library for use. For example, if the storage information includes that the task template is used to test network performance, the tester can directly select the task template from the task template library based on the relevant description and apply it to the network performance test.

[0037] In a specific embodiment of this specification, citing the above example, after the user creates a target task template for testing the network performance of the operating system, the target task template is stored in the task template library of the test platform through the template saving function provided by the test platform, as the initial task template for subsequent test tasks. In the case of subsequent network performance tests of the same type, the task template saved in the task template library can be directly selected, thereby reducing the configuration operations of the tester on the task template and improving the efficiency of subsequent tests.

[0038] Step 204: Determine a target test suite corresponding to the test task in a test suite corresponding to at least one test type according to the test task, wherein the target test suite includes a test configuration and a test script, and the test configuration is obtained by converting target variable parameters of the target test suite.

[0039] Among them, since the test platform will store test suites corresponding to multiple different test types in advance, after determining the test task, the target test suite to be executed this time can be selected based on the test task. The target test suite is a test suite selected from the test suites corresponding to at least one test type and used to execute this test task.

[0040] In actual applications, since multiple different test suites may be used in a test task, the number of target test suites determined can also be multiple, and each target test suite includes a corresponding test configuration and test script. Among them, the test configuration can be understood as the configuration file corresponding to the test suite, and the test configuration is mainly used to describe the parameters of the test. The test configuration is the configuration file obtained after the target variable parameters of the test suite are standardized, see Figure 3A , Figure 3A The diagram shows a variable parameter diagram of a test suite in an object testing method provided in an embodiment of the present specification, wherein, taking the initial test suite as the test suite netperf as an example, the test suite may include a set test conf (config, configuration) name, that is, Figure 3A The test items in the , and the running variable items specified by each test conf, such as IP, runtime (running time), send_size (amount of data sent), etc., the parameter values ​​of the corresponding running variables in each test conf are different. These running variables will be passed to the test script in the form of parameters, so that the test script will perform tests according to the variable parameters of each test conf. In specific implementation, the test platform will standardize the initial test suite for unified management and maintenance. Therefore, the variable parameters of each test conf in the initial test suite need to be converted to standardization, see Figure 3B , Figure 3B The following is a schematic diagram of file conversion of a test configuration in an object testing method provided by an embodiment of the present specification, wherein the target variable parameters in the test suite are converted, for example, the target variable parameter is "TCP_RR ipv4 60 none", which is converted to "netperf:tese=TCP_RR, IP=ipv4, runtime=60s", and the corresponding test configuration is generated by performing standard conversion on the target variable parameters in the configuration file in the test suite. When the test suite is running, it will integrate the parameters in the configuration file and pass them to the test script for execution.

[0041] A test script can be understood as a script used to execute a test suite. A test script can execute test suite acquisition, test suite compilation and installation, test execution, result processing, test suite uninstallation, and other stages. Each test script includes three script files: install.sh, run.sh, and redame.md. The install.sh script is used to define the test suite acquisition, dependency packages, test suite compilation, and installation process; the run.sh script is used to define the test execution process and result processing method; and the redame.md file is used for necessary descriptions of the test suite.

[0042] In specific implementation, the test platform needs to standardize the conversion of different types of test suites before the test suites are connected, and support testers to convert manually according to the standards, or use the automated transformation function provided by the test platform to convert. In this way, the test platform can effectively integrate various types of test suites and uniformly manage their execution process by establishing a unified integration standard.

[0043] Furthermore, determining a target test suite corresponding to the test task in a test suite corresponding to at least one test type according to the test task includes: determining the task type of the test task, selecting a test suite set corresponding to the task type from the test suites corresponding to the at least one test type, wherein the test suite set includes at least one test suite; determining a target test suite corresponding to the test task in the test suite set according to the suite attribute information carried in the test task.

[0044] Among them, the task type can be understood as the type of test task. For example, if the test task is a functional test task, the corresponding task type is the functional type; if the test task is a compatibility test task, the corresponding task type is the compatibility type. The test suite set can be understood as the set corresponding to each task type. The test suite set contains all test suites corresponding to the task type. For example, the test suite set corresponding to the functional test type contains all test suites of the functional test type, such as ltp, xfstests, etc. The test suite corresponding to at least one test type can be stored in the test suite database. The test suite database stores test suites corresponding to all test types. For the convenience of screening, the test suite set corresponding to the task type can be screened out from the test suite database according to the task type of the test task. Subsequently, the target test suite can be further selected from the test suite set according to the suite attribute information carried in the test task.

[0045] In actual applications, in addition to filtering out the corresponding test suite set according to the task type, you can also filter according to the field type, such as the test suite for testing the CPU and the test suite for testing the file system. You can also filter the test suites in the test suite database by different fields, and then further filter based on the suite attribute information carried in the test task. Suite attribute information can be understood as the attribute information corresponding to the test suite, which includes the test suite name, test configuration name, test suite version number, etc. Based on the suite attribute information, you can select the target test suite corresponding to the test task from the test suite set.

[0046] In a specific embodiment of the present specification, the task type of the test task is indeed a performance test type, a test suite set corresponding to the performance test type is selected in the test suite database, and based on the suite attribute information carried in the test task, the target test suite corresponding to the test task is selected in the test suite set as the iperf test suite.

[0047] Based on this, the efficiency of selecting the target test suite is improved by screening in the test suite database using task types and further screening in the test suite collection according to suite attribute information.

[0048] Step 206: Execute the test task according to the test configuration through the test script to obtain the test result of the target object.

[0049] After the test suite of the test task is determined, the test script in the test suite can be used to execute the test task according to the test configuration, thereby obtaining the test result of the target object.

[0050] In actual applications, since the test script needs to receive the running variables in the test configuration when executing, the test script needs to execute the test task according to the test configuration. Furthermore, the test script executes the test task according to the test configuration to obtain the test result of the target object, including: obtaining the target running field according to the test configuration, and updating the test script based on the target running field to obtain the target test script; executing the test task through the target test script to obtain the test result of the target object.

[0051] Among them, the target run field can be understood as the run variable field defined in each test conf in the test configuration. For example, in the test suite netperf, the target run field obtained according to the test configuration is "netperf:tese=TCP_RR, IP=ipv4, runtime=60s...netperf:tese=TCP_CRR, IP=ipv4, runtime=60s". After obtaining the target run field, the test script can be updated based on the target run field. Updating the test script based on the target run field can be understood as integrating the parameters in the test configuration file and passing them to the test script to obtain the target test script. So that the subsequent target test script can be executed with the run variable parameters defined in the test configuration.

[0052] In actual applications, the run.sh script file in the target test script is mainly used to start, run and uninstall the test script. During its running process, when it runs to the run() method, it will receive various target running fields passed by the test configuration, thereby updating the test script based on the target running fields.

[0053] Based on this, the test platform standardizes the received open source test suites to obtain the test configurations and test scripts corresponding to the test suites. When executing test tasks, the test tasks are completed through the coordination of test configurations and test scripts, which facilitates the test platform to manage the execution of test tasks.

[0054] Furthermore, the test task is executed through a target test script to obtain a test result of the target object, including: determining execution attribute information of the target test script according to the test task; and executing the test task according to the execution attribute information through the target test script to obtain a test result of the target object.

[0055] Among them, the execution attribute information of the target test script can be understood as the attribute information involved in the execution process of the target test script. The execution attribute information may include attributes such as execution priority, serial and parallel execution mode, and operation architecture. Through the target test script, the test task is executed according to the execution attribute information, so that the target test script can be executed according to the execution attribute information set by the tester.

[0056] In actual applications, the execution attribute information can be filled in by the tester when creating the test task, or it can be obtained by the test task when determining the target test suite. After determining the execution attribute information of the target test script, the target test script can execute the test task according to the execution attribute information. By executing the test task according to the execution attribute information, the test task is correctly executed, ensuring the accuracy of the test results.

[0057] In a specific embodiment of the present specification, execution attribute information of the target test script is determined according to the test task. The execution attribute information includes determining the priority, dependency, serial and parallel execution mode, etc. of the target test script in the target test suite. The target test script executes the test task according to the execution attribute information, thereby obtaining the test result of the target object.

[0058] Furthermore, the test task is executed according to the execution attribute information through the target test script to obtain the test result of the target object, including: determining the test environment resources corresponding to the target test script according to the execution attribute information; and executing the test task through the target test script in the test environment resources.

[0059] The test environment resources may be understood as the execution environment resources of the target test script. The test task is executed by executing the target test script in the test environment resources.

[0060] In actual applications, the test platform will allocate test machines to the test suites, and the test machines are the execution environment resources. The test tasks are executed by running the target test scripts on the test machines. In specific implementation, when allocating test environment resources for a single-machine test suite, a test machine will be allocated to it; when allocating test environment resources for a distributed test suite, a cluster containing multiple test machines will be allocated to it, so that the test suite can be executed normally.

[0061] In a specific embodiment of the present specification, the operating environment requirements of the test suite are determined based on the execution attribute information, and the operating environment requirements of the iperf test suite are determined to be stand-alone execution, thereby determining that the test environment resources corresponding to the iperf test suite are a stand-alone test environment. The test platform allocates a test machine to the test suite, and in the test machine, runs the target test script corresponding to the test suite, thereby executing the network performance test task.

[0062] Based on this, by configuring the corresponding test environment for the test suite, it is ensured that the target test script corresponding to the test suite can be executed in the correct test environment, ensuring that the test task can be executed smoothly.

[0063] Furthermore, after obtaining the test results of the target object, it also includes: analyzing the test results according to the time dimension to generate a timing analysis report corresponding to the test results; or analyzing the test results according to the environment dimension to generate a comparative analysis report corresponding to the test results.

[0064] Among them, the timing analysis report can be understood as a test report obtained by analyzing the test results from the time dimension; the comparative analysis report can be understood as a test report obtained by analyzing the test results from the environment dimension.

[0065] In practical applications, timing analysis is to draw a curve of the test results from the time dimension. The curve chart can intuitively analyze the mutation of the test results to quickly and accurately identify problems. The timing analysis report can include a timing analysis chart. Usually, the data in the same project is filtered by the task label, and then the test results are displayed on the timeline to quickly obtain a timing analysis chart. If you need to check whether a certain indicator of the performance test is stable, you can check the performance indicator trend chart to help determine its stability; or if you need to check whether a certain use case of the functional test is stable, you can check the Test conf pass rate to help determine its stability; or a use case currently fails, but it may be a known problem. You can check whether the historical data on the functional result trend chart are all failures to help determine the severity of the result.

[0066] Accordingly, comparative analysis is to compare and analyze the test results of the same test suite in different test environments, or to compare and analyze the test results of different test suites in the same test environment. In comparative analysis, it is necessary to select the benchmark test results to compare with other test results.

[0067] In a specific embodiment of the present specification, when performing a network performance test on an operating system, you can choose to use the same test suite to perform performance testing on it in multiple time periods. After the test is completed, you can obtain multiple test results for different time periods. You can generate a timing analysis graph and a timing analysis report based on the multiple test results, so that the tester can confirm the changes and impacts of the network performance test in different time periods through the timing analysis report.

[0068] In another specific embodiment of the present specification, when a load performance test is performed on an operating system running at full load, the test results under the operating system can be compared and analyzed with the test results under other operating systems to generate a comparative analysis report, thereby determining development problems of the current operating system and facilitating subsequent testers to optimize and repair development problems.

[0069] Based on this, through automated analysis of test results in different dimensions, testers can quickly analyze problems encountered during testing based on automatically generated test reports, making it easier to troubleshoot and fix problems later.

[0070] Furthermore, the method also includes: determining an initial test suite in response to a test suite integration instruction; determining variable parameters of the initial test suite, and generating an initial test configuration corresponding to the initial test suite according to the variable parameters; determining an initial installation script and an initial run script of the initial test suite, and generating an initial test script of the initial test suite based on the initial installation script and the initial run script, wherein the initial installation script is used to install the initial test suite, and the initial run script is used to run the initial test suite; storing the initial test suite according to the initial test configuration and the initial test script.

[0071] Among them, when the test platform is connected to an open source test suite, it needs to be standardized. The initial test suite can be understood as a test suite that has not been standardized, such as an open source test suite. The variable parameters of the initial test suite can be understood as the running variable parameters defined in the test suite. The initial test configuration of the initial test suite can be generated according to the variable parameters. The initial test configuration can be understood as the test configuration conf file generated after the test platform automatically converts or the tester manually converts according to the standard.

[0072] The initial installation script and initial run script of the initial test suite are the installation script and test script set for the test suite. The installation script is used to install the test suite, including test suite acquisition, dependency packages, test suite compilation and installation; the test script is used to define the test execution process and result processing method.

[0073] In actual application, after obtaining the initial installation script and the initial operation script, the initial test script can be generated according to the initial installation script and the initial operation script. In order to facilitate the use of the test script by subsequent testers, a description file can also be added to the initial test script. After obtaining the initial test configuration and the initial test script, the initial test suite can be stored based on the initial test configuration and the initial test script.

[0074] During specific implementation, the initial test suite is stored, that is, the initial test configuration and initial test script corresponding to the initial test suite are stored. The initial test configuration and initial test script can be stored according to the corresponding test suite name, and can be placed in a code hosting repository such as github, gitee, etc., so that the test platform can display the interface on the suite management page based on the code hosting repository. The stored test suites can also be maintained through the test suite management function of the test platform, including version control and change modification management. When the test suite function is updated and released, the test suite version can be upgraded, and the specified version can be selected for execution when executing the test suite.

[0075] By converting the initial test suite into a standardized one, the converted test suite can be integrated into the test platform, simplifying the subsequent test process for the target object and improving the test efficiency.

[0076] Further, generating an initial test configuration corresponding to the initial test suite according to the variable parameters includes: converting the variable parameters to obtain the running fields corresponding to the variable parameters; and generating the initial test configuration corresponding to the initial test suite based on the running fields.

[0077] The conversion of the variable parameters is to convert the variable parameters into standardized running fields, and then the initial test configuration corresponding to the test suite can be generated based on the running fields.

[0078] In a specific embodiment of the present specification, the variable parameter is determined to be "TCP_RRipv460 none", and the run field after conversion is "netperf:tese=TCP_RR, IP=ipv4, runtime=60s". The initial test configuration corresponding to the initial test suite is generated according to the run field corresponding to each configuration conf.

[0079] Based on this, by converting variable parameters into standardized running fields, an initial test configuration corresponding to the test suite can be generated based on the running fields, thereby facilitating testing using the test suite according to the initial test configuration.

[0080] Furthermore, an initial test script of the initial test suite is generated based on the initial installation script and the initial running script, including: updating the initial installation script according to the installation dependency data of the initial test suite to obtain a target installation script; updating the initial running script according to the running code data of the initial test suite to obtain a target running script; and combining the target installation script and the target running script to generate the initial test script of the initial test suite.

[0081] In actual applications, the initial installation script needs to be updated according to the installation dependency data of the initial test suite. The installation dependency data is the dependency data required when installing the initial test suite, including dependencies, installation environment and other data. After obtaining the target installation script, the initial test suite can be installed, compiled and other operations can be performed based on the target installation script. The initial running script needs to be updated according to the running code data of the initial test suite. The running code data is the running code included in the initial test suite. After obtaining the target running script, the initial test suite can be executed based on the target running script. After combining the target installation script and the target running script, the initial test script of the initial test suite can be obtained.

[0082] Based on this, by generating an initial test script of the initial test suite, it is convenient to execute the corresponding test tasks by running the initial test script later, thereby improving the task execution efficiency.

[0083] Furthermore, in order to enable testers to quickly understand the information of the test suite stored in the test platform, the directory of the test suite can be maintained through the test suite management page, and the initial test suite can be specifically stored, including: determining the management information of the initial test suite; and storing the initial test configuration and the initial test script according to the management information.

[0084] Among them, the management information of the initial test suite can be understood as the attribute information of the initial test suite. The management information may include the operating mode, field, priority, serial and parallel, version number and other information of the test suite. Through the management information, the initial test configuration and initial test script corresponding to the initial test suite can be stored in the corresponding management database for easy display.

[0085] See also Figure 3C , Figure 3CA schematic diagram of a test suite management page in an object testing method provided by an embodiment of the present specification is shown, which includes management directories of multiple test suites, and the management directory stores management information corresponding to each test suite, such as the running mode of the ltp test suite is a distributed cluster, the field is functional testing, the priority is 10, and the serial and parallel behaviors are parallel. Correspondingly, it can also include the version number, change date, change description, etc. of the test suite.

[0086] Based on this, by storing the initial test configuration and the initial test script corresponding to the initial test suite in the corresponding management database, it is convenient to maintain and manage the initial test suite.

[0087] An object testing method provided in this specification is applied to a test platform, and the test platform stores a test suite corresponding to at least one test type, including determining a test task for a target object; determining a target test suite corresponding to the test task in the test suite corresponding to the at least one test type according to the test task, wherein the target test suite includes a test configuration and a test script, and the test configuration is obtained by converting the target variable parameters of the target test suite; and executing the test task according to the test script according to the test configuration to obtain the test result of the target object. The test suite is integrated through the test platform, and when testing the target object, the test task of the target object is determined, and the target test suite corresponding to the test task is determined in the test suite corresponding to at least one test type according to the test task, thereby reducing the difficulty of using the test suite. And when executing the test task later, the test script included in the target test suite is executed according to the test configuration to obtain the test result of the target object. By integrating different types of test suites into the test platform, the test process for the target object is simplified, and the test suite is convenient for storing. After the test results are obtained in the subsequent test, the test report can also be quickly analyzed to improve the test efficiency.

[0088] The following combination Figure 4 , taking the application of the object testing method provided in this specification in functional testing as an example, the object testing method is further described. Figure 4 A processing flow chart of an object testing method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0089] Step 402: Determine the test requirements of the target object, and select an initial task template from a task template library according to the test requirements, wherein the task template library includes at least one task template.

[0090] In one practicable manner, it is determined that the target object is a developed software product, and the functions of the software product need to be tested. An initial task template is selected from a task template library according to the test requirements.

[0091] In one achievable manner, in response to a test suite integration instruction, an initial test suite is determined; variable parameters of the initial test suite are determined, the variable parameters are converted, and operation fields corresponding to the variable parameters are obtained; and an initial test configuration corresponding to the initial test suite is generated based on the operation fields.

[0092] Determine the initial installation script and the initial running script of the initial test suite, update the initial installation script according to the installation dependency data of the initial test suite, and obtain the target installation script. Update the initial running script according to the running code data of the initial test suite, and obtain the target running script. Combine the target installation script and the target running script to generate the initial test script of the initial test suite. Store the initial test suite according to the initial test configuration and the initial test script.

[0093] Step 404: Acquire task configuration information of the associated target object, and update the initial task template based on the task configuration information to obtain a target task template.

[0094] In one achievable manner, task configuration information set by a tester for a software product is obtained, and an initial task template is updated according to the task configuration information to obtain a target task template.

[0095] Step 406: Create a test task for the target object according to the target task template.

[0096] In one achievable manner, a test task of a target object is created by calling a target task template. Accordingly, in the process of calling the target task template, relevant information in the target task template may also be adjusted to generate a test task that meets the test requirements.

[0097] Step 408: Determine the task type of the test task, and select a test suite set corresponding to the task type in the test suite database.

[0098] In one achievable manner, it is determined that the task type of the test task is a functional test, and a test suite set corresponding to the functional test is selected from a test suite database.

[0099] Step 410: Determine the target test suite corresponding to the test task in the test suite set according to the suite attribute information carried in the test task.

[0100] In one achievable manner, a target test suite corresponding to the test task is selected from the test suite set according to the suite name carried in the test task.

[0101] Step 412: Obtain the target running field according to the test configuration, and update the test script based on the target running field to obtain the target test script.

[0102] In one achievable manner, a target running field is obtained according to the test configuration, and a test script is updated based on the target running field to obtain a target test script.

[0103] Step 414: Execute the test task through the target test script to obtain the test result of the target object.

[0104] In one achievable manner, the test task is executed through the target test script to obtain the test result of the target object.

[0105] In one achievable manner, the execution attribute information of the target test script is determined according to the test task, and the test task is executed according to the execution attribute information through the target test script to obtain the test result of the target object.

[0106] In one achievable manner, the test environment resources corresponding to the target test script are determined according to the execution attribute information, and the test task is executed in the test environment resources through the target test script.

[0107] This specification provides an object testing method that integrates test suites through a test platform. When testing a target object, the test task of the target object is determined, and the target test suite corresponding to the test task is determined in at least one test suite based on the test task, thereby reducing the difficulty of using the test suite. And when the test task is subsequently executed, the test script included in the target test suite is executed according to the test configuration to obtain the test result of the target object. By integrating test suites of different test types into the test platform, the test configuration and test script of the test suite are managed in an integrated and standardized manner, which facilitates the storage of the test suite, and tests are performed according to the test configuration and test script, simplifying the test process of the target object and improving the test efficiency.

[0108] Corresponding to the above method embodiment, this specification also provides an object testing device embodiment, Figure 5 FIG. 1 is a schematic diagram showing the structure of an object testing device provided by an embodiment of the present specification. Figure 5 As shown, the device comprises: A first determination module 502 is configured to determine a test task for a target object; A second determination module 504 is configured to determine, according to the test task, a target test suite corresponding to the test task in the test suite corresponding to the at least one test type, wherein the target test suite includes a test configuration and a test script, and the test configuration is obtained by converting a target variable parameter of the target test suite; The execution module 506 is configured to execute the test task according to the test configuration through the test script to obtain the test result of the target object.

[0109] Optionally, the first determination module 502 is further configured to determine the test requirements of the target object, and select an initial task template from a task template library based on the test requirements, wherein the task template library includes at least one task template; obtain task configuration information of the associated target object, update the initial task template based on the task configuration information, and obtain a target task template; and create a test task for the target object based on the target task template.

[0110] Optionally, the first determination module 502 is further configured to determine storage information of the target task template in response to a storage instruction for the target task template; and store the target task template in the task template library according to the storage information.

[0111] Optionally, the second determination module 504 is further configured to determine the task type of the test task, select a test suite set corresponding to the task type from the test suites corresponding to the at least one test type, wherein the test suite set includes at least one test suite; and determine the target test suite corresponding to the test task in the test suite set according to the suite attribute information carried in the test task.

[0112] Optionally, the execution module 506 is further configured to obtain a target running field according to the test configuration, and update the test script based on the target running field to obtain a target test script; execute the test task through the target test script to obtain a test result of the target object.

[0113] Optionally, the execution module 506 is further configured to determine execution attribute information of the target test script according to the test task; execute the test task according to the execution attribute information through the target test script to obtain the test result of the target object.

[0114] Optionally, the execution module 506 is further configured to determine the test environment resources corresponding to the target test script according to the execution attribute information; and execute the test task in the test environment resources through the target test script.

[0115] Optionally, the device also includes an analysis module, which is configured to analyze the test results according to a time dimension and generate a timing analysis report corresponding to the test results; or to analyze the test results according to an environment dimension and generate a comparative analysis report corresponding to the test results.

[0116] Optionally, the device also includes a conversion module, which is configured to determine an initial test suite in response to a test suite integration instruction; determine variable parameters of the initial test suite, and generate an initial test configuration corresponding to the initial test suite based on the variable parameters; determine an initial installation script and an initial run script of the initial test suite, and generate an initial test script of the initial test suite based on the initial installation script and the initial run script, wherein the initial installation script is used to install the initial test suite, and the initial run script is used to run the initial test suite; and store the initial test suite according to the initial test configuration and the initial test script.

[0117] Optionally, the conversion module is further configured to convert the variable parameters to obtain the running fields corresponding to the variable parameters; and generate an initial test configuration corresponding to the initial test suite based on the running fields.

[0118] Optionally, the conversion module is further configured to update the initial installation script according to the installation dependency data of the initial test suite to obtain a target installation script; update the initial running script according to the running code data of the initial test suite to obtain a target running script; and combine the target installation script and the target running script to generate an initial test script of the initial test suite.

[0119] Optionally, the conversion module is further configured to determine management information of the initial test suite; and store the initial test configuration and the initial test script according to the management information.

[0120] The object testing device provided in this specification is applied to a test platform, and the test platform stores a test suite corresponding to at least one test type, including: a first determination module, configured to determine a test task for a target object; a second determination module, configured to determine a target test suite corresponding to the test task in the test suite corresponding to the at least one test type according to the test task, wherein the target test suite includes a test configuration and a test script, and the test configuration is obtained by converting the target variable parameters of the target test suite; an execution module, configured to execute the test task according to the test configuration through the test script to obtain the test result of the target object. The test suite is integrated through the test platform, and when testing the target object, the test task of the target object is determined, and the target test suite corresponding to the test task is determined in at least one test suite according to the test task, thereby reducing the difficulty of using the test suite. And when executing the test task later, the test script included in the target test suite is executed according to the test configuration to obtain the test result of the target object. By integrating different types of test suites into the test platform, the test process for the target object is simplified, and the test suite is convenient for storing. After the test results are obtained in the subsequent test, the test report can also be quickly analyzed to improve the test efficiency.

[0121] The above is a schematic scheme of an object testing device of this embodiment. It should be noted that the technical scheme of the object testing device and the technical scheme of the object testing method described above are of the same concept, and the details not described in detail in the technical scheme of the object testing device can be found in the description of the technical scheme of the object testing method described above.

[0122] Figure 6 The block diagram of a computing device 600 according to an embodiment of the present specification is shown. The components of the computing device 600 include but are not limited to a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and the database 650 is used to store data.

[0123] The computing device 600 also includes an access device 640 that enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 640 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0124] In one embodiment of the present specification, the above components of the computing device 600 and Figure 6 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 6 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0125] The computing device 600 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 600 may also be a mobile or stationary server.

[0126] The processor 620 is used to execute the following computer executable instructions, which implement the steps of the above-mentioned object testing method when executed by the processor.

[0127] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the object testing method described above are of the same concept, and the details not described in detail in the technical scheme of the computing device can be found in the description of the technical scheme of the object testing method described above.

[0128] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which can implement the steps of the above-mentioned object testing method when executed by a processor.

[0129] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the object testing method described above are of the same concept, and the details not described in detail in the technical scheme of the storage medium can be found in the description of the technical scheme of the object testing method described above.

[0130] An embodiment of the present specification also provides a computer program product, including a computer program or instructions, which implement the steps of the above-mentioned object testing method when executed by a processor.

[0131] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the object testing method described above are of the same concept, and the details not described in detail in the technical scheme of the computer program product can be found in the description of the technical scheme of the object testing method described above.

[0132] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0133] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0134] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0135] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0136] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can understand and use this specification well.

Claims

1. An object testing method, applied to a test platform, wherein the test platform stores a test suite corresponding to at least one test type, comprising: Determine the test tasks for the target objects; Determine, according to the test task, a target test suite corresponding to the test task in the test suite corresponding to the at least one test type, wherein the target test suite includes a test configuration and a test script, and the test configuration is obtained by converting a target variable parameter of the target test suite; The test task is executed according to the test configuration through the test script to obtain the test result of the target object.

2. The method according to claim 1, determining a test task for a target object comprises: Determine the test requirements of the target object, and select an initial task template from a task template library according to the test requirements, wherein the task template library includes at least one task template; Acquire task configuration information of the associated target object, and update the initial task template based on the task configuration information to obtain a target task template; A test task for the target object is created according to the target task template.

3. According to the method of claim 2, after obtaining the target task template, the method further comprises: In response to a storage instruction for the target task template, determining storage information of the target task template; The target task template is stored in the task template library according to the storage information.

4. The method according to claim 1, determining, according to the test task, a target test suite corresponding to the test task in the test suite corresponding to the at least one test type, comprising: Determine a task type of the test task, and select a test suite set corresponding to the task type from the test suites corresponding to the at least one test type, wherein the test suite set includes at least one test suite; According to the suite attribute information carried in the test task, a target test suite corresponding to the test task is determined in the test suite set.

5. The method according to claim 1, executing the test task according to the test configuration through the test script to obtain the test result of the target object, comprises: Acquire a target running field according to the test configuration, and update the test script based on the target running field to obtain a target test script; The test task is executed through the target test script to obtain the test result of the target object.

6. The method according to claim 5, executing the test task through a target test script to obtain the test result of the target object, comprises: Determine execution attribute information of a target test script according to the test task; The test task is executed according to the execution attribute information through the target test script to obtain the test result of the target object.

7. The method according to claim 6, executing the test task according to the execution attribute information through the target test script to obtain the test result of the target object, comprising: Determine the test environment resources corresponding to the target test script according to the execution attribute information; In the test environment resources, the test task is executed through a target test script.

8. The method according to claim 1, after obtaining the test result of the target object, further comprises: Analyze the test results according to the time dimension and generate a timing analysis report corresponding to the test results; or, The test results are analyzed according to the environmental dimension, and a comparative analysis report corresponding to the test results is generated.

9. The method according to any one of claims 1 to 8, further comprising: In response to the test suite integration instruction, determining an initial test suite; Determining variable parameters of the initial test suite, and generating an initial test configuration corresponding to the initial test suite according to the variable parameters; Determine an initial installation script and an initial running script of the initial test suite, and generate an initial test script of the initial test suite based on the initial installation script and the initial running script, wherein the initial installation script is used to install the initial test suite, and the initial running script is used to run the initial test suite; The initial test suite is stored according to the initial test configuration and the initial test script.

10. The method according to claim 9, generating an initial test configuration corresponding to the initial test suite according to the variable parameters, comprising: Convert the variable parameters to obtain the operation fields corresponding to the variable parameters; An initial test configuration corresponding to the initial test suite is generated based on the run field.

11. The method according to claim 9, generating an initial test script of the initial test suite based on the initial installation script and the initial running script, comprising: The initial installation script is updated according to the installation dependency data of the initial test suite to obtain a target installation script; The initial running script is updated according to the running code data of the initial test suite to obtain a target running script; The target installation script and the target running script are combined to generate an initial test script of the initial test suite.

12. The method according to claim 9, storing the initial test suite comprises: determining management information for the initial test suite; The initial test configuration and the initial test script are stored according to the management information.

13. An object testing device, applied to a test platform, wherein the test platform stores a test suite corresponding to at least one test type, comprising: A first determination module is configured to determine a test task for a target object; A second determination module is configured to determine, according to the test task, a target test suite corresponding to the test task in the test suite corresponding to the at least one test type, wherein the target test suite includes a test configuration and a test script, and the test configuration is obtained by converting a target variable parameter of the target test suite; The execution module is configured to execute the test task according to the test configuration through the test script to obtain the test result of the target object.

14. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the object testing method according to any one of claims 1 to 12 are implemented.

15. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the object testing method according to any one of claims 1 to 12.

16. A computer program product, comprising a computer program or instructions, which implement the steps of the object testing method according to any one of claims 1 to 12 when executed by a processor.

Images