Software testing methods and computing equipment

Abstract

The present application provides a software testing method and computing device. The method includes: obtaining multiple software packages to be tested and a common test script respectively; generating a test case and test configuration information for each software package according to the common test script, wherein the test configuration information is used to indicate the corresponding relationship between the software package and the test case; and calling the test case of each software package in sequence according to the test configuration information of each software package through a batch test control module to perform fuzzy testing on each software package. According to this solution, the efficiency of batch software package testing can be improved.

Description

Technical Field

[0001] The present application relates to the field of server technology, and in particular to a software testing method and computing device. Background Art

[0002] With the development of information technology, computer software has gradually penetrated into all aspects of social life. With the explosive growth of computer software, the software quality issues that follow have also greatly affected our lives. In order to ensure the stability and security of the software during operation, the software needs to be tested before it is released.

[0003] At present, software testing usually requires building a test project corresponding to the software, and then integrating fuzz testing technology based on the test project to test the software. In this way, each software needs to build a test project to complete the test of each software. In the case of a large number of software (for example, there are a large number of software in the operating system), building a test project takes a lot of time.

[0004] Therefore, the efficiency of software testing in the related art is low. Summary of the invention

[0005] The embodiments of the present application provide a software testing method and computing device, which can improve the efficiency of batch software package testing.

[0006] In a first aspect, an embodiment of the present application provides a software testing method, the method comprising:

[0007] Get multiple software packages to be tested and public test scripts respectively;

[0008] Generating a test case and test configuration information for each of the software packages according to the common test script, wherein the test configuration information is used to indicate a corresponding relationship between the software packages and the test cases;

[0009] The batch test control module sequentially calls the test cases of each software package according to the test configuration information of each software package to perform fuzzy testing on each software package.

[0010] According to the solution, test cases of software packages are generated in batches through public test scripts, thereby improving the efficiency of building test projects for a large number of software packages. At the same time, by integrating a batch test control module, test cases of software packages can be called in sequence, thereby performing fuzz testing on the software packages. In this way, the automation of fuzz testing of software packages is achieved. According to this solution, the efficiency of batch software package testing can be improved.

[0011] In a possible implementation, before the batch test control module sequentially calls the test cases of each software package according to the test configuration information of each software package to perform fuzzy testing on each software package, the method further includes:

[0012] The test cases and test configuration information of each software package are stored in the installation directory where the batch test control module is located.

[0013] In a possible implementation, the batch test control module is a mugen tool.

[0014] In a possible implementation, the identification information of the software package includes a target identification, and generating the test case and test configuration information of each software package according to the common test script includes:

[0015] According to the target identifier, a target test script is obtained, wherein the target test script is used to indicate a target execution function that needs to be executed by the test software package;

[0016] Generating the test cases and test configuration information of the software package according to the common test script includes:

[0017] According to the common test script and the target test script, test cases and test configuration information of the software package are generated.

[0018] In this way, the software packages that need to perform special operations during the test process can be identified through target identification, so as to match the target test script, realize the automated construction of test cases, and thus improve the efficiency of batch software package testing.

[0019] In a possible implementation, generating the test cases and test configuration information of the software package according to the common test script and the target test script includes:

[0020] Integrate the common test script and the target test script to obtain an integrated test script;

[0021] The integrated test script is compiled to obtain the test cases and test configuration information of the software package.

[0022] In this way, the test cases can be automatically constructed, thereby improving the efficiency of batch software package testing.

[0023] In a possible implementation, the method further includes:

[0024] Obtain the test log corresponding to the software package;

[0025] Parsing the test log to determine a keyword in the test log that indicates a failure;

[0026] According to the keywords used to indicate the test failure in the test log, the failure reasons causing the failure of the test case execution are matched from a pre-configured fault information library.

[0027] In this way, the test results of the batch tested software packages can be automatically analyzed, thereby improving the efficiency of solving software package failures.

[0028] In a possible implementation, the fault information database further includes solutions corresponding to different failure causes, and the method further includes:

[0029] According to the failure cause that causes the test case to fail to execute, a solution corresponding to the failure cause is matched from the fault information library.

[0030] In this way, a solution to the software package failure can be obtained quickly, thereby improving the efficiency of solving the software package failure.

[0031] In a possible implementation, before parsing the test log and determining a keyword in the test log that indicates a failure, the method further includes:

[0032] Based on the test log, determine whether the test case caused the test failure;

[0033] The parsing the test log to determine a keyword in the test log used to indicate a failure includes:

[0034] In the case where the test failure is not caused by the test case, the test log is parsed to determine a keyword in the test log that indicates the failure.

[0035] In a possible implementation, the common test script includes a test environment preparation function, a test software execution function, and a test environment recovery function;

[0036] in,

[0037] The test environment preparation function refers to the operations that need to be performed on the test environment before testing the software;

[0038] The test software execution function refers to the operations that need to be performed during the test software process;

[0039] The test environment recovery function refers to the operations required to restore the test environment after the software test is completed.

[0040] In a second aspect, an embodiment of the present application provides a software testing device, the device comprising:

[0041] A first acquisition module is used to respectively acquire a plurality of software packages to be tested and a common test script;

[0042] A generating module, used to generate a test case and test configuration information for each of the software packages according to the common test script, wherein the test configuration information is used to indicate a corresponding relationship between the software package and the test case;

[0043] The calling module is used to call the test cases of each software package in sequence according to the test configuration information of each software package through the batch test control module to perform fuzzy testing on each software package.

[0044] According to the solution, test cases of software packages are generated in batches through public test scripts, thereby improving the efficiency of building test projects for a large number of software packages. At the same time, by integrating a batch test control module, test cases of software packages can be called in sequence, thereby performing fuzz testing on the software packages. In this way, the automation of fuzz testing of software packages is achieved. According to this solution, the efficiency of batch software package testing can be improved.

[0045] In a possible implementation manner, the device further includes:

[0046] The storage module is used to store the test cases and test configuration information of each software package in the installation directory where the batch test control module is located.

[0047] In a possible implementation, the batch test control module is a mugen tool.

[0048] In a possible implementation manner, the identification information of the software package includes a target identification, and the generating module is further configured to:

[0049] According to the target identifier, a target test script is obtained, wherein the target test script is used to indicate a target execution function that needs to be executed by the test software package;

[0050] According to the common test script and the target test script, test cases and test configuration information of the software package are generated.

[0051] In this way, the software packages that need to perform special operations during the test process can be identified through target identification, so as to match the target test script, realize the automated construction of test cases, and thus improve the efficiency of batch software package testing.

[0052] In a possible implementation, the generating module is used to:

[0053] Integrate the common test script and the target test script to obtain an integrated test script;

[0054] The integrated test script is compiled to obtain the test cases and test configuration information of the software package.

[0055] In this way, the test cases can be automatically constructed, thereby improving the efficiency of batch software package testing.

[0056] In a possible implementation manner, the device further includes:

[0057] A fifth acquisition module, used to acquire the test log corresponding to the software package;

[0058] A parsing module, used to parse the test log and determine a keyword in the test log that indicates a failure;

[0059] The matching module is used to match the failure reason causing the failure of the test case execution from a pre-configured fault information library according to the keyword used to indicate the test failure in the test log.

[0060] In this way, the test results of the batch tested software packages can be automatically analyzed, thereby improving the efficiency of solving software package failures.

[0061] In a possible implementation, the fault information database further includes solutions corresponding to different failure causes, and the device further includes:

[0062] The matching module is used to match the solution corresponding to the failure cause from the fault information library according to the failure cause that causes the test case to fail to execute.

[0063] In this way, a solution to the software package failure can be obtained quickly, thereby improving the efficiency of solving the software package failure.

[0064] In a possible implementation manner, the device further includes:

[0065] The determination module is used to determine whether the test case caused the test failure based on the test log;

[0066] The parsing module is used to parse the test log and determine the keyword in the test log that indicates the failure when the test failure is not caused by the test case.

[0067] In a possible implementation, the common test script includes a test environment preparation function, a test software execution function, and a test environment recovery function;

[0068] in,

[0069] The test environment preparation function refers to the operations that need to be performed on the test environment before testing the software;

[0070] The test software execution function refers to the operations that need to be performed during the test software process;

[0071] The test environment recovery function refers to the operations required to restore the test environment after the software test is completed.

[0072] In a third aspect, an embodiment of the present application provides a computing device, including: a memory and a processor;

[0073] Memory and processor coupling;

[0074] a memory for storing computer program instructions;

[0075] a processor for executing computer program instructions;

[0076] When the computer program instructions stored in the memory are executed, the processor is configured to perform the method provided in the first aspect.

[0077] In a fourth aspect, an embodiment of the present application provides a computing device cluster, which includes the computing device involved in the third aspect, and the computing device is used to execute the method provided in the first aspect or any possible implementation manner of the first aspect.

[0078] In a fifth aspect, an embodiment of the present application provides a software test chip, characterized in that the chip is used to run computer program instructions to execute the method provided in the first aspect or any possible implementation of the first aspect. Exemplarily, the device can be a processor.

[0079] In one example, the chip may include a processor, which may be coupled to a memory, read instructions in the memory, and execute the method provided in the first aspect or any possible implementation of the first aspect according to the instructions. The memory may be integrated in the chip or may be independent of the chip.

[0080] In a sixth aspect, an embodiment of the present application provides a computer storage medium, in which instructions are stored. When the instructions are executed on a computing device, the computing device executes the method provided in the first aspect or any possible implementation of the first aspect.

[0081] In a seventh aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computing device, causes the computing device to execute the method provided in the first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 It is a schematic diagram of the architecture of a test device provided in an embodiment of the present application;

[0083] Figure 2 is a schematic diagram of the architecture of another testing device provided in an embodiment of the present application;

[0084] Figure 3 It is a schematic diagram of the architecture of another test device provided in an embodiment of the present application;

[0085] Figure 4 It is a flowchart of a software testing method provided in an embodiment of the present application;

[0086] Figure 5 It is a flowchart of another software testing method provided in an embodiment of the present application;

[0087] Figure 6 It is a schematic diagram of a flow chart of analyzing test results provided by an embodiment of the present application;

[0088] Figure 7 It is a flowchart of another software testing method provided in an embodiment of the present application;

[0089] Figure 8 It is a structural schematic diagram of a software testing device provided in an embodiment of the present application;

[0090] Fig. 9 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0091] Fig.10 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0092] Fig.11 It is a structural diagram of another computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0093] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0094] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0095] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refers to two or more systems, and multiple terminals refers to two or more terminals.

[0096] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0097] Before introducing this solution, the technical terms involved in this solution are first explained.

[0098] 1. Fuzz testing: An automated software testing technique that discovers software vulnerabilities by providing unexpected inputs to the target system and monitoring abnormal results. This method can reveal important software vulnerabilities in the program and verify error patterns in the real world. In fuzz testing, randomly generated data (also called fuzz) is used to attack the system under test to observe what destructive behavior the program will exhibit when encountering this data. The characteristic of fuzz testing is that it does not rely on logic, but tries as many possible inputs as possible to discover potential vulnerabilities.

[0099] 2. Dependency package: library files used to assist software operation. Dependency packages may include environment files required for software operation, such as operating system library files.

[0100] With the development of information technology, computer software has gradually penetrated into all aspects of social life. With the explosive growth of computer software, the software quality issues that follow have also greatly affected our lives. Software defects are a key factor affecting software quality, and they are also inevitable in the software development process. In order to ensure the stability and security of the software during operation, the software needs to be tested before it is released.

[0101] Currently, automated fuzz testing tools (such as fuzzing for open source software, OSS-Fuzz) simulate software behaviors in various situations by generating a large amount of random input data. It passes these random input data to the software and monitors the software's behavior and response, thereby determining whether the software has security issues and software defects during the software testing process.

[0102] Software testing usually requires building a test project corresponding to the software, and then integrating fuzz testing tools based on the test project to test the software. In this way, each time a software is tested, a test project corresponding to the software needs to be built. Some products include a large number of software, such as operating systems. Therefore, when there are a large number of software, building a test project takes a lot of time, which reduces the efficiency of software testing.

[0103] Based on this, an embodiment of the present application provides a software testing method. The method generates test cases for software packages in batches through public test scripts. At the same time, by integrating the mugen tool, the test cases of the software package can be called in sequence, thereby performing fuzz testing on the software package. In this way, the automation of fuzz testing of the software package is achieved. According to this solution, the efficiency of building a test project for a large number of software packages is improved, and the efficiency of batch software package testing can be improved.

[0104] The software testing method provided in the embodiment of the present application is applied to the testing device. Figure 1 As shown, the test device may be a computing device 101 .

[0105] In some embodiments, the test device may further include a display 102. The test device 101 performs batch testing on multiple software packages to be tested by running the test software. The display 102 can display a graphical user interface (GUI) provided by the test software. The user can input instructions or information through the GUI displayed by the display 102, for example, input the software package to be tested through the GUI, configure the use case function required in the process of testing the software package, etc.

[0106] In some embodiments, multiple virtual machines are deployed in the test device, and the multiple virtual machines can communicate with each other. Figure 2As shown, virtual machines 21, 22 and 23 are deployed in the test device. Among them, the test software A includes three parts, namely test software A1, test software A2 and test software A3. Among them, test software A1 is deployed in virtual machine 21, test software A2 is deployed in virtual machine 22, and test software A3 is deployed in virtual machine 23. Different virtual machines run different parts of the software, thereby executing the software testing method provided in the embodiment of the present application.

[0107] It should be noted that the test device can be a computing device or a cluster composed of multiple computing devices. Exemplarily, the computing device can be a server. In the case where the test device includes multiple computing devices, the multiple computing devices can communicate with each other. Therefore, it can be understood that multiple virtual machines can be deployed in different computing devices respectively, or in one computing device, or some virtual machines can be deployed in the same computing device, etc. The deployment of virtual machines is not specifically limited in the embodiments of the present application.

[0108] In the embodiments of the present application, Figure 3 As shown, the test software includes a test case generation module 31, a batch test control module 32 and a fuzz test module 33. Among them, the test case generation module 31 is used to generate test configuration information and corresponding test cases for each software package based on a common test script. The batch test control module 32 is used to control the fuzz test module 33 to perform batch tests on the software packages. Exemplarily, the batch test control module 32 can be a control tool for batch testing (such as a mugen tool). The fuzz test module 33 can perform batch tests on multiple software packages under the control of the batch test control module 32. Exemplarily, the fuzz test module 33 can be a fuzz test tool (such as OSS-Fuzz).

[0109] As an example, according to Figure 2 In the deployment mode of the test software, the test case generation module 31 can be deployed in the virtual machine 21, the batch test control module 32 can be deployed in the virtual machine 22, and the fuzzy test module 33 can be deployed in the virtual machine 23. Here, the test case generation module 31, the batch test control module 32 and the fuzzy test module 33 are not limited to the above deployment mode, and can also be other deployment modes, for example, the test case generation module 31 and the batch test control module 32 are deployed in the same virtual machine, and the fuzzy test module 33 is deployed in another virtual machine, etc., which is not specifically limited in the embodiments of the present application.

[0110] The above is the architecture of the test device provided in the embodiment of the present application. Figure 1 and Figure 2The architecture of the test equipment shown provides a detailed description of the software testing method provided in the embodiment of the present application.

[0111] Figure 4 The software testing method provided in the embodiment of the present application is applied to Figure 1 The test equipment shown. Figure 4 As shown, the software testing method provided in the embodiment of the present application includes the following steps S401 to S410.

[0112] S401, obtaining identification information of each of a plurality of software packages to be tested.

[0113] A software package refers to a program or a group of programs with specific functions used to complete specific tasks. After the user develops the software package, the performance of the software package needs to be tested. The identification information of the software package is used to indicate the identity of the software package. Exemplarily, the identification information of the software package may include the name of the software package and the version number of the software package.

[0114] The testing device can determine the identity of the software package to be tested by acquiring the identification information of each of the multiple software packages to be tested.

[0115] As a possible implementation manner, the user can input identification information of each of the multiple software packages to be tested through a graphical user interface of the testing device.

[0116] As another possible implementation, after developing multiple software packages, the user stores the identification information of the multiple software packages in the same file. The user can upload the file to the test device. For example, the identification information of the multiple software packages is shown in Table 1, and the user can transmit Table 1 to the test device through a computing device that communicates with the test device (such as a computing device that develops the software packages).

[0117] S402: Acquire the software package according to the identification information of the software package.

[0118] The test device can obtain the software package according to the identification information of the software package.

[0119] As a possible implementation, the test device can communicate with the computing device that develops the software package, and the test device can send a request to the computing device that develops the software package, and the request carries the identification information of the software package. The computing device that develops the software package sends the software package to the test device according to the identification information of the software package.

[0120] S403, obtaining a public test script.

[0121] A public test script can be a piece of test code. The public test script includes a test environment preparation function, a test environment execution function, and a test environment recovery function. The test environment preparation function refers to the operations that need to be performed on the test environment before testing the software. For example, obtaining the dependency packages required in the process of testing the software. The test software execution function refers to the operations that need to be performed in the process of testing the software. For example, the startup instructions for starting the fuzz test, etc. The test environment recovery function refers to the operations that need to be performed to restore the test environment after the software test is completed. For example, uninstalling dependent packages, clearing the source code of the software package that has been tested, etc.

[0122] For example, Figure 5 As shown, the public test script can be stored in a database, and the test device can read the public test script from the database. For example, the test device can send a request to the database to request the public test script. It is understandable that the database can be deployed in the test device or in other devices, and the embodiments of the present application are not limited to this.

[0123] S404, determine whether the identification information of the software package contains the target identification. If yes, execute S406, if not, execute S405.

[0124] In some embodiments, the identification information may further include a target identification, wherein the target identification is used to indicate a special operation that the test software package needs to perform.

[0125] S405: Generate test cases and test configuration information of the software package according to the common test script.

[0126] S406, obtaining a target test script.

[0127] The target test script is used to indicate the target execution function that the test software package needs to execute, and the target execution function test software package needs to execute special operations. Figure 5 As shown, the test script corresponding to the special operation can be stored in a database, and the test device can read the test script corresponding to the special operation matching the target identifier, ie, the target test script, from the database.

[0128] S407, generating test cases and test configuration information of the software package according to the common test script and the target test script.

[0129] Specifically, the common test script and the target test script are integrated, and the integrated test script is encoded to obtain the test case of the software package.

[0130] For example, in S405 and S407, the format of the test case may be a .sh file. After the test device generates the test configuration information, the test configuration information is stored in the test configuration file (eg, Figure 5 In the test configuration list shown in FIG. 1 , the test configuration information of all software packages can be uniformly stored in the test configuration file. The file format of the test configuration file can be a .json file format.

[0131] The test configuration information is used to indicate the correspondence between the software package and the test case. For example, Figure 5 As shown, the software packages to be tested include software A to software packages E. Among them, the test case of software package A is test case 1, the test case of software package B is test case 2, the test case of software package C is test case 3, the test case of software package D is test case 4, and the test case of software package E is test case 5. In this way, the above steps S402 to S407 are looped to sequentially generate test cases for each of the multiple software packages to be tested.

[0132] In some embodiments, test cases for multiple software packages may be generated in parallel. For example, in S402, each software package may be obtained based on identification information of the multiple software packages. In S405 to S407, test cases and test configuration information for each software package may be generated in parallel. The manner of generating test cases is described in detail in S403 to S407 above, which will not be repeated here.

[0133] S408, storing the test cases and test configuration files of the software package in the installation directory where the batch test control module is located.

[0134] After the test device generates the test cases and test configuration files of the software package, the test cases and test configuration files of the software package are stored in the installation directory where the batch test control module is located. As a possible implementation method, the identifier of the test case of the software package is associated with the software package. For example, software package A corresponds to test case 1, and the installation directory where the test control module is located is / root / mugen / , then the storage path of test case 1 is / root / mugen / 1, where 1 represents the identifier of test case 1. According to the correspondence between the test case and the software package in the test configuration file, when testing software package 1, test case 1 is called according to the storage path / root / mugen / 1.

[0135] S409, determining whether the test cases of the multiple software packages to be tested have been generated. If yes, executing S409, if not, jumping to S402, replacing the identification information of the software package to obtain the next new software package.

[0136] S410, calling the test cases of the software packages in sequence, and performing fuzz testing on the multiple software packages.

[0137] A batch test control module (e.g., mugen tool) and a fuzz test tool (e.g., OSS-Fuzz) can be installed in the test device. The batch test control module can control the fuzz test tool to call the test cases of the software package in sequence according to the test configuration file to test the software package.

[0138] For example, Figure 5 As shown in the figure, the test case of software package A is test case 1. Under the control of the mugen tool, the OSS-Fuzz tool calls test case 1 to test software package A. After the test of software package A is completed, the OSS-Fuzz tool calls test case 2 to test software package B. And so on, until the test of software package E is completed.

[0139] In some embodiments, the batch test control module can send test instructions to the fuzz testing tool, so that the batch test control module controls the fuzz testing tool to test the software packages in sequence.

[0140] As a possible implementation, the test instruction can be used to instruct to execute the fuzz test of all software packages. Here, the test instruction includes the correspondence between the software packages and the test cases. Thus, the fuzz test tool calls the test cases of the software packages in sequence according to the correspondence. For example, Figure 5 As shown, the software packages to be tested include software package A, software package B, software package C, software package D and software package E. According to the test configuration information, software package A corresponds to test case 1, software package B corresponds to test case 2, software package C corresponds to test case 3, software package D corresponds to test case 4, and software package E corresponds to test case 5. Then, the test instruction carries the correspondence between the above software packages and test cases. According to the test instruction, the fuzz testing tool can call test case 1 to test software package A in sequence according to the arrangement order of the software packages in the test instruction. After the test of software package A is completed, the fuzz testing tool calls test case 2 to test software package B. And so on, until the test of software package E is completed.

[0141] As another possible implementation, the test instruction is used to instruct the execution of a fuzz test of a certain software package. When the fuzz test of the software package is completed, the fuzz test tool feeds back the test log to the mugen tool, thereby triggering the mugen tool to send the test instruction to the fuzz test tool again. This cycle continues until all software package tests are completed. For example, the software packages to be tested include software packages A, B, C, D, and E. Among them, software package A corresponds to test case 1, software package B corresponds to test case 2, software package C corresponds to test case 3, software package D corresponds to test case 4, and software package E corresponds to test case 5. The mugen tool sends a test instruction A01 to the fuzz test tool, wherein the test instruction A01 indicates the correspondence between software package A and test case 1. According to the test instruction A01, the fuzz test tool calls test case 1 to test software package A, and transmits the log generated after the test is completed to the mugen tool. Then, the mugen tool sends a test instruction B02 to the fuzz test tool. According to the test instruction B02, the fuzz test tool calls test case 2 to test software package B, and transmits the log generated after the test is completed to the mugen tool. This continues until all packages have been tested.

[0142] In some embodiments, the test configuration information also includes the number of computing resources required when testing the software package, such as the number of central processing unit (CPU) cores, the amount of memory, etc.

[0143] Exemplarily, the batch test control module sends test instructions to the fuzz testing tool, wherein the test instructions may include the number of computing resources required for testing each software package, so that the fuzz testing tool calls computing resources to test the software package according to the number of computing resources, thereby avoiding waste of computing resources.

[0144] After the software package is tested, a test log for each software package is generated. According to the test log, the reason for the failure of the test case execution is analyzed. The method of analyzing the failure of the test case execution is as follows: first, the test log is parsed to determine the keywords used to indicate the failure in the test log. Then, according to the keywords used to indicate the test failure in the test log, the failure reason that caused the failure of the test case execution is matched from the pre-configured fault information library.

[0145] Specifically, Figure 6 As shown, the process of performing failure analysis on the test case includes S601 to S604.

[0146] S601, obtaining test results.

[0147] In an achievable manner, the test device may read the test results generated by the fuzz testing tool, wherein the test results include a test log, which is the result of different test operations generated during the execution of the test case.

[0148] S602: According to the test result, determine whether the test case caused the test failure. If not, execute S603.

[0149] The test results can indicate the factors that caused the test failure. The factors that caused the test failure can be the test case itself or the tested software package. If the factor that caused the test failure is the test case, then you need to rebuild the test case and then retest the performance of the software package. If the factor that caused the test failure is the software package, then you need to analyze the vulnerabilities in the software package based on the test log and repair the software package.

[0150] S603, parsing the test log to determine a keyword used to indicate a failure.

[0151] The test log includes the results of various test operations during the test process. If the result is failure, the keywords in the result are extracted.

[0152] S604: Match the failure reason of the test case execution failure according to the keyword indicating failure and the pre-configured fault information library.

[0153] The fault information library includes the correspondence between different failure keywords and failure reasons. Therefore, according to the keyword indicating failure, the failure reason can be matched from the fault information library. In this way, the automatic analysis of the test results is realized, and the analysis efficiency of the test results is improved.

[0154] In some embodiments, the fault information library may also include solutions corresponding to different failure reasons. After determining the failure reason of the test case execution failure, the solution may be matched from the fault information library according to the failure reason. In this way, the efficiency of fault resolution is improved.

[0155] According to the embodiment of the present application, test cases of software packages are generated in batches through common test scripts, thereby improving the efficiency of building test projects for a large number of software packages. At the same time, by integrating a batch test control module, test cases of software packages can be called in sequence, thereby performing fuzz testing on the software packages. In this way, the automation of fuzz testing of software packages is achieved. According to this solution, the efficiency of batch software package testing can be improved.

[0156] Figure 7 FIG. 1 is a flow chart of a software testing method provided in an embodiment of the present application. Figure 7As shown, the software testing method provided in the embodiment of the present application includes S701 to S703.

[0157] S701, respectively obtain multiple software packages to be tested and public test scripts.

[0158] The method of obtaining multiple software packages to be tested can be specifically referred to above. Figure 4 The introduction and implementation of S401 and S402 will not be repeated here.

[0159] For details on obtaining public test scripts, please refer to the above Figure 4 The introduction and implementation of S404 to S407 will not be repeated here.

[0160] S702: Generate test cases and test configuration information for each software package according to the common test script, where the test configuration information is used to indicate the corresponding relationship between the software package and the test case.

[0161] In some embodiments, the identification information of the software package includes a target identification, and generating test cases and test configuration information of the software package according to the public test script includes:

[0162] According to the target identifier, a target test script is obtained, where the target test script is used to indicate a target execution function that needs to be executed by the test software package;

[0163] Generate test cases and test configuration information for the software package based on the common test scripts and target test scripts.

[0164] Specifically, the common test script and the target test script are integrated to obtain an integrated test script; the integrated test script is compiled to obtain test cases and test configuration information of the software package.

[0165] In some embodiments, the common test script includes a test environment preparation function, a test software execution function, and a test environment recovery function.

[0166] In some embodiments, the batch test control module is a mugen tool.

[0167] For details, please refer to the above Figure 4 The introduction and implementation of S404 to S407 will not be repeated here.

[0168] In some embodiments, after the test cases and configuration information of the software package are generated, the test cases and test configuration files of the software package are stored in the installation directory where the batch test control module is located. For details, please refer to the above introduction to S408, which will not be repeated here.

[0169] S703, calling the test cases of each software package in sequence according to the test configuration information of each software package through the batch test control module to perform fuzzy testing on each software package.

[0170] For details, please refer to the above Figure 4 The introduction and implementation of S408 to S410 will not be repeated here.

[0171] In some embodiments, the method further comprises:

[0172] Get the test log corresponding to the software package;

[0173] Parse the test logs to identify keywords in the test logs that indicate failures;

[0174] According to the keywords used to indicate test failure in the test log, the failure reasons that cause the test case execution failure are matched from a pre-configured failure information library.

[0175] Specifically, the fault information database also includes solutions corresponding to different failure causes, and the method also includes:

[0176] According to the failure reason that caused the test case execution failure, the solution corresponding to the failure reason is matched from the fault information library.

[0177] In some embodiments, the failure of the test case execution may also be caused by the test case itself. In order to improve the efficiency of test failure analysis, before parsing the test log and determining the keywords used to indicate failure in the test log, it is also necessary to determine whether the test case itself caused the test failure. In the case where the test case itself does not cause the test failure, the test log is parsed to determine the keywords used to indicate failure in the test log. For details, please refer to the detailed description of S602, which will not be repeated here.

[0178] For details, see the above Figure 6 The implementation introduced in S601 to S604 will not be repeated here.

[0179] According to the embodiment of the present application, test cases of software packages are generated in batches through common test scripts, thereby improving the efficiency of building test projects for a large number of software packages. At the same time, by integrating a batch test control module, test cases of software packages can be called in sequence, thereby performing fuzz testing on the software packages. In this way, the automation of fuzz testing of software packages is achieved. According to this solution, the efficiency of batch software package testing can be improved.

[0180] Based on the same concept as the embodiment of the present application method, the embodiment of the present application also provides a software testing device. The software testing device includes several modules, each module is used to execute each step in the software testing method provided by the embodiment of the present application, and the division of the modules is not limited here. It can be clearly understood by the technicians in the relevant field that in practical applications, the various steps in the software testing method provided by the embodiment of the present application can be assigned to different modules as needed, that is, the internal structure of the device is divided into different modules to complete all or part of the functions described above. Each module in the embodiment can be integrated in a processing unit, or each unit can exist physically alone, or two or more modules can be integrated in one unit, and the above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of each module are only for the convenience of distinguishing from each other, and are not used to limit the scope of protection of the present application. The specific working process of the module in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0181] For example, the software testing device is used to execute the embodiment of the present application. Figure 4 The corresponding embodiment provides a software testing method.

[0182] Figure 8 Schematic diagram of the structure of the software testing device provided in the embodiment of the present application. Figure 8 As shown, the software testing device provided in the embodiment of the present application includes:

[0183] The first acquisition module 801 is used to respectively acquire a plurality of software to be tested and a common test script;

[0184] In a specific implementation, the first acquisition module includes a second acquisition module and a third acquisition module; wherein,

[0185] A second acquisition module is used to acquire the software package corresponding to the identification information according to the identification information of each software package;

[0186] The third acquisition module is used to obtain the public test script;

[0187] In some examples, Figure 3 The test case generation module 31 shown may include an acquisition module 801 and a generation module 802. The call module 803 may include Figure 3 A batch test control module 32 and a fuzzy test module 33 are shown.

[0188] A generating module 802, used to generate a test case and test configuration information for each software package according to a common test script, wherein the test configuration information is used to indicate a corresponding relationship between the software package and the test case;

[0189] The calling module 803 is used to call the test cases of each software package in sequence according to the test configuration information of each software package through the batch test control module to perform fuzzy testing on each software package.

[0190] According to the solution, test cases of software packages are generated in batches through public test scripts, thereby improving the efficiency of building test projects for a large number of software packages. At the same time, by integrating a batch test control module, test cases of software packages can be called in sequence, thereby performing fuzz testing on the software packages. In this way, the automation of fuzz testing of software packages is achieved. According to this solution, the efficiency of batch software package testing can be improved.

[0191] In a possible implementation, the device further includes:

[0192] The storage module is used to store the test cases and test configuration information of each software package in the installation directory where the batch test control module is located.

[0193] In a possible implementation, the batch test control module is a mugen tool.

[0194] In a possible implementation, the identification information of the software package includes a target identification, and the generating module is further used for:

[0195] According to the target identifier, a target test script is obtained, where the target test script is used to indicate a target execution function that needs to be executed by the test software package;

[0196] Generate test cases and test configuration information for the software package based on the common test scripts and target test scripts.

[0197] In this way, the software packages that need to perform special operations during the test process can be identified through target identification, so as to match the target test script, realize the automated construction of test cases, and thus improve the efficiency of batch software package testing.

[0198] In a possible implementation, the generation module is used to:

[0199] Integrate the common test script and the target test script to obtain an integrated test script;

[0200] Compile the integrated test script to obtain the test cases and test configuration information of the software package.

[0201] In this way, the test cases can be automatically constructed, thereby improving the efficiency of batch software package testing.

[0202] In a possible implementation, the device further includes:

[0203] The fifth acquisition module is used to obtain the test log corresponding to the software package;

[0204] A parsing module, used to parse the test log and determine the keywords in the test log that indicate failure;

[0205] The matching module is used to match the failure cause of the test case execution failure from a pre-configured failure information library according to the keywords used to indicate the test failure in the test log.

[0206] In this way, the test results of the batch tested software packages can be automatically analyzed, thereby improving the efficiency of solving software package failures.

[0207] In a possible implementation, the fault information database further includes solutions corresponding to different failure causes, and the device further includes:

[0208] The matching module is used to match the solution corresponding to the failure cause from the fault information library according to the failure cause that causes the test case execution failure.

[0209] In this way, a solution to the software package failure can be obtained quickly, thereby improving the efficiency of solving the software package failure.

[0210] In a possible implementation, the device further includes:

[0211] The determination module is used to determine whether the test case caused the test failure based on the test log;

[0212] The parsing module is used to parse the test log and determine the keywords in the test log that indicate the failure when the test case is not the cause of the test failure.

[0213] In a possible implementation, the common test script includes a test environment preparation function, a test software execution function, and a test environment recovery function;

[0214] in,

[0215] The test environment preparation function refers to the operations that need to be performed on the test environment before testing the software;

[0216] The test software execution function refers to the operations that need to be performed during the test software process;

[0217] The test environment recovery function refers to the operations required to restore the test environment after the software test is completed.

[0218] Based on the same concept as the method embodiment of the present application, the embodiment of the present application also provides a computing device.

[0219] Fig. 9 Schematic diagram of a computing device provided in an embodiment of the present application. Fig. 9As shown, computing device 900 includes a processor 901 , a memory 902 , and a network interface 903 .

[0220] The processor 901 may be a CPU, or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0221] Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous connection dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0222] For example, a computer program can be stored on the memory 902, and when the processor 901 executes the computer program, the steps in the embodiment of the testing method of the above-mentioned software are implemented. Alternatively, when the processor 901 executes the computer program, the functions of each module in the above-mentioned device embodiment are implemented, such as implementing the test case generation module 31, the batch test control module 32, and the fuzzy test module 33 in the aforementioned embodiment, thereby implementing the testing method of the aforementioned software. Exemplarily, the computer program can be divided into one or more modules / units, and one or more modules / units can be a series of computer program instruction segments that can perform specific functions. One or more modules / units are stored in the memory 902 and executed by the processor 901 to complete the present application.

[0223] The network interface 903 is used to send and receive data, for example, to send data processed by the processor 901 to other computing devices, or to receive data sent by other computing devices.

[0224] Of course, to simplify, Fig. 9 Only some of the components of the computing device 900 related to the present application are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, the computing device 900 may also include any other appropriate components according to specific application conditions. In addition, the computing device may be a desktop computer, a notebook, a handheld computer, a cloud server, etc. Those skilled in the art will understand that Fig. 9 This is only an example of the computing device 900 and does not constitute a limitation of the computing device. The computing device may include more or fewer components than shown in the figure, or some components may be combined, or different components may be included. For example, the computing device may also include an input device, an output device, a network access device, a bus, etc. For example, the input device may be a microphone array, and may also include, for example, a keyboard, a mouse, etc. For example, the output device may output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0225] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes a computing device. The computing device may be a server. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0226] like Fig.10 As shown, the computing device cluster includes a computing device 900. The memory 902 in one or more computing devices 900 in the computing device cluster may store the same instructions for executing the software testing method.

[0227] In some possible implementations, the memory 902 of one or more computing devices 900 in the computing device cluster may also store partial instructions for executing the software testing method. In other words, the combination of one or more computing devices 900 may jointly execute the instructions for executing the software testing method.

[0228] It should be noted that the memory 902 in different computing devices 900 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the test platform. That is, the instructions stored in the memory 902 in different computing devices 900 can implement the functions of one or more modules in the test case generation module 31, the batch test control module 32, and the fuzzy test module 33.

[0229] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.11 A possible implementation is shown. Fig.11 As shown, two computing devices 900A and 900B are connected via a network. Specifically, they are connected to the network via a communication interface in each computing device. In this type of possible implementation, the memory 902 in the computing device 900A stores instructions for executing the functions of the test case generation module 31 and the batch test control module 32. At the same time, the memory 902 in the computing device 900B stores instructions for executing the functions of the fuzzy test module 33.

[0230] Understandably, Fig.11 The implementation shown can be used as a software test system, wherein computing device 900A is a first computing device, and computing device 900B can be used as a second computing device. The first computing device is used to execute steps S701 to S704 in the aforementioned embodiment, and the second computing device is used to execute step S705 in the aforementioned embodiment. The specific implementation process can be found in the detailed description of S701 to S705 in the aforementioned embodiment, which will not be repeated here.

[0231] Fig.11 The connection method between the computing device clusters shown may be based on the consideration that the software testing method provided in the present application requires communication between multiple containers and multiple devices under test, and therefore it is considered that the functions implemented by the multiple containers are handed over to the computing device 900B for execution.

[0232] It should be understood that Fig.11 The functions of the computing device 900A shown in FIG. 9A may also be completed by multiple computing devices 900. Similarly, the functions of the computing device 900B may also be completed by multiple computing devices 900.

[0233] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Fig.10 and Fig.11 The difference is that the memory 902 in one or more computing devices 900 in the computing device cluster may store the same instructions for executing the software testing method.

[0234] In some possible implementations, the memory 902 of one or more computing devices 900 in the computing device cluster may also store partial instructions for executing the software testing method. In other words, the combination of one or more computing devices 900 may jointly execute the instructions for executing the software testing method.

[0235] It should be noted that the memory 902 in different computing devices 900 in the computing device cluster can store different instructions for executing part of the functions of the software test system. That is, the instructions stored in the memory 902 in different computing devices 900 can implement the functions of one or more devices in the test case generation module 31, the batch test control module 32, and the fuzzy test module 33.

[0236] In addition, the embodiment of the present application may also provide a computer program product, which includes computer program instructions, and when the computer program instructions are executed by the processor, the processor executes the steps in the noise reduction method of various embodiments of the present application described in the above "Method" section of this specification. Among them, the computer program product can be written in any combination of one or more programming languages ​​to perform computer program codes for performing the operations of the embodiments of the present application, and the programming languages ​​include object-oriented programming languages, such as Java, C++, etc., and also include conventional procedural programming languages, such as "C" language or similar programming languages. Among them, the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer program code can be executed completely on the user computing device, partially on the user device, as an independent software, partially on the user computing device and partially on the remote computing device, or completely on the remote computing device or server.

[0237] In addition, the embodiment of the present application may also provide a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions, when executed by the processor, cause the processor to execute the steps in the display control method according to various embodiments of the present application described in the "Method" section of the present specification. The computer-readable storage medium may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0238] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0239] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted by a computer-readable storage medium. The computer instructions can be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

[0240] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0241] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0242] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0243] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0244] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0245] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

[0246] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

Claims

1. A software testing method, characterized in that: The method comprises: Get multiple software packages to be tested and public test scripts respectively; Determining whether the software package includes a target identifier, where the target identifier is used to indicate the identity of the software package; Generate a test case and test configuration information for each of the software packages; wherein the test configuration information is used to indicate a corresponding relationship between the software packages and the test cases; Wherein, in the case where the software package does not include the target identifier, generating a test case and the test configuration information for each software package according to the common test script; In the case where the software package includes the target identifier, obtaining a target test script of the software package, and generating a test case and the test configuration information of the software package according to the target test script and the common test script, wherein the common test script includes a test environment preparation function, a test software execution function and a test environment recovery function, and the target test script is used to indicate a target execution function that needs to be executed to test the software package; The batch test control module sequentially calls the test cases of each software package according to the test configuration information of each software package to perform fuzzy testing on each software package.

2. The method according to claim 1, characterized in that Before the batch test control module sequentially calls the test cases of each software package according to the test configuration information of each software package to perform fuzzy testing on each software package, the method further includes: The test cases and test configuration information of each software package are stored in the installation directory where the batch test control module is located.

3. The method according to claim 1, characterized in that The batch test control module is the mugen tool.

4. The method according to claim 1, characterized in that: Generating test cases and test configuration information of the software package according to the common test script and the target test script, including: Integrate the common test script and the target test script to obtain an integrated test script; The integrated test script is compiled to obtain the test cases and test configuration information of the software package.

5. The method according to claim 1, characterized in that The method further comprises: Obtain the test log corresponding to the software package; Parsing the test log to determine a keyword in the test log that indicates a failure; According to the keywords used to indicate the test failure in the test log, the failure reasons causing the failure of the test case execution are matched from a pre-configured fault information library.

6. The method according to claim 5, characterized in that The fault information database also includes solutions corresponding to different failure causes, and the method further includes: According to the failure cause that causes the test case to fail to execute, a solution corresponding to the failure cause is matched from the fault information library.

7. The method according to claim 6, characterized in that Before parsing the test log to determine a keyword in the test log used to indicate a failure, the method further includes: Based on the test log, determine whether the test case caused the test failure; The parsing the test log to determine a keyword in the test log used to indicate a failure includes: In the case where the test failure is not caused by the test case, the test log is parsed to determine a keyword in the test log that indicates the failure.

8. The method according to any one of claims 1 to 7, characterized in that: The test environment preparation function refers to the operations that need to be performed on the test environment before testing the software; The test software execution function refers to the operation that needs to be performed during the test software process; The test environment recovery function refers to the operation required to restore the test environment after the software test is completed.

9. A computing device, characterized in that include: Memory and processor; The memory is coupled to the processor; a memory for storing computer program instructions; a processor for executing the computer program instructions; When the computer program instructions stored in the memory are executed, the processor is configured to perform the method according to any one of claims 1 to 8.

Images