Application program testing method and device, storage medium, and electronic device

Abstract

The present invention discloses a method and device for testing an application, a storage medium, and an electronic device. The method relates to the field of cloud computing, wherein the method comprises: traversing an application file set to extract a test program carrying preset identification information to obtain multiple test cases; obtaining a target program, a test strategy identifier, and a test parameter value for each test case to establish a case value library; based on the case value library, matching the program identifier of the application to be tested with the test strategy identifier in the case value library to obtain the program parameter value of the target program that has been successfully matched; using the program parameter value as a test seed to perform a fuzzy test on the application to be tested. The present invention solves the technical problem in the related art that the ability to hit potential defects of the application is low by performing fuzzy testing on the application by randomly selecting parameter values.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing, and in particular to a method and device for testing an application program, a storage medium, and an electronic device. Background Art

[0002] In the related art, most of the common fuzz testing frameworks perform fuzz testing by randomly injecting parameter values ​​of the target method. This is because the essence of fuzz testing is random testing. Although randomly injected parameter values ​​may hit potential defects in the system, it may also run for a long time without any results. The main reason is its randomness. For example, if there is a target method that only accepts parameter values ​​of zero or positive integers, but the randomly injected parameter values ​​are always generated in the negative range or in a string, then this series of tests will be futile, that is, the ability to hit potential defects of the application by fuzz testing the application by randomly selecting parameter values ​​is low.

[0003] Figure 1 It is a schematic diagram of injecting random values ​​according to a fuzzy test in the prior art, such as Figure 1 As shown in the figure, the black circle represents the range of parameter values ​​that the target method can accept, and the white circle represents the random value generated by the fuzz test. The two circles have no intersection. If the random value is always in the white circle, anomalies will continue to occur during the testing of the target test method, and the fuzz test work will be completely ineffective.

[0004] Therefore, a technical solution is needed that can control the random values ​​or specify some directions at the beginning of the fuzz test to improve the quality and effect of the fuzz test.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present invention provide a method and device for testing an application, a storage medium, and an electronic device, so as to at least solve the technical problem in the related art that the ability to hit potential defects of the application by fuzz testing the application by randomly selecting parameter values ​​is low.

[0007] According to one aspect of an embodiment of the present invention, a method for testing an application is provided, comprising: traversing an application file collection to extract test programs carrying preset identification information to obtain multiple test cases; obtaining a target program, a test strategy identifier and a test parameter value for each of the test cases to establish a case value library; based on the case value library, matching the program identifier of the application to be tested with the test strategy identifier in the case value library to obtain the program parameter value of the target program that has been successfully matched; and using the program parameter value as a test seed to perform a fuzzy test on the application to be tested.

[0008] Optionally, the step of obtaining the target program, test strategy identifier and test parameter value of each test case includes: obtaining a software engineering reflection mechanism for the application to be tested; using the software engineering reflection mechanism to separate each test case in the case value library to obtain the target program, the test strategy identifier and the test parameter value in each test case.

[0009] Optionally, the step of using the program parameter value as a test seed to perform a fuzzy test on the application to be tested includes: mutating each of the program parameter values ​​as a test seed for the fuzzy test to obtain multiple test sets, wherein each of the test sets includes at least: the program parameter value and multiple mutated parameter values ​​corresponding to the program parameter value; injecting the parameter values ​​in each of the test sets into the test method corresponding to the test set in turn to perform a fuzzy test on the application to be tested.

[0010] Optionally, the step of injecting the parameter values ​​in each of the test sets into the test method corresponding to the test set in sequence, and performing fuzz testing on the application to be tested, further includes: injecting the parameter values ​​in the target test set and multiple variant parameter values ​​into the target test method corresponding to the target test set in sequence for testing, wherein the target test set is one of the multiple test sets; within a preset time period, monitoring whether the test process has a program exception, and obtaining a program monitoring status; when the program monitoring status indicates that the test process has a program exception, recording exception data, wherein the exception data includes at least: the number of exceptions, the test identifier of the target test method, and a target test seed; when the program monitoring status indicates that the test process has no program exception, selecting the next test set from the multiple test sets to continue the step of performing fuzz testing on the application to be tested.

[0011] Optionally, after recording the abnormal data, the method further includes: performing regression testing and data analysis on the application to be tested according to the abnormal data.

[0012] Optionally, the step of performing regression testing on the application to be tested according to the abnormal data includes: performing defect repair on the application to be tested according to the test identifier corresponding to the test set in the abnormal data to obtain a repaired application; injecting parameter values ​​in the target test set into the repaired application in sequence, and continuing to perform fuzzy testing on the repaired application.

[0013] Optionally, the step of performing data analysis on the application to be tested according to the abnormal data includes: evaluating a test result score of a target test seed in the abnormal data.

[0014] According to another aspect of an embodiment of the present invention, there is also provided a testing device for an application, comprising: a traversal unit, for traversing an application file collection to extract a test program carrying preset identification information to obtain a plurality of test cases; a first processing unit, for obtaining a target program, a test strategy identifier and a test parameter value for each of the test cases to establish a case value library; a second processing unit, for matching, based on the case value library, the program identifier of the application to be tested and the test strategy identifier in the case value library to obtain the program parameter value of the successfully matched target program; and a testing unit, for using the program parameter value as a test seed to perform a fuzzy test on the application to be tested.

[0015] Optionally, the first processing unit includes: an acquisition subunit, used to obtain a software engineering reflection mechanism for the application to be tested; and a separation subunit, used to adopt the software engineering reflection mechanism to separate each of the test cases in the case value library to obtain the target program, the test strategy identifier and the test parameter value in each of the test cases.

[0016] Optionally, the testing unit includes: a mutation subunit, used to mutate each of the program parameter values ​​as a test seed for the fuzzy test to obtain multiple test sets, wherein each of the test sets includes at least: the program parameter value and multiple mutated parameter values ​​corresponding to the program parameter value; a testing subunit, used to inject the parameter values ​​in each of the test sets into the test method corresponding to the test set in sequence to perform fuzzy testing on the application to be tested.

[0017] Optionally, the test sub-unit includes: a first test module, used to inject parameter values ​​and multiple variant parameter values ​​in a target test set into a target test method corresponding to the target test set in sequence for testing, wherein the target test set is one of the multiple test sets; a detection module, used to monitor whether a program exception occurs in the test process within a preset time period to obtain a program monitoring status; a recording module, used to record exception data when the program monitoring status indicates that a program exception occurs in the test process, wherein the exception data includes at least: the number of exceptions, the test identifier of the target test method, and a target test seed; a second test module, used to select the next test set from the multiple test sets when the program monitoring status indicates that no program exception occurs in the test process, so as to continue to perform the step of fuzz testing on the application to be tested.

[0018] Optionally, the testing subunit further includes: a processing module, configured to perform regression testing and data analysis on the application to be tested according to the abnormal data after recording the abnormal data.

[0019] Optionally, the processing module includes: a repair sub-module, used to repair defects in the application to be tested according to the test identifier corresponding to the test set in the abnormal data, so as to obtain a repaired application; and a testing sub-module, used to inject parameter values ​​in the target test set into the repaired application in sequence, and continue to perform fuzzy testing on the repaired application.

[0020] Optionally, the processing module further includes: an evaluation submodule, configured to evaluate the test result score of the target test seed in the abnormal data.

[0021] In the present invention, the application file set is first traversed to extract the test program carrying the preset identification information to obtain multiple test cases, and then the target program, test strategy identifier and test parameter value of each test case are obtained to establish a case value library, and then the program identifier of the application to be tested and the test strategy identifier in the case value library are matched based on the case value library to obtain the program parameter value of the target program that matches successfully, and finally the program parameter value is used as the test seed to perform fuzz testing on the application to be tested. In the present disclosure, by extracting test cases from the application file set, using the test parameter value of the test case as the seed of the fuzz test, and performing fuzz testing on the application to be tested, the ability to hit the potential defects of the application is improved, and the purpose of controlling the parameter value selection direction of the fuzz test is achieved, thereby solving the technical problem of low ability to hit the potential defects of the application by fuzz testing the application by randomly selecting parameter values ​​in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0023] Figure 1 is a schematic diagram of injecting random values ​​into a fuzzy test according to the prior art;

[0024] Figure 2 is a flow chart of a fuzzy testing method according to an embodiment of the present invention;

[0025] Figure 3 is a flow chart of an optional application testing method according to an embodiment of the present invention;

[0026] Figure 4is a flow chart of an optional UTDD case-based fuzzy testing method according to an embodiment of the present invention;

[0027] Figure 5 is a schematic diagram of an optional application program testing device according to an embodiment of the present invention;

[0028] Figure 6 The present invention is a hardware structure block diagram of an electronic device (or mobile device) according to a method for testing an application program in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] In order to facilitate those skilled in the art to understand the present invention, some nouns or terms involved in each embodiment of the present application are explained below:

[0032] Fuzz testing: is a method of discovering software vulnerabilities by semi-automatically inputting unexpected inputs to the target system and monitoring abnormal results. It is often used to detect more hidden software vulnerabilities given by software or computer systems, such as memory leaks, deadlocks, stack overflows and other unexpected anomalies. It can be used for black box, gray box or white box testing.

[0033] Test-driven development, Unit Test Driven Development, referred to as UTDD, is a core practice and technology in agile development, and also a design methodology. The core idea is to write unit test case code before developing functional code, and the test code determines what product code needs to be written. It is the core practice of extreme programming.

[0034] JUnit: A framework for writing unit test code in the Java programming language. Normally, each Java program has a corresponding test class written using JUnit.

[0035] It should be noted that the application testing method and device in the present disclosure can be used in the process of program testing of various software products in the field of cloud computing, and can also be used in any field other than the field of cloud computing. In the process of program testing of applications, the application field of the application testing method and device in the present disclosure is not limited.

[0036] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. For example, an interface is set between this system and the relevant user or organization. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0037] The present invention can be applied to the testing process of various software products, control systems, and client applications (including but not limited to: mobile clients, PCs, etc.) of various financial institutions. Through the tested applications installed on the mobile clients, the business content of the financial institutions (including but not limited to: transfer, financial management, funds, payment, account checking, advertising, recommendation and other business functions) can be realized.

[0038] In the digital information age, various software products can be seen everywhere. In order to ensure the stability of software product service operation and maintain the company's good image and reputation, various software companies are using various testing methods to improve software quality, such as unit testing, integration testing, process testing, adaptability testing, chaos testing, mutation testing, etc. At present, there is a testing method called fuzz testing that is gradually becoming popular. As a supplement to traditional testing methods, it has been used by more and more companies / enterprises. Practice has also proved that fuzz testing can find many system defects that are difficult to find in conventional testing, which helps to improve the quality of application software. The problem areas that the present invention can solve include but are not limited to the source code level, that is, each specific software engineering programming (such as a java program) is used as a target object, and the execution result is observed by calling the method in the program to see if there is an abnormality, so as to realize the robustness test of the software system.

[0039] Taking JAVA as an example, fuzz testing mainly adopts gray box testing method. Common fuzz testing frameworks include JFUZZ, JAZZER, etc. When using these frameworks, you need to write a small amount of test code to call the target method in the JAVA program in the test code. The input parameters of the target method are randomly injected through the fuzz testing engine. Each set of random values ​​represents a test case, and then observe whether the target method will produce exceptions during operation. Among them, the test case can include the target class and the fuzz test case class with Fuzz as the suffix.

[0040] Combine the following Figure 2 The fuzz testing process is described.

[0041] Figure 2 is a flow chart of a fuzzy testing method according to an embodiment of the present invention. Figure 2 As shown, during the fuzz testing process, test cases are generated to test the target program, and the program running status of the target program is monitored. If an exception occurs, the exception is analyzed. If no exception occurs, the target program continues to be tested according to the next generated test case.

[0042] The present invention extracts the test parameter values ​​in the existing UTDD cases as seeds for fuzzy testing initial values, so that the fuzzy test can be diffused and mutated within a relatively effective range, thereby avoiding a large number of blind random tests, improving the fuzzy testing effect, discovering more potential defects in the system, and improving the quality of the application.

[0043] The present invention is described in detail below in conjunction with various embodiments.

[0044] Embodiment 1

[0045] According to an embodiment of the present invention, an optional method embodiment for testing an application is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0046] Figure 3 is a flow chart of an optional application testing method according to an embodiment of the present invention. Figure 3 As shown, the method comprises the following steps:

[0047] Step S301, traversing the application file set to extract the test program carrying the preset identification information to obtain multiple test cases;

[0048] Step S302, obtaining the target program, test strategy identifier and test parameter value of each test case to establish a case value library;

[0049] Step S303, based on the case value library, matching is performed using the program identifier of the application program to be tested and the test strategy identifier in the case value library to obtain the program parameter value of the successfully matched target program;

[0050] Step S304: Use the program parameter value as a test seed to perform a fuzzy test on the application to be tested.

[0051] Through the above steps, the application file collection is first traversed to extract the test program with preset identification information to obtain multiple test cases, and then the target program, test strategy identifier and test parameter value of each test case are obtained to establish a case value library, and then the program identifier of the application to be tested and the test strategy identifier in the case value library are matched based on the case value library to obtain the program parameter value of the target program that successfully matches, and finally the program parameter value is used as the test seed to perform fuzz testing on the application to be tested. In this embodiment, by extracting test cases from the application file collection, using the test parameter value of the test case as the seed of the fuzz test, and performing fuzz testing on the application to be tested, the ability to hit the potential defects of the application is improved, and the purpose of controlling the parameter value selection direction of the fuzz test is achieved, thereby solving the technical problem of low ability to hit the potential defects of the application by fuzz testing the application by randomly selecting parameter values ​​in the related art.

[0052] The embodiments of the present invention are further described below in combination with the above-mentioned implementation steps.

[0053] Step S301, traverse the application file set to extract the test program carrying preset identification information to obtain multiple test cases.

[0054] An optional implementation method, traversing an application file collection to extract test programs carrying preset identification information to obtain multiple test cases, includes: in the application file collection, extracting applications carrying preset suffix identification information to obtain multiple application files; in multiple application files, extracting test programs carrying preset identification information to obtain multiple test cases.

[0055] The above-mentioned application file collection can be a software engineering file (for example, a JAVA engineering file), which can include multiple software programs. The above-mentioned preset suffix identification information can be the suffix of the application file. For example, the file with the suffix ".java" is a java program file. In the above-mentioned application file collection, all java application files with the suffix ".java" can be extracted.

[0056] The above-mentioned preset identification information may be information identifying the application type carried in the application file. For example, the "@Test" annotation in the header of the application may be a UTDD test program. In an application carrying preset suffix identification information, multiple test cases or test programs may be extracted based on the above-mentioned preset identification information.

[0057] Step S302, obtaining the target program, test strategy identifier and test parameter value of each test case to establish a case value library.

[0058] The target program may be an application or application name tested by the test case, the test strategy identifier may be a program method name in the target program, and the test parameter value may be a parameter value of the program method input with the test strategy identifier. A case value library may be established through the target program, test strategy identifier and test parameter value of each test case.

[0059] The following is a detailed description of how to obtain the target program, test strategy identifier and test parameter values ​​of the test case.

[0060] Optionally, the step of obtaining the target program, test strategy identifier and test parameter value for each test case includes: obtaining a software engineering reflection mechanism for the application to be tested; using the software engineering reflection mechanism to separate each test case in the case value library, and obtaining the target program, test strategy identifier and test parameter value in each test case.

[0061] Among them, software engineering reflection mechanisms can include multiple types. In this embodiment, taking the Java reflection mechanism as an example, the steps of obtaining the target program, test strategy identifier and test parameter value of each test case are schematically illustrated. For example, from the existing UTDD case code (corresponding to the above-mentioned multiple test cases), the Java reflection mechanism is used to extract and separate the target program, test strategy identifier and corresponding test parameter value of each UTDD case.

[0062] The target program, test strategy identifier and test parameter value of each test case are stored in the library to establish a case value library. An example of the case value library is shown in Table 1 below:

[0063] Table 1 Case value library

[0064]

[0065] Step S303: Based on the case value library, the program identifier of the application program to be tested and the test strategy identifier in the case value library are used for matching to obtain the program parameter value of the successfully matched target program.

[0066] The program identifier of the above-mentioned application to be tested can be the program name of the application to be tested and the method name in the application to be tested. The program identifier of the application to be tested can be matched with the target program and test strategy identifiers in the case value library to obtain the program parameter value of the successfully matched target program, wherein the program parameter value can be one or more.

[0067] Step S304, using the program parameter value as a test seed to perform fuzz testing on the application to be tested.

[0068] The test seed may be a seed for injecting parameter values ​​into a fuzzy test of an application to be tested. The test seed may generate a plurality of mutated parameter values ​​that are the same or similar to the test seed data type through seed mutation.

[0069] An optional implementation method, using program parameter values ​​as test seeds to perform fuzz testing on the application to be tested, includes: mutating each program parameter value as a test seed for fuzz testing to obtain multiple test sets, wherein each test set includes at least: a program parameter value and multiple mutated parameter values ​​corresponding to the program parameter value; injecting the parameter values ​​in each test set into the test method corresponding to the test set in turn to perform fuzz testing on the application to be tested.

[0070] Each program parameter value can be used as a test seed for fuzz testing. Each test seed can generate multiple mutated parameter values ​​by mutation. The program parameter value corresponding to each test seed and multiple mutated parameter values ​​can form a test set, which in turn form multiple test sets. The parameter values ​​(program parameter values ​​and multiple mutated parameter values) in each test set are injected into the test method corresponding to the test set in turn to perform fuzz testing on the application to be tested.

[0071] Optionally, the parameter values ​​in each test set are injected into the test method corresponding to the test set in sequence, and the step of performing fuzz testing on the application to be tested also includes: injecting the parameter values ​​in the target test set and multiple variant parameter values ​​in sequence into the target test method corresponding to the target test set for testing, wherein the target test set is one of multiple test sets; within a preset time period, monitoring whether the test process has a program exception, and obtaining a program monitoring status; when the program monitoring status indicates that the test process has a program exception, recording exception data, wherein the exception data includes at least: the number of exceptions, the test identifier of the target test method, and the target test seed; when the program monitoring status indicates that the test process has no program exception, selecting the next test set from the multiple test sets to continue the step of performing fuzz testing on the application to be tested.

[0072] During the fuzz testing of the application to be tested, it is also possible to monitor whether the test process has program exceptions. Taking one of the test sets (that is, the above-mentioned target test set) as an example, the process of monitoring whether the test process has program exceptions during the fuzz testing of the application to be tested is explained. When the parameter values ​​in the test set are input into the program method corresponding to the application to be tested, the test process (that is, the running process of the application to be tested) is monitored for program exceptions and errors within the preset time of continuously running the application to be tested, and the program monitoring status is obtained; when the program monitoring status indicates that the test process has program exceptions or errors, the abnormal data is recorded, wherein the above-mentioned abnormal data at least includes: the number of abnormalities, the test identifier of the target test method, and the target test seed.

[0073] Optionally, the test identifier of the target test method in this embodiment can be of multiple types, for example, the program method name of the input parameter value in the application to be tested. The test identifier can be represented by numbers, characters, letters, or a combination of numbers, characters and letters.

[0074] When the program monitoring status indicates that no program exception occurs in the test process, a next test set is selected from the multiple test sets to continue the step of performing fuzz testing on the application to be tested.

[0075] As an optional implementation of this embodiment, after recording the abnormal data, the method further includes: performing regression testing and data analysis on the application to be tested according to the abnormal data.

[0076] The above-mentioned regression test may refer to the process of performing fuzz testing after fixing defects in the application to be tested, and the above-mentioned data analysis may refer to analyzing the test effectiveness of the test seed or program parameter value on the application to be tested.

[0077] Optionally, the step of performing regression testing on the application to be tested based on the abnormal data includes: performing defect repair on the application to be tested based on the test identifier corresponding to the test set in the abnormal data to obtain a repaired application; injecting parameter values ​​in the target test set into the repaired application in sequence, and continuing to perform fuzz testing on the repaired application.

[0078] According to the test identifier corresponding to the test set in the abnormal data, the location of the defect in the application under test can be found and quickly located, so as to quickly and accurately repair the defect in the application under test and obtain the repaired application. After obtaining the repaired application, the parameter values ​​in the target test set can be injected into the repaired application in turn to monitor whether the repaired application still has the same defect, and then continue to perform fuzz testing on the repaired application.

[0079] Optionally, the step of performing data analysis on the application to be tested based on the abnormal data includes: evaluating the test result score of the target test seed in the abnormal data.

[0080] In the process of data analysis of the application to be tested based on the abnormal data, the test result score of the target test seed in the abnormal data can be evaluated. For example, the test effect of the fuzzy testing of the target test seed can be determined based on the number of anomalies monitored during the fuzz testing of the application to be tested by the target test seed.

[0081] The embodiments of the present invention can extract the test parameter values ​​in the test case as seeds for the initial values ​​of the fuzzy test, so that the fuzzy test can be diffused and mutated within a relatively effective range, thereby avoiding a large number of blind random tests, improving the fuzzy test effect, discovering more potential defects in the system, and improving the quality of the application.

[0082] The present invention is described below in conjunction with another optional embodiment.

[0083] Embodiment 2

[0084] This embodiment proposes another method for testing an application program, and illustrates the method by taking a solution of performing fuzz testing on an application program based on a UTDD case as an example.

[0085] This embodiment connects UTDD cases and fuzzy test cases in series to break the barriers between cases. Without increasing the burden on developers, the test values ​​in the UTDD stock cases are extracted according to the Java reflection mechanism, and used as the seeds of the fuzzy test values. Then, divergent mutation is performed with the seeds as the center, which fully improves the effective coverage of the fuzzy test values. In this embodiment, the focus is on solving the seed validity problem when fuzz testing the target program file, thereby improving the quality of application testing.

[0086] The processing flow of this embodiment is described in detail below:

[0087] Figure 4 is a flowchart of an optional UTDD case-based fuzzy testing method according to an embodiment of the present invention, such as Figure 4 The main processing flow is shown in the figure below:

[0088] 1. Traverse UTDD cases. Traverse all JAVA program files with .java as the suffix in software engineering. If the content in the program file, each corresponding program method has the "@Test" annotation in its header, it is considered to be a UTDD test program, that is, an existing UTDD case, which needs to be parsed and processed. Other files are directly ignored.

[0089] 2. Extraction of UTDD case input parameter values. From the existing UTDD case codes, the Java reflection mechanism is used to extract and separate the target program, test target method name and corresponding test parameter input values ​​of each UTDD case.

[0090] 3. Store test case values. Extract and separate the target program, test target method name, and corresponding test parameter input values ​​of each UTDD case and store them in the database to obtain a case value library.

[0091] 4. Get case values ​​from the library as fuzz test seeds. According to the established case value library, use the program name and method name of the target program for matching, take out the case values ​​of the relevant matching items, inject them into the target test method as seeds, and run the fuzz test.

[0092] 5. Fuzz test case operation. Continue to run the fuzz test case for a period of time (this period can be adjusted according to the test method and system resources) to monitor whether the fuzzy mutation based on the UTDD test value causes program operation anomalies.

[0093] 6. Observe the test results and replace the case values. If no program anomalies are detected after a certain period of time, the next set of test values ​​for the target method will be sequentially extracted from the storage case value library for injection and testing again; if a program anomaly is detected, the target program method will be defect-fixed, and after the repair, the original test values ​​will be used for regression testing again.

[0094] The fuzz test effectiveness value of each set of test values ​​or each fuzz test seed, that is, the number of defects or anomalies found, can also be automatically registered for regression testing and data analysis.

[0095] This embodiment reuses and extracts the test input values ​​of existing UTDD cases and uses them as the initial seed values ​​of fuzzy testing, so that the fuzzy test parameter values ​​can be fuzzily mutated within a more effective range, and the target method can be tested in a more targeted and effective manner. Compared with traditional fuzzy testing, the fuzzy testing of this embodiment can avoid a large number of invalid fuzzy tests and help discover more system defects in a shorter time.

[0096] The present invention is described below in conjunction with another optional embodiment.

[0097] Embodiment 3

[0098] This embodiment also provides an optional testing device for an application program, and each implementation unit included in the testing device corresponds to each step of the above embodiment.

[0099] Figure 5 is a schematic diagram of an optional application program testing device according to an embodiment of the present invention. Figure 5 As shown, the test device includes: a traversal unit 51, a first processing unit 52, a second processing unit 53, and a test unit 54, wherein:

[0100] A traversal unit 51 is used to traverse the application file set to extract the test program carrying the preset identification information to obtain multiple test cases;

[0101] A first processing unit 52 is used to obtain a target program, a test strategy identifier and a test parameter value of each test case to establish a case value library;

[0102] The second processing unit 53 is used to match the program identifier of the application program to be tested with the test strategy identifier in the case value library based on the case value library to obtain the program parameter value of the target program that is successfully matched;

[0103] The testing unit 54 is used to use the program parameter value as a test seed to perform fuzz testing on the application program to be tested.

[0104] The test device for the above-mentioned application can traverse the application file set through the traversal unit 51 to extract the test program carrying the preset identification information to obtain multiple test cases, obtain the target program, test strategy identifier and test parameter value of each test case through the first processing unit 52 to establish a case value library, and use the program identifier of the application to be tested and the test strategy identifier in the case value library to match based on the case value library through the second processing unit 53 to obtain the program parameter value of the target program that successfully matches, and use the program parameter value as a test seed to perform fuzz testing on the application to be tested through the testing unit 54. In this embodiment, by extracting test cases from the application file set, using the test parameter value of the test case as the seed of the fuzzy test, and performing fuzz testing on the application to be tested, the ability to hit the potential defects of the application is improved, and the purpose of controlling the parameter value selection direction of the fuzzy test is achieved, thereby solving the technical problem of low ability to hit the potential defects of the application by performing fuzz testing on the application by randomly selecting parameter values ​​in the related art.

[0105] Optionally, the first processing unit includes: an acquisition subunit, used to obtain a software engineering reflection mechanism for the application to be tested; and a separation subunit, used to use the software engineering reflection mechanism to separate each test case in the case value library to obtain the target program, test strategy identifier and test parameter value in each test case.

[0106] Optionally, the test unit includes: a mutation sub-unit, used to mutate each program parameter value as a test seed for fuzz testing, to obtain multiple test sets, wherein each test set includes at least: a program parameter value and multiple mutated parameter values ​​corresponding to the program parameter value; a test sub-unit, used to inject the parameter values ​​in each test set into the test method corresponding to the test set in sequence, to perform fuzz testing on the application to be tested.

[0107] Optionally, the test sub-unit includes: a first test module, used to inject parameter values ​​and multiple variant parameter values ​​in a target test set into a target test method corresponding to the target test set for testing in sequence, wherein the target test set is one of multiple test sets; a detection module, used to monitor whether a program exception occurs in the test process within a preset time period, and obtain a program monitoring status; a recording module, used to record exception data when the program monitoring status indicates that a program exception occurs in the test process, wherein the exception data includes at least: the number of exceptions, the test identifier of the target test method, and the target test seed; a second test module, used to select the next test set from multiple test sets when the program monitoring status indicates that no program exception occurs in the test process, so as to continue to execute the steps of fuzz testing the application to be tested.

[0108] Optionally, the test subunit further includes: a processing module, which is used to perform regression testing and data analysis on the application to be tested according to the abnormal data after recording the abnormal data.

[0109] Optionally, the processing module includes: a repair sub-module, used to repair defects in the application under test according to the test identifier corresponding to the test set in the abnormal data, and obtain a repaired application; a testing sub-module, used to inject parameter values ​​in the target test set into the repair application in sequence, and continue to perform fuzz testing on the repaired application.

[0110] Optionally, the processing module further includes: an evaluation submodule, used to evaluate the test result score of the target test seed in the abnormal data.

[0111] The testing device for the above-mentioned application may also include a processor and a memory. The above-mentioned traversal unit 51, first processing unit 52, second processing unit 53, testing unit 54, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to implement corresponding functions.

[0112] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters are adjusted to extract test cases from the application file set, and the test parameter values ​​of the test cases are used as fuzz test seeds to perform fuzz testing on the application to be tested.

[0113] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0114] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the test method of any of the above-mentioned application programs by executing the executable instructions.

[0115] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a testing method for any of the above-mentioned application programs.

[0116] Figure 6 1 is a hardware structure block diagram of an electronic device (or mobile device) according to a method for testing an application program in an embodiment of the present invention. Figure 6As shown, the electronic device may include one or more (602a, 602b, ..., 602n are used to illustrate) processors 602 (the processor 602 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 604 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 6 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 6 More or fewer components as shown, or with Figure 6 Different configurations are shown.

[0117] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0118] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0119] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0120] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0121] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0122] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0123] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for testing an application, characterized in that: include: Traversing the application file set to extract the test program carrying the preset identification information to obtain multiple test cases; Obtaining a target program, a test strategy identifier, and a test parameter value for each of the test cases to establish a case value library; Based on the case value library, matching is performed using the program identifier of the application program to be tested and the test strategy identifier in the case value library to obtain the program parameter value of the successfully matched target program; The program parameter value is used as a test seed to perform a fuzzy test on the application to be tested.

2. The testing method according to claim 1, characterized in that: The step of obtaining the target program, test strategy identifier and test parameter value of each test case comprises: Acquire a software engineering reflection mechanism for the application to be tested; The software engineering reflection mechanism is adopted to separate each of the test cases in the case value library, and the target program, the test strategy identifier and the test parameter value in each of the test cases are obtained.

3. The testing method according to claim 1, characterized in that: The step of using the program parameter value as a test seed to perform a fuzzy test on the application to be tested comprises: Mutating each of the program parameter values ​​as a test seed of the fuzzy test to obtain a plurality of test sets, wherein each of the test sets at least includes: the program parameter value and a plurality of mutated parameter values ​​corresponding to the program parameter value; The parameter values ​​in each of the test sets are sequentially injected into the test method corresponding to the test set, and fuzz testing is performed on the application to be tested.

4. The testing method according to claim 3, characterized in that: The step of injecting the parameter values ​​in each of the test sets into the test method corresponding to the test set in sequence and performing fuzz testing on the application to be tested also includes: Injecting parameter values ​​and multiple variant parameter values ​​in a target test set into a target test method corresponding to the target test set in sequence for testing, wherein the target test set is one of the multiple test sets; Within the preset time, monitor whether the test process has any program anomalies and obtain the program monitoring status; When the program monitoring status indicates that a program exception occurs in the test process, abnormal data is recorded, wherein the abnormal data at least includes: an abnormal number, a test identifier of the target test method, and a target test seed; When the program monitoring status indicates that no program anomaly occurs in the test process, a next test set is selected from the multiple test sets to continue the step of performing fuzz testing on the application to be tested.

5. The testing method according to claim 4, characterized in that: After recording the abnormal data, it also includes: Regression testing and data analysis are performed on the application to be tested according to the abnormal data.

6. The testing method according to claim 5, characterized in that: The step of performing regression testing on the application to be tested according to the abnormal data comprises: According to the test identifier corresponding to the test set in the abnormal data, the defect of the application to be tested is repaired to obtain a repaired application; The parameter values ​​in the target test set are injected into the repair application in sequence, and the repair application is continuously subjected to fuzz testing.

7. The testing method according to claim 5, characterized in that: The step of performing data analysis on the application to be tested according to the abnormal data comprises: A test result score of a target test seed in the abnormal data is evaluated.

8. A testing device for an application, characterized in that: include: A traversal unit, used for traversing the application file set to extract the test program carrying the preset identification information to obtain a plurality of test cases; A first processing unit, configured to obtain a target program, a test strategy identifier, and a test parameter value of each test case to establish a case value library; A second processing unit is used to match the program identifier of the application program to be tested with the test strategy identifier in the case value library based on the case value library to obtain the program parameter value of the target program that is successfully matched; The testing unit is used to use the program parameter value as a test seed to perform a fuzzy test on the application to be tested.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the test method of the application program according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the test method of the application program described in any one of claims 1 to 7.

Images