A Fuzz Testing Method for RestFul API

Abstract

The invention discloses a fuzzy testing method for RestFul API, comprising the following steps: S1: parsing the interface document of the input Web API; S2: using the semantic matching analysis method to respectively construct the resource dependency tree; using the semantic matching analysis technology to respectively construct the data dependency tree; S3: based on the resource dependency tree and the data dependency tree, the interface request sequence is obtained after traversing by the tree depth first traversal algorithm; S4: according to the request interface sequence, a test case is generated for each interface in turn to test the corresponding interface; S5: constructing an HTTP request based on the test sample generated by S4 and sending it to the target program to perform fuzzy testing on it; S6: based on the HTTP request generated by S5, obtaining its response result; S7: repeating S4 to test the interface until the test budget time has been exhausted or the coverage of all interfaces reaches the expected level and the fuzzy test is terminated. The invention effectively improves the effectiveness and efficiency of interface fuzzy testing.

Description

Technical Field

[0001] The invention belongs to the technical field of software testing, and in particular relates to a fuzzy testing method for RestFul API. Background Art

[0002] With the continuous development of information technology, the Internet and software products are increasingly developing towards an open and shared model. Open sharing of information through application programming interfaces (APIs) has become an important trend in the development of the Internet and software products. Today, most Web services are programmatically accessible to users through REST APIs. Since Web services carry a large amount of information business of enterprises, the reliability and security of Web service interfaces have become increasingly important. Interface fuzzing is one of the main means of Web interface security testing. However, in actual use, due to the complex dependencies between interfaces and the complexity of the internal logic of the interfaces, Web application interface fuzzing has problems such as low effectiveness of generated test cases, excessive use case scale, and low use case coverage.

[0003] The defects of the prior art are:

[0004] 1. The existing Restful API interface fuzzing test does not consider the resource dependency and data dependency between interfaces. Each interface is tested directly and independently, resulting in many tests failing to meet the business logic, and thus the test efficiency is low;

[0005] 2. When performing mutation, the existing methods only mutate the parameter values ​​but not the structure of the composite parameters, resulting in incomplete testing;

[0006] 3. When conducting tests, existing methods also use code line coverage as a guiding evaluation indicator, which will cause two deficiencies: a. Some branches with fewer code lines are difficult to detect, resulting in low coverage; b. Using only the code line coverage indicator is relatively simple and cannot detect the coverage of different branch structures of the code, thus failing to guide the test engine to use different refined mutation strategies for different targets. Summary of the invention

[0007] The purpose of the present invention is to overcome the problems of the prior art and disclose a fuzzy testing method for RestFul API. The method of the present invention innovatively introduces interface dependency analysis technology to solve the dependency relationship of different APIs in Web services, and at the same time generates high-quality mutation samples by designing an efficient mutation algorithm, thereby effectively improving the effectiveness and efficiency of interface fuzz testing.

[0008] The object of the present invention is achieved through the following technical solutions:

[0009] A fuzz testing method for RestFul API, the fuzz testing method comprising the following steps:

[0010] S1: Parse the input Web API interface document to obtain the request path, request method, request parameters and request response information of each request interface;

[0011] S2: Based on the request paths and request parameters of each interface obtained in S1, a resource dependency tree is constructed using a semantic matching analysis method.

[0012] Based on the request parameters and request responses of each interface obtained in S1, the data dependency trees are constructed respectively using semantic matching analysis technology;

[0013] S3: Based on the resource dependency tree and data dependency tree obtained in S2, the interface request sequence is obtained by traversing the tree depth-first traversal algorithm;

[0014] S4: Generate test cases for each interface in turn according to the order of requesting interfaces to test the corresponding interface;

[0015] In the process of test case generation,

[0016] If there is no initial seed, the corresponding parameter value is randomly generated according to the parameter definition of each interface, and then the initial seed is constructed;

[0017] If a seed already exists, a sample is taken from the existing seed with a preset probability each time, and other parameters are mutated to generate a new test seed;

[0018] S5: construct an HTTP request based on the test sample generated in S4 and send it to the target program to perform fuzz testing on it;

[0019] S6: Based on the HTTP request generated by S5, obtain its response result. If the response status code is 200OK, store the request parameters or request parameter values ​​of the request in the external resource dependency pool for direct use by other dependent interfaces in the future.

[0020] A probe is inserted into the target program to collect the execution status of the test samples, and then the fitness values ​​of the test samples are collected, and the seed samples are updated according to the fitness values;

[0021] S7: Repeat S4 to test the interface until the test budget time is exhausted or the coverage of all interfaces reaches the expected level, and then end the fuzz testing.

[0022] According to a preferred implementation, in step S2, a resource dependency tree is constructed in the following manner:

[0023] First, split all paths with the / symbol to get the elements of each path, then sort them according to the length of the elements, and create a tree with the node with the least path elements as the root node;

[0024] For other nodes, the path prefix obtained after removing the last element is compared with the path with fewer nodes than the path. If a match is successful, a new tree node is created with the node as its child node.

[0025] If a path node cannot be matched successfully when compared with all nodes with fewer elements than this node, a new tree is created with this node as a new root node.

[0026] According to a preferred embodiment, after the tree is created, all node request methods and the request order of each node are determined based on a preset semantic template; and after determining the request method of each node and the request order of each node, the parameter dependency relationship between each node request method and its sibling nodes and parent node is determined.

[0027] According to a preferred embodiment, when determining parameter dependency, if the parameter is of a native type, the following method is used for confirmation: when the parameter name of a request and the name of the parameter of a return response of a request are exactly the same, it is determined that the two are dependent; after ignoring the case of the two strings and removing special symbols, if they are the same, then the two are dependent; use the Porter stem algorithm to compare the stems of the two strings to determine whether they represent parameters with the same semantics, and if they are the same, then the two are dependent.

[0028] According to a preferred embodiment, when determining parameter dependency, if the parameter is a composite type, the following method is used to confirm: determine whether the two parameters have a dependency based on whether they have the same schema; if the schemas of the two parameters are the same, it is determined that the parameters are dependent; when the schema of a parameter of a request is the same as the schema of the parameter returned in the response, it is determined that the two are dependent.

[0029] According to a preferred embodiment, in step S4, in the process of generating seed request parameters, for parameters with dependencies, the dependency values ​​are directly retrieved from the external resource dependency pool according to the dependencies for filling; for non-dependent tests, random initialization or mutation is used to generate parameter values.

[0030] According to a preferred embodiment, in step S6, the sample evaluation is completed by inserting probes at all branches of the target program and then calculating the branch distance between the path executed by the test sample and the target branch.

[0031] According to a preferred embodiment, in step S6, when the seed samples are updated, a multi-objective optimization algorithm is used to determine whether a new test sample needs to be added to the seed queue.

[0032] According to a preferred embodiment, in step S6, the fitness value is compared with other sample values ​​in the existing seed queue to determine whether the new test case needs to be added to the seed sample.

[0033] The aforementioned main scheme of the present invention and its further options can be freely combined to form multiple schemes, all of which are schemes that can be adopted and claimed for protection by the present invention. After understanding the scheme of the present invention, those skilled in the art can understand that there are multiple combinations based on the prior art and common knowledge, all of which are technical schemes to be protected by the present invention, and they are not exhaustively listed here.

[0034] Beneficial effects of the present invention:

[0035] The method of the present invention first generates the request sequence of the interface by analyzing the dependency relationship between interfaces (including resource dependency and data dependency), and then generates test cases to perform fuzzy testing on the interface after mutating the interface using a strategy mutation algorithm. In order to further improve the effectiveness of the test, the test engine is guided to generate higher quality test case data by collecting the fitness value (a metric used to evaluate the quality of the test case) of the test case running on the tested program to improve the effectiveness of the test.

[0036] The method of the present invention innovatively introduces interface dependency analysis technology to solve the dependency relationship between different APIs in Web services, and at the same time generates high-quality mutation samples by designing an efficient mutation algorithm, thereby effectively improving the effectiveness and efficiency of Fuzzing testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the process of forming a resource dependency tree of the present invention;

[0038] Figure 2 is based on Figure 1 Schematic diagram of the structure of the dependency tree obtained by the process shown. DETAILED DESCRIPTION

[0039] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0040] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0041] The present invention discloses a fuzzy testing method for RestFul API. The testing process is divided into three stages, which are as follows.

[0042] 1. Interface dependency analysis phase: Since there are resource dependencies and data dependencies between different interfaces in Web services, this phase mainly parses the interface documents of the target application to obtain the dependencies between interfaces and generates the corresponding interface request sequence based on the dependencies.

[0043] Specifically, in Web services, the RESTFul API is stateful, and accessing or operating a specific resource usually depends on the state of the related resource. For example, the access of a resource by the GET or DELETE method first depends on the existence of the resource, so the prerequisite for successfully executing the GET and DELETE operations is to first call POST to create the resource.

[0044] Therefore, when performing Web API fuzzing, we first need to establish resource dependencies between different operations. And generate the corresponding request operation sequence based on the dependency between resources to ensure that the request operation is in line with semantic logic. Secondly, when performing fuzzing requests on Web API, in addition to resource dependencies, there are also data dependencies. For example, the parameter value of a certain HTTP request depends on the response value returned by the previous request. Therefore, we need to make the request correctly, analyze the dependency between interfaces, and generate a specific request sequence based on the interface dependency.

[0045] 2. Test case generation and mutation phase. This phase mainly performs fuzzing tests on the interfaces in sequence according to the interface execution order. It generates corresponding initial seed test cases based on the definition document of each interface to test the interface, and continuously mutates the seeds during the test process to generate new test cases for testing the interface.

[0046] 3. Test seed update phase. During the test process, in order to improve the effectiveness of the test, the present invention adopts a gray box test method based on fitness value guidance, which adds excellent seeds generated during the test process to the seed pool through an evolutionary algorithm, and generates more high-quality seeds through mutation of these seeds, and then performs fuzzing test on the target program, thereby effectively improving the efficiency of the test.

[0047] Specifically, a probe is added to a specific location of the target program by program instrumentation, and then the execution information of each request in the target program is collected and the fitness value of each request is calculated. Finally, a decision is made based on the fitness value score to determine whether it needs to be updated to the seed group.

[0048] The fuzzy testing method for RestFul API of the present invention specifically includes the following steps.

[0049] Step 1: The fuzz testing engine first parses the input Web API interface document (Swagger document) to obtain the request path, request method, request parameter format, and response information format of all request interfaces.

[0050] In step 1, Swagger analysis uses the parser of the standard io.swagger.v3.parser.OpenAPIV3Parser library to implement parsing and obtain the parsed data. The parsed swagger data contains swagger's info, path, component, swagger and other information (OpenAPI object).

[0051] Step 2: Based on the request path and request parameters of each interface obtained in step 1, a resource dependency tree is constructed using semantic matching analysis technology. Based on the request parameters and request response of each interface obtained in step 1, a data dependency tree is constructed using semantic matching analysis technology.

[0052] In step 2, in the process of building the resource dependency tree, three goals need to be achieved: determine the request order of different paths; determine the request order of different methods under the same path; determine the specific resource dependency relationship between each method and the related methods in the sibling nodes or parent nodes.

[0053] To describe the details clearly, a Web service with four interfaces (" / products", " / products / {id}", " / products / {id} / {respository}", " / products / {id} / {respository} / projectname") is used as an example for explanation.

[0054] Specifically, using Figure 1 The dependency tree is constructed in the following way: first, all paths are split with the / symbol to obtain the elements of each path, and then the elements are sorted according to their lengths. The node with the least path elements is used as the root node to create a tree. For other nodes, the path prefix obtained after removing the last element is compared with the paths with fewer nodes than the path. If a match is successful, a new tree node is created with the node as its child. If a path node cannot be matched successfully when compared with all nodes with fewer elements than the node, a new tree is created with the node as a new root node. The final dependency tree is as follows: Figure 2 shown.

[0055] After the tree is created, each node has different request methods (such as GET, DELEETE, PATCH and POST, etc.), and different request methods have different semantics. For example, POST generally creates resources, GET obtains resources, PATCH updates resources, and DELETE deletes resources. Therefore, when making requests to different methods of a node, it is necessary to determine the request order according to the semantics of the method. The present invention uses the following semantic template to determine the request order of different methods.

[0056] Semantic template: POST->GET->HEAD->OPTIONS->TRACE->PUT->PATCH->DELETE.

[0057] Semantic explanation: All methods are executed in the order of POST, GET, HEAD, OPTIONS, TRACE, PUT, PATCH, and DELETE.

[0058] After determining the request order of all methods and all nodes, you also need to determine the parameter dependencies between each method and its sibling nodes and parent nodes, and specify the dependencies between specific parameters. For example, in the above example, executing the GET method of / products / {id} / needs to rely on the parameter {userid} of the POST method in the / products / path. When determining parameter dependencies, two methods are used, as follows:

[0059] 1) If the parameter is a primitive type (such as String, Number, Boolean, time type (Datetime, Date, Time), etc.), use the following method to determine whether the node has dependencies:

[0060] If the parameter name of a request is exactly the same as the parameter name of a response returned by the request, it is determined that there is a dependency between the two.

[0061] After ignoring upper and lower case characters and removing special symbols such as underscores, if two strings are the same, there is a dependency between the two. The present invention refers to this method as IS (Ignore Special).

[0062] The Porter Steamming algorithm is used to compare the stems of two strings to determine whether they represent the same semantic parameter. This algorithm is referred to as PS in the following name.

[0063] 2) If the parameter is a composite type (object), determine whether the two parameters have the same schema to determine whether they are dependent. If the schemas of the two parameters are the same, it is determined that the parameters are dependent: If the schema of a request parameter is the same as the schema of the parameter returned in the response, it is determined that the two are dependent.

[0064] In step 2, data dependency means that the parameter value of a certain HTTP request depends on the response value returned by the previous request. The construction relationship of its data dependency graph is the same as the construction process of the resource dependency source graph, except that the data dependency construction is compared with the responses returned by other node methods. The comparison method used is the same as the comparison method used for resource dependency, so it will not be repeated here.

[0065] Step 3: Based on the resource dependency tree and data dependency tree obtained in step 2, the interface request sequence is obtained by traversing the tree using a depth-first traversal (DFS) algorithm.

[0066] Step 4: Generate test cases for each interface in turn according to the order of requesting the interfaces and test them.

[0067] In the test seed case generation and mutation steps, there are two test sample generation strategies: one is to randomly generate corresponding parameter values ​​according to the parameter definition of each interface if there is no initial seed, and then build the initial seed. The other is to generate a new test seed by sampling a sample from the existing seed with a certain probability each time and mutating other parameters.

[0068] In the process of generating seed request parameters, for parameters with dependencies, the dependency values ​​are directly retrieved from the external resource dependency pool according to the dependencies, instead of generating the request parameter values ​​through random initialization or mutation.

[0069] In step 4, in the process of generating test seeds based on the random initialization method, the request parameters are classified into two types: basic data types and composite types, where the basic types include Number type data, String type data, and Boolean.

[0070] For Number, the initialization value is generated by randomly generating integers or floating-point numbers. During the generation process, if the parameter constraints are specified in the interface document, such as the maximum or minimum value range of the parameter, the random number is taken within the specific range. If the gene is of String type, some characters are randomly selected from the printable ASCII code range to form a string of length n for initialization, where n is a random value in the range of [1,255]; if the data type is Boolean, a value is randomly selected from the set (True, False) for initialization.

[0071] Composite types, such as JSON structures or array structures, are initialized in the same way. The difference is that the sub-elements of the composite elements are initialized. The specific process will not be repeated here.

[0072] In the process of sampling mutation, a sample is first selected from the existing seeds with a certain probability, and then its parameters are mutated. During mutation, for Number type data, the random value is also generated, and for string type parameters, new request parameters are generated through three mutation methods: adding, deleting, and modifying.

[0073] The specific operations are as follows: Add, randomly insert a string of length n into the string to form a new string, where n is a random number in the range [1,255]; Delete, randomly delete some strings in the string, and find a substring at a random position in the string to delete; Change, replace some substrings in the string with other substrings of the same length; For the type parameters of the json structure, in addition to mutating its parameter value, it also mutates it, and specifically defines four operators: 1. Drop, randomly delete a sub-element; 2. Select, randomly select a sub-element; 3. Duplicate, randomly copy a sub-element; 4. Type, randomly change the data type of a sub-element.

[0074] Step 5: Based on the test sample generated in step 4, an HTTP request is constructed and sent to the target program to perform fuzz testing on it.

[0075] Step 6, based on the HTTP request generated in step 5, obtain its response result. If the response status code is 200OK, store the request parameters or request parameter values ​​of the request in the external resource dependency pool for direct use by other dependent interfaces.

[0076] Insert probes into the target program to collect the execution status of test samples, and then collect the fitness values ​​of the test samples and feed them back to the mutation engine. The mutation engine updates the seed samples according to the fitness values.

[0077] In step 6, after the target program to be tested is instrumented, a probe is inserted into a specific position of the target program, and then the quality of the test sample is evaluated by the fitness value of the probe target sample.

[0078] In the present invention, the quality of a sample is evaluated by inserting probes at all branches of the target program and then calculating the branch distance h between the execution path of the test sample and the target branch.

[0079] In step 6, each branch target maintains a seed queue, and the upper limit of the number of seeds contained in the seed queue is 30. During the update, the multi-objective optimization algorithm (MIO) is used to decide whether to add new test samples to the seed queue. Specifically, the fitness value is compared with other sample values ​​in the existing seed queue to decide whether to add new test cases to the seed samples.

[0080] The specific rules are as follows: 1. If the moderation value is zero, that is, h_{k}=0, then no matter what the following conditions are, it will not be added to the seed cluster; 2. If h_{k}=1, it means that the branch target has been covered, then the sample is added to the test set T_{k} of the corresponding target, and the group is reduced to one individual and no longer expanded (that is, always keep |T_{k}|=1|). In this case, the existing test samples of T_{k} will be replaced only when a new test case has a better heuristic score in other targets besides the target value (for example, the branch coverage heuristic score of the new test is the same as that of the existing sample, but its code coverage heuristic score will be higher); if the value of h_{k} is between 0 and 1, then if the target test set is not full (that is, |T_{k}|<30|), then the new test case is added to the target set; if the test samples in the test case set of the target branch are full (that is, |T_{k}|=30|), then the sample with the worst heuristic score in the test set will be replaced.

[0081] Step 7: Continue to repeat step 4 to test the interface until the test budget time is exhausted or the coverage of all interfaces reaches the expected level, then exit the fuzz test.

[0082] The method of the present invention first generates the request sequence of the interface by analyzing the dependency relationship between interfaces (including resource dependency and data dependency), and then generates test cases to perform fuzzy testing on the interface after mutating the interface using a strategy mutation algorithm. In order to further improve the effectiveness of the test, the test engine is guided to generate higher quality test case data by collecting the fitness value (a metric used to evaluate the quality of the test case) of the test case running on the tested program to improve the effectiveness of the test.

[0083] The method of the present invention innovatively introduces interface dependency analysis technology to solve the dependency relationship between different APIs in Web services, and at the same time generates high-quality mutation samples by designing an efficient mutation algorithm, thereby effectively improving the effectiveness and efficiency of Fuzzing testing.

[0084] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A fuzzy testing method for RestFul API, characterized in that: The fuzz testing method comprises the following steps: S1: Parse the input Web API interface document to obtain the request path, request method, request parameters and request response information of each request interface; S2: Based on the request paths and request parameters of each interface obtained in S1, a resource dependency tree is constructed using a semantic matching analysis method. Based on the request parameters and request responses of each interface obtained in S1, the data dependency trees are constructed respectively using semantic matching analysis technology; S3: Based on the resource dependency tree and data dependency tree obtained in S2, the interface request sequence is obtained by traversing the tree depth-first traversal algorithm; S4: Generate test cases for each interface in turn according to the order of requesting interfaces to test the corresponding interface; In the process of test case generation, If there is no initial seed, the corresponding parameter value is randomly generated according to the parameter definition of each interface, and then the initial seed is constructed; If a seed already exists, a sample is taken from the existing seed with a preset probability each time, and other parameters are mutated to generate a new test seed; S5: construct an HTTP request based on the test sample generated in S4 and send it to the target program to perform fuzz testing on it; S6: Based on the HTTP request generated by S5, obtain its response result. If the response status code is 200OK, store the request parameters or request parameter values ​​of the request in the external resource dependency pool for direct use by other dependent interfaces in the future. A probe is inserted into the target program to collect the execution status of the test samples, and then the fitness values ​​of the test samples are collected, and the seed samples are updated according to the fitness values; In step S6, the sample evaluation is completed by inserting probes at all branches of the target program and then calculating the branch distance between the path executed by the test sample and the target branch; In step S6, when updating the seed samples, a multi-objective optimization algorithm is used to determine whether a new test sample needs to be added to the seed queue; In step S6, the fitness value is compared with other sample values ​​in the existing seed queue to determine whether the new test case needs to be added to the seed sample; S7: Repeat S4 to test the interface until the test budget time is exhausted or the coverage of all interfaces reaches the expected level, and then end the fuzz testing.

2. The fuzzy testing method according to claim 1, characterized in that: In step S2, the resource dependency tree is constructed in the following manner: First, split all paths with the / symbol to get the elements of each path, then sort them according to the length of the elements, and create a tree with the node with the least path elements as the root node; For other nodes, the path prefix obtained after removing the last element is compared with the path with fewer nodes than the path. If a match is successful, a new tree node is created with the node as its child node. If a path node cannot be matched successfully when compared with all nodes with fewer elements than this node, a new tree is created with this node as a new root node.

3. The fuzzy testing method according to claim 2, characterized in that: After the tree is created, all the request methods for each node and the request order for each node are determined based on the preset semantic template; After determining the request method of each node and the request order of each node, the parameter dependency between each node request method and the sibling nodes and the parent node is determined.

4. The fuzzy testing method according to claim 3, characterized in that: When determining parameter dependencies, if the parameter is a native type, use the following method to confirm: When the parameter name of a request is exactly the same as the parameter name of the response returned by a request, it is determined that there is a dependency between the two. After ignoring the case of the two strings and removing special symbols, if they are the same, there is a dependency between the two; The Porter stem algorithm is used to compare the stems of two strings to determine whether they represent the same semantic parameters. If they are the same, there is a dependency between the two.

5. The fuzzy testing method according to claim 3, characterized in that: When determining parameter dependencies, if the parameter is of a composite type, use the following method to confirm: Determine whether the two parameters have a dependency based on whether they have the same schema. If the two parameters have the same schema, it is determined that the parameters are dependent. When the schema of a request parameter is the same as the schema of the parameter returned in the response, it is determined that there is a dependency between the two.

6. The fuzzy testing method according to claim 1, characterized in that: In step S4, in the process of generating seed request parameters, For parameters with dependencies, the dependency values ​​are directly retrieved from the external resource dependency pool based on the dependencies for filling; for non-dependent tests, random initialization or mutation is used to generate parameter values.

Images