Generation Device, Generation Method, and Program

Abstract

To provide a generation device, when generating a test code from API definition information, correcting the test code into a further appropriate test code by information not described in the API definition information.SOLUTION: A generation device includes an analysis part, a test scenario generation part and a test code generation part. The analysis part analyzes API (Application Programming Interface) definition information for defining a structure of the API and generates structure analysis data of the API. The test scenario generation part generates test scenario data of the API from test setting information including a test type of the API and a test item for each test type, and the structure analysis data, and corrects the test scenario data according to correction information to generate corrected test scenario data. The test code generation part generates a test code from the corrected test scenario data.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] Embodiments of the present invention relate to a generation device, a generation method, and a program.

Background Art

[0002] Techniques for generating test code from documents such as API (Application Programming Interface) definition files have been conventionally known. For example, techniques for automatically generating test code for Web APIs from API definition information such as OpenAPI definition files have been conventionally known.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technology, when generating test code from API definition information, it has been difficult to correct the test code into a more appropriate test code due to information not described in the API definition information.

Means for Solving the Problems

[0005] The generation apparatus according to the embodiment includes an analysis unit, a test scenario generation unit, and a test code generation unit. The analysis unit analyzes API definition information that defines the structure of an API (Application Programming Interface) and generates structure analysis data of the API. The test scenario generation unit generates test scenario data of the API from test setting information including a test type of the API and test items for each test type, and the structure analysis data, and generates corrected test scenario data by correcting the test scenario data according to correction information. The test code generation unit generates a test code from the corrected test scenario data.

Brief Description of Drawings

[0006]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6A

Figure 6B

Figure 6C

Figure 6D

Figure 7A

Figure 7B

Figure 7C

Figure 7D

Figure 8A

Figure 8B

Figure 8C

Figure 8D

Figure 9A

Figure 9B

Figure 9C

Figure 9D

Figure 10A

Figure 10B

Figure 10C

Figure 10D

Figure 11

Figure 12A

Figure 12B

Figure 12C

Figure 12D

Figure 13A

Figure 13B

Figure 13C

Figure 13D

Figure 14A

Figure 14B

Figure 14C

Figure 14D

Figure 15A

Figure 15B

Figure 15C

Figure 15D

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Embodiments for Carrying Out the Invention

[0007] Embodiments of a generation device, a generation method, and a program will be described in detail below with reference to the accompanying drawings.

[0008] For example, when automatically generating test code for a Web API from an OpenAPI document, there is information that cannot be read from the OpenAPI document.

[0009] For example, values registered in a test target DB (database) may be expected as inputs to the Web API. Specifically, assume that an API for resetting a user's password is defined in the OpenAPI document. The API has a property named email, and its Type is string and Format is email. In this case, even if a test is created by inputting a string that matches the Type and Format, such as sample@example.com, a successful response will not be returned. If a successful response is expected, the email registered in the test target DB needs to be input.

[0010] Also, for example, there may be a case where the OpenAPI document and the reference unit are different. Specifically, in the OpenAPI document, the maximum and minimum values of a string can be described, and this unit is the number of characters. On the other hand, assume that in the server implementation, the limit of the maximum value of a string is handled based on Byte. If only half-width alphanumeric characters are handled using utf-8, there is no particular inconvenience because the number of characters and the number of Bytes match. However, half-width katakana is 3 Bytes per character, full-width Japanese is 3 - 6 Bytes per character, and in the case of combining characters, it is even possible to create a character with an infinite number of Bytes. Therefore, even if there is an upper limit on the character size, there may be a case where the upper limit number of characters cannot be described in the OpenAPI document.

[0011] Thus, due to information that cannot be read from the OpenAPI document, for example, there is a problem that the response expected by the automatically generated test code is different from the actual response. Also, for example, there is a problem that test code that tests the input of the automatically generated test with the input of the expected content cannot be automatically generated from the OpenAPI document. Also, for example, despite limitations such as the range of variable values, there is a problem that boundary value testing cannot be performed because it is not described in the OpenAPI document.

[0012] These problems may also be solved by adding information to the OpenAPI document to be loaded. Specifically, boundary value testing can be created by writing the maximum number of characters as 1 character per 1 byte in the OpenAPI document. Also, by describing the actually registered email as an example, a test of the correct input value can be created. However, it is better to avoid writing descriptions that may mislead users or descriptions that may leak internal information in the OpenAPI document, which is not intended for testing but for publishing specifications.

[0013] Also, when the information of the actually registered email cannot be obtained, etc., it may be possible to handle it by replacing the response, but this cannot be dealt with by rewriting the OpenAPI document.

[0014] For these reasons, there is a need for a means to overwrite (correct) the test code without rewriting the API definition information such as the OpenAPI document.

[0015] Hereinafter, embodiments of a generation device that can correct test code into more appropriate test code by information not described in the API definition information when generating test code from the API definition information will be described.

[0016] First, the development flow of the WebAPI of the embodiment will be described.

[0017] FIG. 1 is a diagram for explaining the development flow of the Web API according to the embodiment. FIG. 1 shows an example of the development flow of the Web API of the cloud system of an air conditioner manufacturer that provides infrastructure services. Generally, the development of a Web API includes a Web API design process, a Web API implementation process, a Web API test code implementation process, and a Web API test process. And after the execution of the Web API test, if there is no problem with the quality of the Web API, the Web API is released.

[0018] A large number of patterns need to be prepared for the test code of the Web API, which has been a burden on test developers. For example, a conventional test code automatic generation system can only generate normal system tests of the Web API that can be confirmed from the Web API definition file and some abnormal system tests. Also, when generating test code from documents, it was difficult to correct the test code into more appropriate test code due to information not described in the documents.

[0019] The generation device according to the embodiment relates to, for example, a device that automatically generates test code for a Web API in Web API development. According to the generation device of the embodiment described below, when generating test code from a document (in the example of the embodiment, an API definition file), the test code can be corrected into more appropriate test code due to information not described in the document. Thereby, the implementation process of more appropriate test code can be automated, so that the time taken for test code development can be significantly shortened.

[0020] Furthermore, by loading the test code generated by the generation device according to the embodiment into a commercially available or free test automatic execution tool (for example, Tavern, etc.), the test process can also be shortened. Thereby, the work with a large number of man-hours in software development, such as test code creation, is automated, so that the burden on test code developers can be reduced.

[0021] [Example of Functional Configuration] FIG. 2 is a diagram showing an example of the functional configuration of the generation device 10 of the embodiment and an example of the test environment 100.

[0022] The generation device 10 generates a test code 106 for testing the test target 30. The test target 30 is, for example, a Web API (FIG. 1) of a cloud system that provides an infrastructure service.

[0023] The test execution device 20 executes the test code 106 generated by the generation device 10 and outputs a test result 107. The test result 107 is, for example, referred to by a user and used for checking the operation of the test target 30.

[0024] The generation device 10 of the embodiment includes an analysis unit 1, a test scenario generation unit 2, and a test code generation unit 4.

[0025] The analysis unit 1 analyzes an API definition file 101 that defines the structure of an API and generates API structure analysis data 102.

[0026] The API definition file 101 is, for example, an API document described in an API description language such as OpenAPI, API Blueprint, and RAML (RESTful API Modeling Language). The API definition file 101 is, for example, prepared in advance by a user.

[0027] The structure analysis data 102 is data indicating the structure of the API specified from the API definition file 101.

[0028] The test scenario generation unit 2 generates API test scenario data 105 from the test setting file 103 and the structure analysis data 102, and generates corrected test scenario data 105 by correcting the test scenario data 105 according to the correction information 104. That is, the test scenario generation unit 2 includes a correction unit 3 that corrects the test scenario data 105. For example, the correction unit 3 corrects the test scenario data 105 generated from the structure analysis data 102 and the test setting file 103 into test scenario data 105 that conforms to the actual test. Note that the correction unit 3 may be included in the test code generation unit 4.

[0029] The test setting file 103 is a file that specifies what kind of test to perform, and includes, for example, the test type of the API and the test items for each test type. For example, the test setting file 103 is a file described as follows in a property with a string as a value for performing a boundary test on the number of characters: {"string":["LenMax","LenMaxGreater","LenMin","LenMinSmaller"]}. In this example of the test setting file 103, it is described in JSON (JavaScript (registered trademark) Object Notation) format, but it may be in a relatively readable description format such as YAML, not limited to JSON.

[0030] In addition to strings, "any (all)", "integer (integer)", "number (numeric value)", and "object (object)" can be used as test targets. Also, not only boundary value tests but also tests such as "requiredLost (absence of required property)", "null (null value)", "zero (zero value)", "empty (empty)", "format (format violation)", and "string (string in integer / numeric value)" can be specified in the test setting file 103. Also, boundary value tests can be described in the test setting file 103 together, such as "threshold".

[0031] The test setting file 103 is prepared in advance by a user, for example.

[0032] [Example of test setting file] FIG. 3 is a diagram showing an example of the test setting file 103 of the embodiment. The test setting file 103 is used to generate test scenario data 105 for checking the functions of the API. In the example of FIG. 3, as variations of the test types of the API, it includes parameter type information (test type of parameter type) of the parameters of the API.

[0033] Parameter type <integer>includes test items "zero", "max", "min", and "string". The test item "zero" is an item that performs a test by substituting 0 (or a null character if it is a string type) into the parameter. The test item "max" is an item that performs a test by substituting the maximum value of the parameter into the parameter. The test item "min" is an item that performs a test by substituting the minimum value of the parameter into the parameter. The test item "string" is an item that performs a test by substituting string data into the parameter.

[0034] Parameter type <string>includes test items "zero", "byte", and "len". The test item "zero" is an item that performs a test by substituting a null character (0 if it is a numerical type) into the parameter. The test item "byte" is an item that performs a test by substituting, for example, string data of 2-byte code into the parameter. The test item "len" is an item that performs a test by substituting, for example, string data with the maximum string length that can be substituted into the parameter into the parameter.

[0035] Note that FIG. 3 is an example, and for test items for each parameter type, test items for testing arbitrary data such as upper limit values, lower limit values, standard values, and abnormal values corresponding to the parameter type may be defined.

[0036] Returning to FIG. 2, the correction information 104 includes a correction method, a correction target, and a correction value.

[0037] The correction methods include update, add, and delete, etc.

[0038] The correction target includes information indicating the target to be corrected. The targets to be corrected are a template and a test. The template includes a request template that is a template of the request part and a response template that is a template of the response part. Also, the test is composed of a set of a request part and a response part. The request part included in one test is called a request test, and the response part is called a response test.

[0039] The correction value includes parameters dependent on the test environment 100 during API testing and values of those parameters, etc. For example, the correction value is described as {"path":" / users","method":"POST","property":"email","value":"john.smith@example.com"}.

[0040] Note that these correction information 104 may be included in the test setting file 103. Alternatively, the correction information 104 may include a description such as a link to an external file. In that case, the test code is generated to read the values or functions of the external file, and the values or functions of the external file are used during test execution. A set of information necessary for tests such as preconditions by such values or functions of an external file is called a fixture.

[0041] The test scenario data 105 indicates the test scenario of the API. This test scenario data 105 is used to generate the test code 106 that actually operates as a program.

[0042] The test code generation unit 4 generates the test code 106 based on the test scenario data 105.

[0043] In the example described later, the test code 106 is described in a format that can be read by the API test tool Tavern, but it may be described in a format suitable for the tool, not limited to Tavern such as Karate, Dredd, JMeter, Postman, or ReadyAPI.

[0044] FIG. 4 is a diagram showing an example of the functional configuration of the test scenario generation unit 2 of the embodiment. The test scenario generation unit 2 of the embodiment includes a template generation unit 21 and a test generation unit 22. The template generation unit 21 includes a correction unit 31. The test generation unit 22 includes a correction unit 32. The correction unit 3 in FIG. 2 corresponds to the correction unit 31 and the correction unit 32 in detail.

[0045] The prototype generation unit 21 generates a prototype 108 based on the structural analysis data 102, the test setting file 103, and the correction information 104. The correction unit 31 corrects the prototype 108 generated from the structural analysis data 102 and the test setting file 103 into a prototype 108 suitable for the actual test environment 100. That is, the prototype generation unit 21 generates a prototype 108 for a test corresponding to a normal test from the structural analysis data 102, and generates a corrected prototype 108 by correcting the prototype 108 according to the correction information 104.

[0046] The test generation unit 22 generates a test 109 based on the structural analysis data 102, the test setting file 103, the correction information 104, and the prototype 108. The correction unit 32 corrects the test 109 generated from the structural analysis data 102, the test setting file 103, and the prototype 108 into a test 109 suitable for the actual test environment 100. Specifically, the test generation unit 22 generates a test corresponding to a sub-normal test from the prototype 108 corrected by the correction unit 31 according to the test type and test items included in the test setting file 103, and generates a test 109 corresponding to the corrected sub-normal test by correcting the test corresponding to the sub-normal test according to the correction information 104.

[0047] Note that the test generation unit 22 may directly generate the test 109 without using the prototype 108. Also, the test generation unit 22 may generate a normal test.

[0048] Specifically, the test generation unit 22 selects one property from the prototypes 108, sequentially generates the test 109 described in the test setting file 103 for that property, and generates the test 109 for all properties.

[0049] The template 108 corresponds to a normal test that provides correct information to the test target and verifies that a correct result is returned. The test 109 corresponds to a semi-normal test that changes the correct information provided by the template 108 to incorrect information, provides the incorrect information to the test target, and verifies that an error is returned. The semi-normal test is, for example, a test such as property loss, format violation, and boundary values.

[0050] That is, the set of the template 108 corresponding to the normal test and the test 109 corresponding to the semi-normal test is the test scenario data 105. However, in the case of the boundary value test, since it is created for the boundary limit value and the value exceeding the boundary, the boundary limit value belongs to the normal test.

[0051] FIG. 5 is a diagram showing an example of the functional configuration of the template generation unit 21 of the embodiment. The template generation unit 21 of the embodiment includes a request generation unit 211 and a response generation unit 212.

[0052] The request generation unit 211 (first request generation unit) generates a request template 110 corresponding to a normal request test based on the structure analysis data 102, and generates a corrected request template 110 by correcting the request template according to the correction information 104. That is, the request generation unit 211 includes a correction unit 311 that corrects the request template 110, which is the request part of the template, according to the correction information 104.

[0053] The response generation unit 212 (first response generation unit) generates a response template 111 corresponding to a normal response test based on the structure analysis data 102, and generates a corrected response template 111 by correcting the response template 111 according to the correction information 104. That is, the response generation unit 212 includes a correction unit 312 that corrects the response template 111, which is the response part of the template, according to the correction information 104.

[0054] Note that the correction part 31 of FIG. 4 described above corresponds to the correction part 311 and the correction part 312 in detail.

[0055] A set of the request template 110 and the response template 111 becomes one of the templates 108.

[0056] Specifically, the request generation unit 211 (correction part 311) reads the test target from the structure analysis data 102, and when there is a test target that matches the correction target of the correction information 104, sets the value written in the correction information 104 to the test target. The test target is, for example, a path, a method, a property, a type of property, and a format of property.

[0057] For a test target that does not match the correction target of the correction information 104, the request generation unit 211 sets the default value of the generation device 10. As the default value, when an example is described in the API definition file 101, the value may be used, or an appropriate value according to the type such as an integer, a numerical value, or a character string may be used.

[0058] Similarly, when there is a test target that matches the correction target of the correction information 104, the response generation unit 212 (correction part 312) sets the value written in the correction information 104 to the test target. For a test target that does not match the correction target of the correction information 104, the response generation unit 212 sets the default value of the generation device 10. As the default value, for example, the status code described first in the API definition file 101 may be used, or the status code at the time of successful request may be used.

[0059] <Correction Example 1> FIG. 6A is a diagram showing Example 1 of the API definition file 101 of the embodiment. In Example 1 of the API definition file 101 in FIG. 6A, the / users path has the method POST, the properties email and name are used in the method POST, and it is described that 201 is returned at the time of success and 400 is returned in case of failure for the response.

[0060] FIG. 6B is a diagram showing Example 1 of test code 106 generated from the API definition file 101 of the embodiment. In Example 1 of the test code 106 in FIG. 6B, a template (normal test) using the value described in example, an empty test for email (quasi-normal test), and an empty test for name (quasi-normal test) are generated.

[0061] For example, if the test target 30 only allows registration of actually usable emails, executing a test using the email value in example will fail. Describing a usable email in example of the API definition file 101 described for the purpose of publication is not desirable from a security perspective.

[0062] FIG. 6C is a diagram showing Example 1 of the correction information 104 of the embodiment. In Example 1 of the correction information 104 in FIG. 6C, as the correction target, replacing (updating) a value, the replacement target being a template, and replacing the value of the request among the templates are described. Also, in Example 1 of the correction information 104 in FIG. 6C, as specific values, the path is / users, the method is POST, the property is email, and the value is unique@sample.com are described.

[0063] Note that in Example 1 of FIG. 6C, although the value unique@sample.com is written as an example of the writing method, an actually usable email is described in value. Among these pieces of information, there is also information that is not essential if the correction target (replacement target) is uniquely determined. Alternatively, information for uniquely determining the replacement target may be further added to the correction information 104.

[0064] FIG. 6D is a diagram showing an example 1 of test code 106 generated from the API definition file 101 and correction information 104 of the embodiment. In example 1 of the test code 106 in FIG. 6D, the value of the email in the template (normal test) and the value of the email in the empty test of name (quasi-normal test) are converted into the values in example 1 of the correction information 104. Note that in example 1 of the correction information 104 in FIG. 6C, the object to be corrected is in the template, but the value of the email in the quasi-normal test (empty test of name) is also changed. As shown in FIG. 6D, when the template is corrected, the test created from the corrected template also changes.

[0065] <Correction Example 2> FIG. 7A is a diagram showing an example 2 of the API definition file 101 of the embodiment. Example 2 of the API definition file 101 is the same as example 1.

[0066] FIG. 7B is a diagram showing an example 2 of the test code 106 generated from the API definition file 101 of the embodiment. Example 2 of the test code 106 generated from the API definition file 101 is the same as example 1.

[0067] FIG. 7C is a diagram showing an example 2 of the correction information 104 of the embodiment. Even when using example 2 of the correction information 104 in FIG. 7C for example 2 of the API definition file 101 in FIG. 7A, the same result (FIG. 7D) as example 1 of the test code 106 in FIG. 6D can be obtained.

[0068] Specifically, in example 2 of the correction information 104 in FIG. 7C, only the property and value are described in the definition. By doing so, not only for the method POST of the path / users shown in FIG. 7A, but all the values of the property email described in the API definition file 101 are changed. In example 1 of the API definition file 101 in FIG. 7A, only the description of the method POST of the path / users is given. Therefore, example 2 of the test code 106 (FIG. 7D) according to example 2 of the correction information 104 in FIG. 7C becomes the same as example 1 of the test code 106 in FIG. 6D.

[0069] In the example 2 of the API definition file 101 in FIG. 7A, for example, if the path / users has the method GET and the property email is used in the method GET, test code 106 with the value of the property email of the method GET also changed can be obtained.

[0070] <Correction Example 3> FIG. 8A is a diagram showing Example 3 of the API definition file of the embodiment. Example 3 of the API definition file 101 is the same as Example 1.

[0071] FIG. 8B is a diagram showing Example 3 of the test code 106 generated from the API definition file 101 of the embodiment. Example 3 of the test code 106 generated from the API definition file 101 is the same as Example 1.

[0072] FIG. 8C is a diagram showing Example 3 of the correction information 104 of the embodiment. FIG. 8D is a diagram showing Example 3 of the test code 106 generated from the API definition file 101 and the correction information 104 of the embodiment. When using Example 3 of the correction information in FIG. 8C for Example 3 of the API definition file 101 in FIG. 8A, Example 3 of the test code 106 in FIG. 8D can be obtained.

[0073] Specifically, in Example 3 of the correction information 104 in FIG. 8C, only the type and value are described in the definition. By doing so, not only for the method POST of the path / users described in the API definition file 101 shown in FIG. 8A, but also all the values of the properties whose type is a string described in the API definition file 101 are changed. For this reason, in Example 3 of the test code 106 in FIG. 8D, the value of the property name has also been changed and it becomes an inappropriate setting in the actual test and thus cannot be used, but it can be seen that the correction of unique@sample.com is reflected in all the properties whose type is a string.

[0074] <Correction Example 4> FIG. 9A is a diagram showing Example 4 of the API definition file 101 of the embodiment. In Example 4 of the API definition file 101 in FIG. 9A, the format "email" is added to the property "email".

[0075] FIG. 9B is a diagram showing Example 4 of the test code 106 generated from the API definition file 101 of the embodiment. Example 4 of the test code 106 generated from the API definition file 101 is the same as Example 1.

[0076] FIG. 9C is a diagram showing Example 4 of the correction information 104 of the embodiment. In Example 4 of the correction information 104 in FIG. 9C, the type, format, and value are described in "definition". By doing so, not only for the method "POST" of the path " / users" shown in FIG. 9A, but also all the values of the properties whose type is a string and format is "email" described in the API definition file 101 are changed.

[0077] FIG. 9D is a diagram showing Example 4 of the test code 106 generated from the API definition file 101 and the correction information 104 of the embodiment. By specifying the format in Example 4 of the correction information 104 in FIG. 9C, it can be seen that the value of the property "name" that was changed together in Example 3 in FIG. 8D is not changed in FIG. 9D.

[0078] <Correction Example 5> FIG. 10A is a diagram showing Example 5 of the API definition file 101 of the embodiment. Example 5 of the API definition file 101 is the same as Example 1.

[0079] FIG. 10B is a diagram showing Example 5 of the test code 106 generated from the API definition file 101 of the embodiment. Example 5 of the test code 106 generated from the API definition file 101 is the same as Example 1.

[0080] FIG. 10C is a diagram showing Example 5 of the correction information 104 of the embodiment. In Example 5 of the correction information 104 in FIG. 10C, it is described that as the correction target, the value is to be updated, the replacement target is a template, and the value of the response in the template is to be replaced. Also, as specific values, in the definition, the path is / users, the method is POST, and the status code is 400. Example 5 of the correction information 104 in FIG. 10C shows a method of correcting the test code 106 when the correct request cannot be made by changing the property as in Example 1 of FIG. 6C (for example, when the detailed information of the test target cannot be obtained and it is unclear what conditions of the email should be set).

[0081] FIG. 10D is a diagram showing Example 5 of the test code 106 generated from the API definition file 101 and the correction information 104 of the embodiment. In Example 5 of the test code 106 in FIG. 10D, the status code of the template (normal system test) is converted to 400.

[0082] FIG. 11 is a diagram showing an example of the functional configuration of the test generation unit 22 of the embodiment. The test generation unit 22 of the embodiment includes a request generation unit 221 and a response generation unit 222.

[0083] The request generation unit 221 (the second request generation unit) generates a request test 112 corresponding to a quasi-normal system request test based on the template 108, the structure analysis data 102, and the test setting file 103, and generates a corrected request test 112 by correcting the request test 112 according to the correction information 104. That is, the request generation unit 221 includes a correction unit 321 that corrects the request test 112 according to the correction information 104.

[0084] The response generation unit 222 (second response generation unit) generates a response test 113 corresponding to a quasi-normal system response test based on the template 108, the structure analysis data 102, and the test setting file 103, and generates a corrected response test 113 by correcting the response test 113 according to the correction information 104. That is, the response generation unit 222 includes a correction unit 322 that corrects the response test 113 according to the correction information 104.

[0085] Note that the correction unit 32 in FIG. 4 described above corresponds to the correction unit 321 and the correction unit 322 in detail.

[0086] A set of the request test 112 and the response test 113 becomes one of the tests 109.

[0087] Specifically, the request generation unit 221 reads the template 108 and the test setting file 103, and when the type and format of the properties in the template 108 match the type and format described in the test setting file 103, it generates respective request tests 112. At this time, when the type and format of the properties in the template 108 match the correction target of the correction information 104, the request generation unit 221 (correction unit 321) sets the value written in the correction information 104, and when they do not match, it sets the default value of the generation device 10.

[0088] Similarly, when the type and format of the properties match the correction target of the correction information 104, the response generation unit 222 (correction unit 322) sets the value written in the correction information 104, and when they do not match, it sets the default value of the generation device 10.

[0089] As shown in FIGS. 5 and 11 described above, the generation device 10 of the embodiment is characterized by including request generation units 211 and 221 (first and second request generation units) and response generation units 212 and 222 (first and second response generation units).

[0090] The correction information 104 includes at least one of a correction value for a request correction target included in the request template 110 or the request test 112 (quasi-normal request test), and a correction function for correcting the request correction target, and request correction specification information for specifying the request correction target by at least one of a path, a method, a property, a type, and a format. The request generation units 211 and 221 correct the request correction target specified by the request correction specification information with the correction value or the correction function.

[0091] For example, the test items of the quasi-normal request test include at least one of a property deficiency test, a format violation test, and a boundary value test. The property deficiency test includes, for example, a test for omitting an essential property in the request. The format violation test includes, for example, a test for setting an unacceptable format. The boundary value test includes, for example, a test for setting a value outside the allowed range.

[0092] Further, the correction information 104 includes at least one of a correction value for a response correction target included in the response template 111 or the response test 113 (quasi-normal response test), and a correction function for correcting the response correction target, and response correction specification information for specifying the response correction target by at least one of a path and a method. The response generation units 212 and 222 correct the response correction target specified by the response correction specification information with the correction value or the correction function.

[0093] For example, the test items of the quasi-normal response test include a response test that returns a status code indicating that it is not normal.

[0094] <Correction Example 6> FIG. 12A is a diagram showing Example 6 of the API definition file 101 of the embodiment. Example 6 of the API definition file of the embodiment is the same as Example 1.

[0095] FIG. 12B is a diagram showing Example 6 of test code 106 generated from the API definition file 101 of the embodiment. Example 6 of the test code 106 generated from the API definition file 101 is the same as Example 1.

[0096] FIG. 12C is a diagram showing Example 6 of the correction information 104 of the embodiment. In Example 6 of the correction information 104 in FIG. 12C, it is described that as the correction target, addition (add) is to be performed, the addition target is a test, and the value of the request in the test is to be replaced. Also, as specific values, in the definition, the path is / users, the method is POST, the property is name, the minimum number of characters is 8, and the maximum number of characters is 32.

[0097] Note that in the definition, only the minimum value or only the maximum value may be written as the boundary value of the number of characters. Also, in the definition, not only strings but also integers and numerical values may be described.

[0098] FIG. 12D is a diagram showing Example 6 of the test code 106 generated from the API definition file 101 and the correction information 104 of the embodiment. In Example 6 of the test code 106 in FIG. 12D, tests with a number of characters exceeding the minimum value, a test with exactly the minimum number of characters, a test with exactly the maximum number of characters, and a test with a number of characters exceeding the maximum value are added to Example 1 (FIG. 12B) of the test code 106 generated from the API definition file 101. Specifically, in Example 6 of the correction information 104 in FIG. 12C, the minimum value 8 and the maximum value 32 are specified as the boundary value tests for the number of characters of name. Therefore, in Example 6 of the test code 106 in FIG. 12D, tests with 7 characters which is the minimum value minus 1, 8 characters which is the same value as the minimum value, 32 characters which is the same value as the maximum value, and 33 characters which is the maximum value + 1 are generated.

[0099] Note that the minimum and maximum number of characters can also be described in the API definition file 101. In that case, the generation device 10 automatically generates boundary value tests. However, for example, in the case of OpenAPI, if it is a string, the minimum or maximum number of characters is specified. If the test target determines the upper and lower limits by the number of bytes instead of the number of characters, there may be a discrepancy between the number of characters and the number of bytes. Therefore, in some cases, the minimum or maximum number of characters is not described in the API definition file 101 so as not to cause user misunderstanding. However, even if there is no description, there are upper and lower limits, so testing is necessary. In such cases, the correction information 104 is used.

[0100] <Correction Example 7> FIG. 13A is a diagram showing Example 7 of the API definition file 101 of the embodiment. Example 7 of the API definition file of the embodiment is the same as Example 1.

[0101] FIG. 13B is a diagram showing Example 7 of the test code 106 generated from the API definition file 101 of the embodiment. Example 7 of the test code 106 generated from the API definition file 101 is the same as Example 1.

[0102] FIG. 13C is a diagram showing Example 7 of the correction information of the embodiment. In Example 7 of the correction information 104 in FIG. 13C, the property, the minimum value, and the maximum value are described in the definition. By doing so, not only for the method POST of the path / users shown in the API definition file 101, but also for the property name described in the API definition file 101 in FIG. 13A, the test 109 is generated. In FIG. 13A, only the description of the method POST of the path / users is given. Therefore, the test code 106 (FIG. 13D) generated from the API definition file 101 in FIG. 13A and Example 7 of the correction information 104 in FIG. 13C is the same as Example 6 of the test code 106 in Correction Example 6 (FIG. 12D).

[0103] Note that, for example, if the path / users has the method GET and the property name is used there, the test 109 is also generated for that value.

[0104] <Correction Example 8> Figure 14A is a diagram showing Example 8 of the API definition file 101 of the embodiment. In Example 8 of the API definition file 101 in Figure 14A, a format is added to the property email. As a result, the value of email described later becomes a string with "@" in the middle.

[0105] Figure 14B is a diagram showing Example 8 of the test code 106 generated from the API definition file 101 of the embodiment. Example 8 of the test code 106 generated from the API definition file 101 is the same as Example 1.

[0106] Figure 14C is a diagram showing Example 8 of the correction information 104 of the embodiment. In Example 8 of the correction information 104 in Figure 14C, the type, minimum value, and maximum value are described in the definition. By doing so, not only for the method POST of the path / users shown in the API definition file 101, but also all the values of the properties of the type string described in the API definition file 101 are changed.

[0107] Figure 14D is a diagram showing Example 8 of the test code 106 generated from the API definition file 101 and the correction information 104 of the embodiment. It can be seen that in Example 8 of the test code 106 in Figure 14D, a boundary value test for the property name (the left side of Figure 14D) and a boundary value test for the property email (the right side of Figure 14D) are additionally generated. Specifically, in Example 8 of the correction information 104 in Figure 14C, the minimum value 8 and the maximum value 32 are specified as the boundary value tests for the number of characters of the string type. Therefore, in Example 8 of the test code 106 in Figure 14D, for the values of the properties of the type string, tests for the number of characters of 7 characters which is the minimum value minus 1, 8 characters which is the same as the minimum value, 32 characters which is the same as the maximum value, and 33 characters which is the maximum value + 1 are generated.

[0108] <Correction Example 9> FIG. 15A is a diagram showing Example 9 of the API definition file 101 of the embodiment. Example 9 of the API definition file 101 of the embodiment is the same as Example 8.

[0109] FIG. 15B is a diagram showing Example 9 of the test code 106 generated from the API definition file 101 of the embodiment. Example 9 of the test code 106 generated from the API definition file 101 is the same as Example 1.

[0110] FIG. 15C is a diagram showing Example 9 of the correction information 104 of the embodiment. In Example 9 of the correction information 104 in FIG. 15C, the type, format, minimum value, and maximum value are described in the definition. By doing so, not only for the method POST of the path / users shown in the API definition file 101, but also for the property where the type described in the API definition file 101 is a string and the format is email, boundary value tests are generated.

[0111] FIG. 15D is a diagram showing Example 9 of the test code 106 generated from the API definition file 101 and the correction information 104 of the embodiment. By specifying the format in the correction information 104 in FIG. 15C, it can be seen that the boundary value test of the property name generated in Example 8 in FIG. 14D is not generated in Example 9 in FIG. 15D.

[0112] <Example of the generation method> FIG. 16 is a flowchart showing an example of the overall flow of the test code generation method of the embodiment. First, the analysis unit 1 reads the API definition file 101 (step S1). Next, the test scenario generation unit 2 reads the test setting file 103 (step S2). Next, the test scenario generation unit 2 reads the correction information 104 (step S3). Next, the test scenario generation unit 2 generates test scenario data 105 based on the information read in steps S1 to S3 (step S4). Finally, the test code generation unit 4 generates the test code 106 based on the test scenario data 105 (step S5).

[0113] FIG. 17 is a flowchart showing an example of a test scenario data generation method according to an embodiment. The flowchart of FIG. 17 shows the details of the process of step S4 described above. First, the template generation unit 21 generates a template 108 corresponding to a normal test (step S11). Next, the test generation unit 22 generates a test 109 corresponding to a quasi-normal test based on the template 108 (step S12).

[0114] FIG. 18 is a flowchart showing an example of a template generation method and a test generation method according to an embodiment. The flowchart of FIG. 18 shows the details of the processes of steps S11 and S12 described above.

[0115] First, the template generation unit 21 and the test generation unit 22 generate requests (step S21). Specifically, the template generation unit 21 generates a request template 110, and the test generation unit 22 generates a request test 112.

[0116] Next, the template generation unit 21 and the test generation unit 22 generate responses (step S22). Specifically, the template generation unit 21 generates a response template 111, and the test generation unit 22 generates a response test 113 (step S22).

[0117] FIG. 19 is a flowchart showing an example of a template generation method according to an embodiment. The flowchart of FIG. 19 shows the details of the processes of steps S21 and S22 of the template generation unit 21 described above.

[0118] First, the template generation unit 21 reads the structure analysis data 102 and the correction information 104 and starts the template generation process. Specifically, first, the template generation unit 21 reads a path from the structure analysis data 102 (step S31) and determines the URI (Uniform Resource Identifier) to be processed (step S32).

[0119] Next, the template generation unit 21 reads the method (e.g., post) in the URI to be processed (e.g., / users:) (step S33). Next, the template generation unit 21 reads the property (e.g., email) in the method (step S34). Next, the template generation unit 21 reads the type and format in the property (steps S35, S36).

[0120] Next, the request generation unit 211 determines whether the test target specified by the processing up to step S36 matches the target of the correction information 104 (step S37). If the test target matches the target of the correction information 104 (step S37, Yes), the request generation unit 211 updates (corrects) the value of the test target included in the request by the method specified by the correction information 104 (step S38). If the test target does not match the target of the correction information 104 (step S37, No), the request generation unit 211 determines the value of the test target included in the request as a value that conforms to the type and format (the default value of the generation device 10) (step S39).

[0121] Next, the response generation unit 212 determines whether the test target specified by the processing up to step S36 matches the target of the correction information 104 (step S40). If the test target matches the target of the correction information 104 (step S40, Yes), the response generation unit 212 updates (corrects) the value of the test target included in the response by the method specified by the correction information 104 (step S41). If the test target does not match the target of the correction information 104 (step S40, No), the response generation unit 212 selects, as the value of the test target included in the response, a response such as the status code first described in the API definition file 101 (the default value of the normal response of the generation device 10) (step S42).

[0122] The template generation unit 21 generates the template 108 by repeating the processing of steps S31 to S42 described above.

[0123] FIG. 20 is a flowchart showing an example of the test generation method of the embodiment. The flowchart of FIG. 20 shows the details of the above-described processes of steps S21 and S22 of the test generation unit 22.

[0124] First, the test generation unit 22 reads the structure analysis data 102 and the correction information 104, and starts the test generation process. Specifically, first, the test generation unit 22 reads the template 108 (step S51). Next, the test generation unit 22 reads the request body of the template 108 (step S52). Next, the test generation unit 22 determines the type of the test target in the test setting file 103 (for example, the parameter type <integer>Read it (step S53). Next, the test generation unit 22 reads the test content of the test setting file 103 (for example, test items "zero", "max", "min", and "string") (step S54).

[0125] Next, the request generation unit 221 determines whether the test target specified by the processing up to step S54 matches the target of the correction information 104 (step S55). If the test target matches the target of the correction information 104 (step S55, Yes), the request generation unit 221 updates (corrects) the value of the test target included in the request by the method specified by the correction information 104 (step S56). If the test target does not match the target of the correction information 104 (step S55, No), the request generation unit 221 creates the value of the test target included in the request with a value according to the test content (for example, the maximum value, etc.) (step S57).

[0126] Next, the response generation unit 222 determines whether the test target specified by the processing up to step S54 matches the target of the correction information 104 (step S58). If the test target matches the target of the correction information 104 (step S58, Yes), the response generation unit 222 updates (corrects) the value of the test target included in the response by the method specified by the correction information 104 (step S59). If the test target does not match the target of the correction information 104 (step S58, No), the response generation unit 222 selects a response such as the quasi-normal status code described in the API definition file 101 (for example, the default value of the quasi-normal response of the generation device 10) as the value of the test target included in the response (step S60).

[0127] Next, the test generation unit 22 adds the test 109 created by the processing in steps S55 to S60 to the array storing the test 109 (step S61).

[0128] The test generation unit 22 generates the test 109 by repeating the processes of steps S51 to S61 described above.

[0129] As described above, in the generation device 10 of the embodiment, the analysis unit 1 analyzes API definition information (in the embodiment, the API definition file 101) that defines the structure of the API, and generates API structure analysis data 102. The test scenario generation unit 2 generates test scenario data 105 for the API from test setting information (in the embodiment, the test setting file 103) including the test type of the API and test items for each test type, and the structure analysis data 102, and corrects the test scenario data 105 according to the correction information 104, thereby generating corrected test scenario data 105. Then, the test code generation unit 4 generates a test code 106 from the corrected test scenario data 105.

[0130] Accordingly, according to the generation device 10 of the embodiment, when generating the test code 106 from the API definition information, the test code 106 can be corrected to a more appropriate test code 106 by information not described in the API definition information. For example, according to the generation device 10 of the embodiment, information that is not described in the API definition information but is required in the actual test environment 100 can be reflected in the test code 106.

[0131] Finally, an example of the hardware configuration of the generation device 10 of the embodiment will be described.

[0132] [Example of Hardware Configuration] FIG. 21 is a diagram showing an example of the hardware configuration of the generation device 10 of the embodiment. The generation device 10 of the embodiment includes a control device 201, a main storage device 202, an auxiliary storage device 203, a display device 204, an input device 205, and a communication device 206. The control device 201, the main storage device 202, the auxiliary storage device 203, the display device 204, the input device 205, and the communication device 206 are connected via a bus 210.

[0133] Note that the generation device 10 of the embodiment may not be provided with some of the above configurations. For example, when the generation device 10 can utilize the input function and display function of an external device, the generation device 10 may not be provided with the display device 204 and the input device 205.

[0134] The control device 201 executes the program read from the auxiliary storage device 203 to the main storage device 202. The main storage device 202 is a memory such as a ROM and a RAM. The auxiliary storage device 203 is an HDD (Hard Disk Drive), a memory card, etc.

[0135] The display device 204 is, for example, a liquid crystal display or the like. The input device 205 is an interface for operating the generation device 10. Note that the display device 204 and the input device 205 may be realized by a touch panel or the like having a display function and an input function. The communication device 206 is an interface for communicating with other devices.

[0136] The program executed by the generation device 10 is recorded in a computer-readable storage medium such as a CD-ROM, a memory card, a CD-R, and a DVD in an installable format or an executable format file and provided as a computer program product.

[0137] Alternatively, the program executed by the generation device 10 may be stored on a computer connected to a network such as the Internet and provided by downloading it via the network. Alternatively, the program executed by the generation device 10 may be provided via a network such as the Internet without being downloaded.

[0138] Alternatively, the program of the generation device 10 may be configured to be provided by being pre-embedded in a ROM or the like.

[0139] The program executed by the generation device 10 has a module configuration including functions that can also be realized by the program among the functional configurations shown in FIGS. 2, 4, 5, and 11 described above. As for each of these functions, in terms of actual hardware, when the control device 201 reads and executes the program from the storage medium, each of the above functional blocks is loaded onto the main storage device 202. That is, each of the above functional blocks is generated on the main storage device 202.

[0140] Note that part or all of each of the functions shown in FIGS. 2, 4, 5, and 11 described above may be realized by hardware such as an IC instead of being realized by software.

[0141] Also, when realizing each function using a plurality of processors, each processor may realize one of the functions or may realize two or more of the functions.

[0142] Although some embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention and are included in the invention described in the claims and the equivalent scope thereof.

Explanation of Reference Numerals

[0143] 1 Analysis Unit 2 Test Scenario Generation Unit 3 Correction Unit 4 Test Code Generation Unit 10 Generation Device 20 Test Execution Device 21 Template Generation Unit 22 Test Generation Unit 30 Object Under Test 31 Correction Unit 32 Correction Unit 100 Test Environment 201 Control Device 202 Main Storage Device 203 Auxiliary storage device 204 Display device 205 Input device 206 Communication device 210 Bus 211 Request generation unit 212 Response generation unit 221 Request generation unit 222 Response generation unit 311 Correction unit 312 Correction unit 321 Correction unit 322 Correction unit< / integer> < / string> < / integer>

Claims

1. An analysis unit that analyzes API definition information defining the structure of an API (Application Programming Interface) and generates structure analysis data of the API; A test scenario generation unit that generates test scenario data of the API from the test setting information including the test type of the API and test items for each test type, and the structure analysis data, and generates corrected test scenario data by correcting the test scenario data according to correction information; A test code generation unit that generates test code from the corrected test scenario data; A generation device comprising the above.

2. The test scenario generation unit: A template generation unit that generates a template for a test corresponding to a normal system test from the structure analysis data and generates a corrected template by correcting the template according to the correction information; A test generation unit that generates a test corresponding to a quasi-normal system test from the corrected template according to the test type and the test items, and generates a test corresponding to a corrected quasi-normal system test by correcting the test corresponding to the quasi-normal system test according to the correction information; The generation device according to Claim 1, comprising the above.

3. The template generation unit: A first request generation unit that generates a request template corresponding to a normal system request test and generates a corrected request template by correcting the request template according to the correction information; A first response generation unit that generates a response template corresponding to a normal system response test and generates a corrected response template by correcting the response template according to the correction information, and the test generation unit: A second request generation unit that generates a quasi-normal system request test and generates a corrected quasi-normal system request test by correcting the quasi-normal system request test according to the correction information; A second response generation unit that generates a quasi-normal system response test and generates a corrected quasi-normal system response test by correcting the quasi-normal system response test according to the correction information; The generation device according to Claim 2, comprising the above.

4. ​ The correction information includes at least one of a correction value for a request correction target included in the request template or the quasi-normal request test, and a correction function for correcting the request correction target, and request correction specification information for specifying the request correction target by at least one of a path, a method, a property, a type, and a format. The first and second request generation units correct a request correction target specified by the request correction specification information with the correction value or the correction function. The generation device according to claim 3.

5. The test items of the quasi-normal request test include at least one of a property missing test, a format violation test, and a boundary value test. The generation device according to claim 4.

6. The correction information includes at least one of a correction value for a response correction target included in the response template or the quasi-normal response test, and a correction function for correcting the response correction target, and response correction specification information for specifying the response correction target by at least one of a path and a method. The first and second response generation units correct a response correction target specified by the response correction specification information with the correction value or the correction function. The generation device according to any one of claims 3 to 5.

7. The test items of the quasi-normal response test include a response test that returns a status code indicating non-normality. The generation device according to claim 6.

8. A step in which the generation device analyzes API definition information defining the structure of an API (Application Programming Interface) and generates structure analysis data of the API. A step in which the generation device generates test scenario data of the API from the test setting information including the test type of the API and the test items for each test type, and the structure analysis data, and generates corrected test scenario data by correcting the test scenario data according to the correction information. A step in which the generation device generates test code from the corrected test scenario data. A generation method including the above steps.

9. A computer An analysis unit that analyzes API definition information defining the structure of an API (Application Programming Interface) and generates structure analysis data of the API; A test scenario generation unit that generates test scenario data of the API from test setting information including the test type of the API and test items for each test type, and the structure analysis data, and generates corrected test scenario data by correcting the test scenario data according to correction information; A test code generation unit that generates test code from the corrected test scenario data; A program for causing the above to function.

Images