Information processing device, information processing method, and program

Abstract

To automatically generate more accurate test scenarios from API specifications by using natural language processing.SOLUTION: An information processing device according to an embodiment includes a communication interface and a processing unit. The communication interface transmits an inference request for an API (Application Programming Interface) calling method included in API specifications to a specification interpretation device that interprets the API specifications using natural language processing, and receives a response indicating the API calling method included in the API specifications from the specification interpretation device. The processing unit determines whether the API calling method indicated in the response is correct, and if the API calling method indicated in the response is correct, generates a test scenario based on the API calling method indicated in the response.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] An embodiment of the present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Recently, automatic test scenario generation technology has become increasingly important in API (Application Programming Interface) development. A test scenario is a series of procedures and test cases used to verify whether an API functions as expected. When test scenarios are created manually, they can lack comprehensiveness and accuracy due to human judgment, errors, or omissions. Especially for large-scale APIs or those with frequently changing specifications, manual updates are extremely labor-intensive, so there is a growing need for automation. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-111367 [Patent Document 2] Patent Publication No. 2021-039718 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technology, it has been difficult to automatically generate more accurate test scenarios from API specifications using natural language processing. [Means for solving the problem]

[0005] According to an embodiment, an information processing device includes a communication interface and a processing unit. The communication interface transmits an inference request for an API (Application Programming Interface) calling method included in an API specification to a specification interpretation device that interprets the API specification using natural language processing, and receives a response from the specification interpretation device indicating the API calling method included in the API specification. The processing unit determines whether the API calling method indicated in the response is correct, and if the API calling method indicated in the response is correct, generates a test scenario based on the API calling method indicated in the response. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 2 is a diagram showing an example of the functional configuration of the information processing apparatus according to the first embodiment. [Figure 2] FIG. 2 is a diagram showing Example 1 of an API specification according to the first embodiment. [Figure 3] FIG. 10 is a diagram showing Example 1 of a response to a prompt in the first embodiment. [Figure 4] FIG. 10 is a diagram showing example 2 of a response to a prompt in the first embodiment. [Figure 5] FIG. 10 is a diagram showing example 3 of a response to a prompt in the first embodiment. [Figure 6A] FIG. 3 is a diagram showing an example of an HTTP request according to the first embodiment. [Figure 6B] FIG. 6B is a diagram showing an example of a response to the HTTP request of FIG. 6A. [Figure 7] FIG. 4 is a diagram showing Example 1 of a series of HTTP requests and responses according to the first embodiment. [Figure 8] FIG. 10 is a diagram showing Example 2 of a series of HTTP requests and responses according to the first embodiment. [Figure 9] FIG. 10 is a diagram showing Example 3 of a series of HTTP requests and responses according to the first embodiment. [Figure 10A] FIG. 10 is a diagram showing Example 1 of how to call an API that is a normal system in the first embodiment. [Figure 10B] FIG. 10 is a diagram showing Example 2 of how to call an API that is a normal system in the first embodiment. [Figure 10C]FIG. 10 is a diagram showing Example 3 of how to call an API that is a normal system in the first embodiment. [Figure 11A] FIG. 10 is a diagram showing an example 1 of an abnormal test scenario according to the first embodiment. [Figure 11B] FIG. 10 is a diagram showing an example 2 of an abnormal test scenario in the first embodiment. [Figure 12A] FIG. 10 is a diagram showing an example 3 of an abnormal test scenario in the first embodiment. [Figure 12B] FIG. 10 is a diagram showing an example 4 of an abnormal test scenario according to the first embodiment. [Figure 13] FIG. 10 is a diagram showing Example 2 of an API specification according to the first embodiment. [Figure 14] FIG. 10 is a diagram showing Example 4 of a response to a prompt in the first embodiment. [Figure 15] FIG. 10 is a diagram showing an example 5 of an abnormal test scenario according to the first embodiment. [Figure 16] 4 is a flowchart showing an example of an information processing method according to the first embodiment. [Figure 17] FIG. 10 is a diagram showing an example of the functional configuration of an information processing apparatus according to a second embodiment. [Figure 18] FIG. 10 is a diagram showing an example of an API specification according to the second embodiment. [Figure 19A] FIG. 10 is a diagram showing an example of a response to a prompt according to the second embodiment. [Figure 19B] FIG. 10 is a diagram showing an example of a response to a prompt according to the second embodiment. [Figure 20] 4 is a flowchart showing an example of an information processing method according to the first embodiment. [Figure 21] FIG. 2 is a diagram showing an example of the hardware configuration of an information processing apparatus according to the first and second embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0007] Hereinafter, embodiments of an information processing device, an information processing method, and a program will be described in detail with reference to the accompanying drawings.

[0008] Recently, OpenAPI (formerly known as Swagger), a specification for describing API specifications, has become popular, allowing for the structured description of API endpoints, parameters, responses, and other details. This structured information is highly suitable as a foundation for automated test case generation.

[0009] However, there are specifications that cannot be fully expressed with the expressive power of OpenAPI. For example, dependencies between multiple APIs (such as using the response of API-A as input for API-B), or how to specify complex parameters, etc. These specifications must be freely written in the OpenAPI description field, or the reader may have to infer them from the endpoint name or parameter name.

[0010] The application of natural language processing technology, such as generative AI like ChatGPT (registered trademark), is now coming into view. It is expected that generative AI will be able to analyze the content of free descriptions and identify API specifications that cannot be fully expressed in OpenAPI. Furthermore, generative AI is expected to be able to interpret API specifications based on best practices and conventions for API design around the world.

[0011] (First embodiment) In the first embodiment, an information processing device is described that automatically generates test scenarios from the description contents of an API specification and inputs and executes various normal and abnormal test patterns into an API execution device under development, thereby improving API quality.

[0012] The first embodiment is an example of generating test scenarios to verify whether responses to various inputs are appropriate for an API execution device that has been developed by a user (e.g., an API developer) and in which at least normal processing is correctly implemented. Note that the first embodiment can be implemented for normal processing even at a stage where normal processing has been implemented but abnormal processing has not yet been implemented.

[0013] In tests using the test scenario of the first embodiment, it can be confirmed that a status code indicating success / failure is returned. In tests of normal processing, it can be confirmed that a status code indicating success is returned. In tests of abnormal processing, it can be confirmed that a status code indicating failure is returned.

[0014] [Example of functional configuration] 1 is a diagram showing an example of the functional configuration of an information processing apparatus 1 according to the first embodiment. The information processing apparatus 1 according to the first embodiment includes a processing unit 10 and a communication IF (Interface) 20.

[0015] The processing unit 10 generates a test scenario 12 based on, for example, an input of an API specification 11 from a user of an organization that creates an API.

[0016] The processing unit 10 is realized by at least one processing device and executes the processing of the information processing device 1. This processing device includes, for example, a control device and an arithmetic device, and is realized by analog or digital circuits, etc. The processing device may be a central processing unit (CPU), a general-purpose processor, a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0017] The processing unit 10 includes a request unit 101, a determination unit 102, a trial unit 103, a generation unit 104, a correction unit 105, and a transmission control unit 106.

[0018] The request unit 101 receives input of the API specification 11 and requests the specification interpretation device 2 to infer (infer) the API invocation method included in each API specification via the communication IF 20. Inferring the API invocation method in the first embodiment means inferring "a series of API invocation methods that will result in a successful status code" for each API. The request to infer the API invocation method included in the API specification may include a request to present multiple candidates indicating the API invocation method. The determination unit 102 determines whether the API invocation method obtained by inference by the specification interpretation device 2 is correct.

[0019] The determination unit 102 determines whether or not the API calling method obtained by inference by the specification interpretation device 2 is correct. For example, if the response to the API request includes a success status code, the determination unit 102 determines that the API calling method is correct.

[0020] The attempt unit 103 attempts to execute an API request by transmitting the API request to the API execution device 3 via the communication IF 20 .

[0021] The generator 104 generates a test scenario 12 based on the API calling method determined to be correct by the determiner 102.

[0022] The correction unit 105 corrects the API calling method based on instructions from the user. For example, the correction unit 105 displays information indicating the API calling method determined to be correct on a display device, and corrects the API calling method based on correction information input by the user.

[0023] The transmission control unit 106 transmits the learning data including information indicating the method of calling the API that has been determined to be correct to the specification interpretation device 2 via the communication IF 20.

[0024] The communication IF 20 communicates with other devices such as the specification interpretation device 2 and the API execution device 3. For example, the communication IF 20 receives a response to a request for inferring an API calling method included in an API specification from the specification interpretation device 2, and transmits at least one API request to the API execution device 3 based on the API calling method indicated in the response.

[0025] The specification interpretation device 2 analyzes the contents of the API specification 11 and infers how to call the API. In the first embodiment, the specification interpretation device 2 is, for example, a server device of ChatGPT (registered trademark).

[0026] The API execution device 3 is a server device developed by the user (for example, an API developer) of the first embodiment based on the API specification 11. It generates output based on the input of a request and returns it as a response. It is assumed that at least normal processing is correctly implemented in the API execution device 3.

[0027] Next, a description will be given of the processing performed by the information processing device 1 according to the first embodiment. The request unit 101 receives input of the API specification 11 in response to an operation input by the user or the like.

[0028] 2 is a diagram showing example 1 of the API specification 11 according to the first embodiment. In the example of FIG. 2, the following information is written as specifications in the OpenAPI format for three APIs:

[0029] ------------------Specification example-- 1. Create a new organization Endpoint: / organizations Method: POST Request Body: name (string): The name of the organization. response: status: 201 Created 400 Bad Request body: id (string): A unique identifier for the organization. name (string): The name of the organization. 2. Create a new user Endpoint: / users Method: POST Request Body: title (string): The user's name. age (int): Age. response: status: 201 Created 400 Bad Request body: id (string): A unique identifier for the user. name (string): The user's name. age (int): Age. 3. Register the user to the organization Endpoint: / organizations / {organizationId} / users Method: POST Parameters: organizationId (string - path): The ID of the organization to which the user will be added. Request Body: userId (string): The ID of the user being added to the organization. response: status: 201 Created 400 Bad Request ------------------------------------

[0030] Next, the usage intended by the creator of the API specification is as follows:

[0031] ---Example of use--- The first API (Figure 2, left) is used to create a new organization. / organizations sends a request to the endpoint using the POST method. The request body contains the name of the new organization to be created, specified as name in JSON format. Example request: { ”name”: ”Sales department” } If successful, the response will return status code 201 along with information about the newly created organization. Example response: { ”id”: ”abc123”, ”name”: ”Sales department” } If an invalid parameter is specified, a status code of 400 is returned.

[0032] The second API (center of Figure 2) is an API for creating a new user. / users sends a request to the endpoint using the POST method. The request body is in JSON format, with the name of the newly created user specified as name and the age specified as age. Example request: { ”name”: ”John Smith”, ”age”: 30 } If successful, the response will return status code 201 along with information about the newly created user. Example response: { ”id”: ”def456”, ”name”: ”John Smith”, ”age”: 30 } If an invalid parameter is specified, a status code of 400 is returned.

[0033] The third API (right side of Figure 2) is an API for registering users to an organization. / organizations / {organizationId} / users sends a request to the endpoint using the POST method. Here, {organizationId} specifies the ID of the organization to which you want to add the user. In the request body, the ID of the user you want to add to the organization is specified as userId, and the user's role in the organization is specified as role. example: { ”userId”: ”def456”, ”role”: ”manager”, } If successful, status code 201 is returned. Example response: { ”message”: ”The user was successfully registered in the organization.” } If an invalid parameter is specified, a status code of 400 is returned. ------------------------------------

[0034] When the processing unit 10 receives the API specification 11 as input, it generates a test scenario 12 according to the following procedure: In the test scenario 12, various inputs are given to the parameters of each API, and it is confirmed that an appropriate status code is obtained for each.

[0035] First, the request unit 101 analyzes the API specification 11 and extracts the endpoints, parameters, responses, etc. of each API defined in the API specification 11. Next, when testing the API, the request unit 101 requests the specification interpretation device 2 to infer how to call the API in order to determine what values ​​should be specified for each parameter in order to perform a normal test.

[0036] The following is an example of a prompt generated by the request unit 101. This prompt asks for a normal test scenario 12 (a method of calling an API that will result in a successful status code) for the first API (POST / organizations) of the three examples in FIG.

[0037] --Example prompt-- Below is a Web API specification written in OpenAPI format. For the target API below, guess at least five ways to call the API that will result in a successful status code from this API, and output the results while strictly adhering to the following constraints. # Target API POST / organizations # Constraints - Never output text other than JSON. - Output in JSON format according to the following JSON schema. - The last element of each stage corresponds to the target API call. { “$schema”: “http: / / json-schema.org / draft-04 / schema#”, ”type”: ”object”, ”properties”: { ”candidates”: { ”type”: ”array”, ”items”: [ { ”type”: ”object”, ”properties”: { ”stages”: { ”type”: ”array”, ”items”: [ { ”type”: ”object”, ”properties”: { ”path”: { "type": "string" }, ”method”: { "type": "string" }, "params": { "type": "object" } }, ”required”: [ "path", "method", "params" ] } ] } }, ”required”: [ "stages" ] } ] } }, ”required”: [ "candidates" ] } - stages specifies the "series of API calls." A "series of API calls" is an array of "API calls." An "API call" corresponds to sending a request to an API and receiving a response, and is composed of a path, method, and parameters. By calling the APIs in the order specified in the "series of API calls," it is expected that the API call to the target API will return a successful status code. - If you assume that the target API needs to call another API before it can return a successful status code, include that API call in the stages array. The final API call is expected to return a successful status code when the APIs are called in order from the top of the stages array. The last element of stages must be the target API call. - If you infer that you need to make the next API call using the response of a previous API call, please indicate this dependency in your response. If you are requesting an API call for the second element of stages using the element foo in the response of the API call for the first element of stages, write {{0.foo}} in params. - candidates specifies multiple candidate patterns for calling the API that are likely to result in a successful status code. Prioritize patterns with fewer API calls and a higher chance of obtaining a successful status code toward the beginning. Simple guesses may not result in a successful status code, so be sure to also list complex patterns involving multiple API calls. # Web API specification openapi: 3.0.0 info: ------------------------------------

[0038] The specification interpretation device 2 returns a response to the above prompt asking about a normal test scenario 12 (a method of calling an API that will result in a successful status code) in response to the first API (POST / organizations) in Figure 2. An example of the response is shown in Figure 3.

[0039] Fig. 3 is a diagram showing example 1 of a response to a prompt in the first embodiment. In the example of Fig. 3, multiple candidates for calling a series of APIs are shown. Note that if the response in Fig. 3 is not as expected, the request unit 101 may make a part or all of the request again.

[0040] The request unit 101 creates similar prompts for other APIs (center and right in Figure 2) and makes requests to the specification interpretation device 2. The response to the second API (POST / users) in Figure 2 is shown in Figure 4, and the response to the third API (POST / organizations / {organizationId} / users) in Figure 2 is shown in Figure 5.

[0041] Fig. 4 is a diagram showing example 2 of a response to a prompt in the first embodiment. Fig. 4 shows an example of a response to the above prompt, which asks for a normal test scenario 12 (a method of calling an API that will result in a successful status code) for the second API (POST / users) in Fig. 2. In the example of Fig. 4, multiple candidates for a series of methods of calling an API are shown.

[0042] Fig. 5 is a diagram showing example 3 of a response to a prompt in the first embodiment. Fig. 5 shows an example of a response to the prompt that asks for a normal test scenario 12 (a method of calling an API that will result in a successful status code) for the third API (POST / organizations / {organizationId} / users) in Fig. 2. In the example of Fig. 5, multiple candidates for a series of methods of calling an API are shown.

[0043] In the first embodiment, a separate prompt is created for each API and sent to the specification interpretation device 2, but it is also possible to write the inference request for how to call all APIs in a single prompt and request it only once from the specification interpretation device 2.

[0044] The request unit 101 transmits the obtained API invocation method (JSON format) to the determination unit 102 and requests it to determine whether the API invocation method is correct. The determination unit 102 tries a series of API invocation method candidates for each API, starting from the top. For example, in the case of the first API (POST / organizations) in Figure 2, the first candidate (candidate) according to Figure 3 is to make the following API call. Method: POST Path: / organizations Parameters: { “name”: ”SampleOrganization1” }

[0045] The determination unit 102 requests the attempt unit 103 to create and send an HTTP request. The attempt unit 103 creates an HTTP request in response to the request from the determination unit 102 and sends it to the API execution unit 3. The API execution unit 3 processes the request and returns an HTTP response. Examples of the request and response are shown in Figures 6A and 6B.

[0046] FIG. 6A is a diagram showing an example of an HTTP request in the first embodiment. FIG. 6B is a diagram showing an example of a response to the HTTP request in FIG. 6A. In the cases of FIGS. 6A and 6B, the request is successful, and status code 201 and details of the created resource are returned in the response body. The determination unit 102 determines that FIG. 6A is a normal way of calling an API when a successful status code is obtained. This completes the determination of the specifications for the first API in FIG. 2.

[0047] The determination unit 102 also determines the specifications of the second API in Figure 2. For the second API, a success status code is obtained by calling the first API in the candidates in Figure 4. Therefore, the determination unit 102 determines that the first API call in the candidates in Figure 4 is the normal way to call the API. Method: POST Path: / users Parameters: { “name”: ”John Doe”, “age”: 30 }

[0048] If the determination unit 102 does not obtain a successful status code (for example, status code 400), it continues to call the API of the next element in the candidates and checks whether a successful status code can be obtained.

[0049] The determination unit 102 also determines the specifications of the third API in Fig. 2. According to Fig. 5, the first candidate consists of the following two API calls. 1st time Method: POST Path: / organizations Parameters: { “name”: ”Tech Corp” } Second time Method: POST Path: / organizations / {{0.id} / users Parameters: { ”userId”: ”existingUserId”, "role": "admin" }

[0050] For the second API call, the contents of the first response are used, as instructed by the prompt. In other words, the id portion of the first response is used for the {{0.id}} portion of the path. The determination unit 102 assembles the body portion of the second HTTP request from the first response. An example of a series of requests and responses is shown in Figure 7.

[0051] FIG. 7 is a diagram showing Example 1 of a series of HTTP requests and responses in the first embodiment. In the second request, the actual ID value (abc123) is used in the {{0.id}} portion of the path. However, the second request fails with status code 400. This is because the character string "existingUserId" specified as the user identifier is an identifier that does not exist in the actual system. From this, the determination unit 102 determines that the request in FIG. 7 is not a normal API call and tries the next candidate in FIG. 5. The next candidate consists of the following two API calls: 1st time Method: POST Path: / users Parameters: { “name”: ”Eve”, “age”: 35 } Second time Method: POST Path: / organizations / existingOrgId / users Parameters: { ”userId”: ”{{0.id}}”, ”role”: ”contributor” }

[0052] As with the first candidate, the second API call uses the contents of the first response. An example of a series of requests and responses is shown in Figure 8.

[0053] FIG. 8 is a diagram showing Example 2 of a series of HTTP requests and responses in the first embodiment. The second request fails with status code 400. This is because the character string "existingOrgId" specified as the organization identifier is an identifier that does not exist in the actual system. Therefore, the determination unit 102 determines that the request in FIG. 8 is not a normal way of calling an API, and tries the next candidate in FIG. 5. The next candidate consists of the following three API calls: 1st time Method: POST Path: / organizations Parameters: { “name”: “My Organization” } Second time Method: POST Path: / users Parameters: { “name”: ”John”, “age”: 30, } Third time Method: POST Path: / organizations / {{0.id} / users Parameters: { ”userId”: ”{{1.id}}”, ”role”: ”member” }

[0054] Of these, the third API call uses the contents of the first and second responses. The id part of the first response is used for the {{0.id}} part of the path. The id part of the second response is used for the {{1.id}} part of the parameter. The determination unit 102 assembles the body part of the third HTTP request from the first and second responses. An example of a series of requests and responses is shown in Figure 9.

[0055] 9 is a diagram showing Example 3 of a series of HTTP requests and responses in the first embodiment. The final (third) request is successful with status code 201. The determination unit 102 determines that the API call shown in FIG. 9 is a normal API call method based on the successful status code obtained. This completes the determination of the specifications for the third API.

[0056] The determining unit 102 passes the specifications determined by the above procedure (the API calling method that is normal: FIGS. 10A to 10C) to the modifying unit 105.

[0057] FIG. 10A is a diagram showing example 1 of how to call an API that is a normal system in the first embodiment. FIG. 10B is a diagram showing example 2 of how to call an API that is a normal system in the first embodiment. FIG. 10C is a diagram showing example 3 of how to call an API that is a normal system in the first embodiment. The correction unit 105 displays the specifications determined by the determination unit 102 (e.g., FIGS. 10A to 10C) on a display device or the like, and asks a user of the information processing device 1 (e.g., an API developer) to confirm whether the specifications determined by the determination unit 102 are as expected. If the specifications are different from expectations, the user inputs correction information via an input device or the like, and the specifications determined by the determination unit 102 are corrected by the correction information.

[0058] Next, the determination unit 102 passes the specifications finally identified through the above procedure to the transmission control unit 106 and requests it to transmit the learning data. The transmission control unit 106 transmits the learning data to the specification interpretation device 2 via the communication IF 20.

[0059] The specification interpretation device 2 uses the learning data to learn which candidate is the correct way to call the API, and uses this to improve accuracy from the next time onwards.

[0060] The determining unit 102 inputs the specifications determined by the above procedure (the API calling method that is normal: FIGS. 10A to 10C) to the generating unit 104 and requests the generating unit 104 to generate a test scenario 12.

[0061] The generation unit 104 generates a test scenario 12 based on the specifications determined by the determination unit 102. The generation unit 104 generates a normal test scenario that is a method of calling an API request in which the response to the API request includes a success status code. The generation unit 104 then generates an abnormal test scenario based on the normal test scenario. The abnormal test scenario includes a test that verifies that invalid parameter inputs are made in various patterns for each parameter and that a failure status code is returned.

[0062] The abnormal test scenario 12 includes, for example, the following tests. Verify that you get status code 400 when calling the API without specifying required parameters. -Confirm that status code 400 is returned when calling the API with an invalid parameter type.

[0063] Each test scenario 12 is generated by rewriting the specification of specific parameters based on the normal API call (FIGS. 10A to 10C) identified by the determination unit 102. Examples of generating scenarios for the two abnormal test cases described above for the first API (POST / organizations) in FIG. 2 are shown in FIGS. 11A and 11B.

[0064] FIG. 11A is a diagram showing example 1 of an abnormal test scenario 12 according to the first embodiment. FIG. 11B is a diagram showing example 2 of an abnormal test scenario 12 according to the first embodiment. In the examples of FIGS. 11A and 11B, the abnormal test scenario 12 is written in the format of tavern, a pytest-based test tool. In the test scenario 12 of FIG. 11A, it is confirmed that a status code of 400 is returned when an API is called without specifying the required parameter name. In the test scenario 12 of FIG. 11B, it is confirmed that a status code of 400 is returned when an API is called by specifying an invalid type (specifying a numeric type where a string type should be specified).

[0065] 12A and 12B show examples of generation in the third API (POST / organizations / {organizationId} / users) in FIG. 2.

[0066] FIG. 12A is a diagram showing example 3 of the abnormal test scenario 12 of the first embodiment. FIG. 12B is a diagram showing example 4 of the abnormal test scenario 12 of the first embodiment. The examples in FIGS. 12A and 12B are test scenarios 12 with the same purpose as in FIG. 2, for the first API. In FIGS. 12A and 12B, the way the parameter role is specified is changed in various ways based on the content of FIG. 10C.

[0067] In the first embodiment, the above two methods for generating the test scenario 12 are described. Alternatively, the following abnormal test scenario 12 may be generated. - Missing authentication headers, including specifying a password to allow only specific users - Format violation (email, etc.) -Values ​​other than the options in enum Invalid ID string Boundary value tests for numeric parameters Specify a null value Length bounds testing for string parameters with lower / upper length bounds

[0068] The generation unit 104 makes the generated test scenario 12 available for download by the user as a file. For example, an API developer can download the test scenario 12 and run a test tool (tavern in the first embodiment) on a local terminal or the like to execute a test using the test scenario 12. By referring to the test results, the API developer can detect malfunctions in the API execution device 3 under development.

[0069] Note that there are other ways to use test scenario 12 besides having the user download it. For example, the system may further include a test execution unit, and processing unit 10 may pass test scenario 12 to the test execution unit and request test execution. In this case, too, the test execution unit may present the test execution results to the API developer as a report, allowing the API developer to detect any defects in the API execution device 3.

[0070] In the first embodiment, the test scenario 12 is automatically generated using the specifications determined by the determining unit 102. However, other uses of the determined specifications are also conceivable. For example, the generating unit 104 may automatically generate sample code showing usage examples of each API. By widely disclosing this sample code together with the API specification 11, it is expected that API users will be able to smoothly understand and master the API usage procedures. For another example, the generating unit 104 may automatically generate a Try It Out page. The Try It Out page is an HTML page that allows users to try out the API. By configuring this page according to the determined specifications, it is possible to encourage API users to smoothly understand the usage procedures. Alternatively, the generating unit 104 may process the information corresponding to FIGS. 10A to 10C into an HTML document and use it as a manual for how to use the API. The manual may then be presented on a display device or the like.

[0071] An example of the operation when an API specification 11 different from that shown in Figure 2 is entered is shown below.

[0072] Fig. 13 is a diagram showing example 2 of API specification 11 according to the first embodiment. Fig. 14 shows an example of a response when a similar prompt is requested of the specification interpretation device 2 regarding API specification 11 in Fig. 13.

[0073] Fig. 14 is a diagram showing example 4 of a response to a prompt in the first embodiment. When the request unit 101 inputs the response in Fig. 14 to the attempt unit 103 and the attempt unit 103 makes an attempt, in the case of Fig. 14, a success status code is obtained for the first candidate. Method: GET Path: / organizations Parameters: { “name”: ”\”SpecificOrgName\”“ }

[0074] This identifies the normal API calling method. The generation unit 104 generates an abnormal test scenario 12 based on this normal API calling method. For example, the test scenario 12 when an invalid type is specified for the parameter count is as shown in Figure 15.

[0075] Fig. 15 is a diagram showing example 5 of the abnormal test scenario 12 of the first embodiment. In the test scenario 12 of Fig. 15, it is confirmed that the status code 400 is returned when an API is called with an invalid type specified in the parameter count.

[0076] [Examples of information processing methods] 16 is a flowchart showing an example of the information processing method of the first embodiment. First, the communication IF 20 transmits an inference request for an API invocation method included in the API specification 11 to the specification interpretation device 2, which interprets the API specification 11 using natural language processing (step S1). Next, the communication IF 20 receives a response indicating an API invocation method included in the API specification 11 from the specification interpretation device 2 (step S2). Next, the processing unit 10 attempts the API request by transmitting at least one API request to the API execution device 3 based on the API invocation method candidates indicated in the response received in step S2 (step S3).

[0077] Next, the processing unit 10 determines whether the API calling method is correct or not based on the trial result of step S3 (step S4). For example, if the response to the API request includes a success status code, the processing unit 10 determines that the API calling method indicated by the response of step S2 is correct.

[0078] If the API calling method is incorrect (No at step S4), the process returns to step S3, and an API request is attempted based on the candidate method for calling the next API included in the response received at step S2.

[0079] If the API calling method is correct (step S4, Yes), the processing unit 2 generates a test scenario 12 based on the API calling method determined to be correct (step S5). Note that the processing unit 2 may present the API calling method determined to be correct to the user and, before generating the test scenario 12, receive correction information from the user as necessary.

[0080] As described above, in the information processing device 1 of the first embodiment, the communication IF 20 transmits a request for inferring how to call an API included in the API specification 11 to the specification interpretation device 2, which interprets the API specification 11 using natural language processing, and receives a response indicating how to call the API included in the API specification 11 from the specification interpretation device 2. The processing unit 10 then determines whether the API calling method indicated in the response is correct, and if the API calling method indicated in the response is correct, generates a test scenario 12 based on the API calling method indicated in the response.

[0081] As a result, the information processing device 1 of the first embodiment can automatically generate a more accurate test scenario 12 from the API specification 11 using natural language processing. Specifically, although uncertainty is an issue when utilizing natural language processing technology, even if a method of inferring an API call that differs from the intention of the creator of the API specification may be inferred, it is possible to generate an expected test scenario 12 (appropriate test scenario 12). This makes it possible, for example, to improve API quality with less effort.

[0082] For example, the processing unit 10 can also identify test items that check matters not described in the API specification based on the response to the API request. This allows the test scenario of the first embodiment to include test items that check matters not described in the API specification.

[0083] (Second embodiment) Next, a second embodiment will be described. In the description of the second embodiment, the same description as in the first embodiment will be omitted, and only the differences from the first embodiment will be described.

[0084] In the second embodiment, as in the first embodiment, an example will be described in which a test scenario 12 is generated to verify whether a response is appropriate when various inputs are given. In the test scenario 12 of the second embodiment, it is verified that the API returns a success status code when valid input is given, and returns a failure status code when invalid input is given. Furthermore, in the case of a failure, it is verified whether the content of the response body is as expected.

[0085] The second embodiment differs from the first embodiment in that a prompt is used to request the specification interpretation device 2 to generate a test scenario 12 based on the API calling method obtained by inference, and after corrections are made as necessary, the final test scenario 12 is generated.

[0086] [Example of functional configuration] 17 is a diagram illustrating an example of the functional configuration of an information processing device 1-2 according to the second embodiment. The information processing device 1-2 according to the second embodiment includes a processing unit 10 and a communication IF 20. The processing unit 10 includes a request unit 101, a determination unit 102, a trial unit 103, and a correction unit 105.

[0087] The request unit 101 receives the input of the API specification 11 and requests the specification interpretation device 2 via the communication IF 20 to infer how to call each API.

[0088] Fig. 18 is a diagram showing an example of the API specification 11 according to the second embodiment. In the example of Fig. 18, the following information is written as specifications in the OpenAPI format.

[0089] ------------------Specification example-- Create a new device Endpoint: / devices Method: POST Request Body: name (string): The name of the device. model_number (string): Model number. response: status: 201 Created 400 Bad Request Body (if successful): id (string): A unique identifier for the organization. name (string): The name of the organization. model_number (string): Model number. Body (failure): error (string): The error, either invalid_name or invalid_model_number. ------------------------------------

[0090] Next, the usage intended by the creator of the API specification is as follows:

[0091] ---Example of use--- This API is used to create a new device. / devices sends a request to the endpoint using the POST method. The request body contains the name and model number of the newly created device specified in JSON format as name and model_number. Example request: { ``name'': ``Air conditioner 1'', ”model_number”: ”RAS-N221” } If successful, the response will return status code 201 along with information about the newly created organization. Example response: { ”id”: ”abc123”, ``name'': ``Air conditioner 1'', ”model_number”: ”RAS-N221” } If an inappropriate parameter is specified, an error message will be returned along with status code 400. The error message will contain "invalid_name" if an inappropriate value is specified for "name", or "invalid_model_number" if an inappropriate value is specified for "model_number". { “error”: “invalid_name” } ------------------------------------

[0092] When the processing unit 10 receives the API specification 11 as input, it generates a test scenario 12 according to the following procedure. First, the request unit 101 passes the API specification 11 to the specification interpretation device 2 and requests that it infer how to call the API. The following is an example of a prompt generated by the request unit 101.

[0093] --Example prompt-- Below is a Web API specification written in OpenAPI format. Please generate a tavern scenario file that tests an API execution device implemented in accordance with this API specification. Please generate tests indicated in the test items below. Please output the test while strictly adhering to the constraints. # Test items - Test to ensure a successful status code is obtained. - Tests to verify that an error occurs when an invalid type is specified for each parameter of the API. - For each parameter that is flagged as required, test to verify that an error occurs if it is not specified. - For each parameter with upper and lower limits for the string length, a test is conducted to verify that an error occurs when a string with a length exceeding the upper limit is specified. A test is conducted to verify that an error occurs when a string with a length less than the lower limit is specified. # Constraints - Never output any text other than the tavern scenario file. - Tests that confirm an error occurs should also verify that the response status code and body content are as expected. # Web API specification openapi: 3.0.0 info: (The following is omitted as it is the same as Figure 18) ------------------------------------

[0094] In response to the prompt, the specification interpreter 2 returns a test scenario 12 based on the API calling method. Examples of the response are shown in Figures 19A and 19B.

[0095] 19A and 19B are diagrams showing examples of responses to prompts in the second embodiment. In the test scenario 12 in Figs. 19A and 19B, not only normal tests but also abnormal tests are described.

[0096] In the abnormal case test, it is confirmed that status code 400 is obtained when various patterns of invalid values ​​(specified by the prompt) are specified for each parameter, and that the error content in the response body includes the name of the parameter. For example, if an invalid value is specified for name, the error in the response body will be invalid_name.

[0097] The content of the response body of the abnormality test is not directly described in the API specification 11, and the specification interpretation device 2 infers this from the API specification 11 and includes the content of the response body of the abnormality test in the test scenario 12.

[0098] Returning to FIG. 17, the request unit 101 inputs the test scenario 12 (in tavern format) obtained as a response from the specification interpretation device 2 to the determination unit 102, thereby requesting that the determination unit 102 determine how to call the API.

[0099] The determination unit 102 actually executes the test scenario 12, and if the test is successful, determines that the specification interpretation indicated by the test scenario 12 input from the request unit 101 is a correct way to call the API. Specifically, the determination unit 102 passes the test scenario 12 to the trial unit 103 and requests that the test be performed. The trial unit 103 performs the test by sending an API request to the API execution device 3 in accordance with the test scenario 12, and generates a test result report.

[0100] When the test is completed, the determining unit 102 passes the test scenario 12 and the test result report generated by the trying unit 103 to the correcting unit 105 .

[0101] The correction unit 105 displays the test scenario 12 and the test result report on a display device or the like, and asks the user (e.g., API developer) of the information processing device 1-2 to confirm whether the determined specifications are as expected. In particular, for test scenarios 12 whose actual test execution results were failures, the user checks whether the contents of the test scenario 12 are as intended, and if they are different from the intention, inputs corrections to the test scenario 12.

[0102] After the user's confirmation, the corrector 105 outputs the final test scenario 12, and the processing of the processor 10 ends.

[0103] [Examples of information processing methods] 20 is a flowchart showing an example of an information processing method according to the second embodiment. First, the communication IF 20 transmits a request to the specification interpretation device 2, including a request to infer how to call an API included in the API specification 11 and to generate a test scenario (step S11). Next, the communication IF 20 receives a response from the specification interpretation device 2, including a test scenario based on how to call the API included in the API specification 11 (step S12). Next, the processing unit 10 attempts the API request by transmitting at least one API request to the API execution device 3 based on the test scenario indicated in the response received in step S12 (step S13).

[0104] Next, the correction unit 105 presents the test scenario and the test results to the user on a display device or the like (step S14).

[0105] Next, the processing unit 10 determines whether the API calling method (the test scenario based on the API calling method obtained as the inference result) is correct or not based on the trial result of step S13 (step S15). For example, if the test is successful, that is, if the expected result is obtained in the normal or abnormal test, the processing unit 10 determines that the API calling method indicated in the response of step S12 is correct.

[0106] If the API is not called correctly (step S15, No), the processing unit 10 accepts input of correction information for correcting the test scenario from the user via the input device (step S16).

[0107] If the API is called correctly (Yes at step S15), the processing unit 10 performs conversion into a predetermined data format and the like to generate the final test scenario 12 (step S17).

[0108] Note that test scenarios may be modified not only when the API is called incorrectly, but also when the API is determined to be called correctly. For example, test scenarios determined to be called correctly may be modified by adding more test item variations (for example, test items that test the range of parameter values ​​(such as upper and lower limit values)).

[0109] Finally, an example of the hardware configuration of the information processing device 1 (1-2) according to the first and second embodiments will be described.

[0110] [Example of hardware configuration] 21 is a diagram showing an example of the hardware configuration of the information processing device 1 (1-2) of the first and second embodiments. The information processing device 1 (1-2) of the first and second embodiments includes a processor 201, a main storage device 202, an auxiliary storage device 203, a display device 204, an input device 205, and a communication device 206. The processor 201, the main storage device 202, the auxiliary storage device 203, the display device 204, the input device 205, and the communication device 206 are connected via a bus 210.

[0111] The information processing device 1 (1-2) may not have some of the above configurations. For example, if the information processing device 1 (1-2) can use the input function and display function of an external device, the information processing device 1 (1-2) may not have the display device 204 and the input device 205.

[0112] The processor 201 executes a program read from the auxiliary storage device 203 to the main storage device 202. The main storage device 202 is a memory such as a ROM and a RAM. The auxiliary storage device 203 is a hard disk drive (HDD), a memory card, or the like.

[0113] The display device 204 is, for example, a liquid crystal display. The input device 205 is an interface for operating the information processing device 1. The display device 204 and the input device 205 may be realized by a touch panel or the like having a display function and an input function. The communication device 206 is an interface for communicating with other devices.

[0114] For example, a program executed by the information processing device 1 (1-2) is provided as a computer program product in the form of a file in an installable or executable format, recorded on a computer-readable storage medium such as a memory card, hard disk, CD-RW, CD-ROM, CD-R, DVD-RAM, or DVD-R.

[0115] Furthermore, for example, the program executed by the information processing device 1 (1-2) may be stored on a computer connected to a network such as the Internet, and may be provided by being downloaded via the network.

[0116] Also, for example, the program executed by the information processing device 1 (1-2) may be provided via a network such as the Internet without being downloaded. Specifically, the information processing may be performed by a so-called ASP (Application Service Provider) type service that realizes processing functions by issuing an execution instruction and obtaining the results, without transferring the program from a server computer.

[0117] Furthermore, for example, the program for the information processing device 1 (1-2) may be provided in a state where it is pre-installed in a ROM or the like.

[0118] The program executed by the information processing device 1 (1-2) has a modular configuration including functions that can be realized by the program among the above-mentioned functional configurations. As for each function, as actual hardware, the processor 201 reads the program from a storage medium and executes it, and the above-mentioned functional blocks are loaded onto the main memory device 202. In other words, the above-mentioned functional blocks are generated on the main memory device 202.

[0119] Note that some or all of the above-described functions may be realized by hardware such as an integrated circuit (IC) rather than by software.

[0120] Furthermore, each function may be realized using a plurality of processors 201, and in this case, each processor 201 may realize one of the functions, or may realize two or more of the functions.

[0121] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims.

[0122] (Addendum) The above-described embodiments can be summarized as the following technical proposals.

[0123] Technical proposal 1 a communication interface that transmits an inference request for an API calling method included in an API (Application Programming Interface) specification to a specification interpretation device that interprets the API specification by natural language processing, and receives a response from the specification interpretation device that indicates the API calling method included in the API specification; a processing unit that determines whether the API calling method indicated by the response is correct, and if the API calling method indicated by the response is correct, generates a test scenario based on the API calling method indicated by the response; An information processing device comprising: Technical proposal 2 the communication interface transmits at least one API request to an API execution device based on the API calling method indicated by the response; When the response to the API request includes a success status code, the processing unit determines that the API calling method indicated by the response is correct. An information processing device according to Technical Proposal 1. Technical proposal 3 the processing unit generates, as the test scenario for a normal system, a method of calling the API request in which a response to the API request includes a success status code, and generates the test scenario for an abnormal system based on the test scenario for the normal system. An information processing device according to Technical Proposal 2. Technical proposal 4 The processing unit also identifies test items for checking matters not described in the API specification based on the response to the API request, The test scenario includes test items for checking matters not described in the API specification. An information processing device according to Technical Proposal 2. Technical proposal 5 The inference request includes a request to present a plurality of candidates indicating how to call the API. An information processing device according to any one of technical proposals 1 to 4. Technical proposal 6 The processing unit displays information indicating the API calling method determined to be correct on a display device, and corrects the API calling method based on correction information input by a user. An information processing device according to any one of technical proposals 1 to 5. Technical proposal 7 the communication interface transmits learning data including information indicating the method of calling the API that has been determined to be correct to the specification interpretation device; An information processing device according to any one of technical proposals 1 to 6. Technical proposal 8 an information processing device transmits an inference request for an API calling method included in an API (Application Programming Interface) specification to a specification interpretation device that interprets the API specification by natural language processing, and receives a response from the specification interpretation device indicating the API calling method included in the API specification; the information processing device determines whether the API calling method indicated by the response is correct, and if the API calling method indicated by the response is correct, generates a test scenario based on the API calling method indicated by the response. Information processing methods. Technical proposal 9 On the computer, transmitting a request for inferring how to call an API (Application Programming Interface) included in an API specification to a specification interpretation device that interprets the API specification by natural language processing, and receiving a response from the specification interpretation device indicating how to call the API included in the API specification; determining whether the API calling method indicated by the response is correct, and if the API calling method indicated by the response is correct, generating a test scenario based on the API calling method indicated by the response; program. [Explanation of symbols]

[0124] 1. Information processing equipment 2. Specification interpretation device 3 API execution device 10 Processing section 20 Communication Interface 101 Request Department 102 Judgment section 103 Trial Department 104 Generation part 105 Correction section 106 Transmission control section 201 processor 202 Main storage 203 Auxiliary storage device 204 Display device 205 Input Device 206 Communication Equipment 210 Bus

Claims

1. a communication interface that transmits an inference request for an API calling method included in an API (Application Programming Interface) specification to a specification interpretation device that interprets the API specification by natural language processing, and receives a response from the specification interpretation device indicating the API calling method included in the API specification; a processing unit that determines whether the API calling method indicated by the response is correct, and if the API calling method indicated by the response is correct, generates a test scenario based on the API calling method indicated by the response; An information processing device comprising:

2. the communication interface transmits at least one API request to an API execution device based on the API calling method indicated by the response; When the response to the API request includes a success status code, the processing unit determines that the API calling method indicated by the response is correct. The information processing device according to claim 1 .

3. the processing unit generates, as the test scenario for a normal system, a method of calling the API request in which a response to the API request includes a success status code, and generates the test scenario for an abnormal system based on the test scenario for the normal system. The information processing device according to claim 2 .

4. The processing unit also identifies test items for checking matters not described in the API specification based on the response to the API request, The test scenario includes a test item for checking matters not described in the API specification. The information processing device according to claim 2 .

5. The inference request includes a request for presentation of a plurality of candidates indicating how to call the API. The information processing device according to claim 1 .

6. The processing unit displays information indicating the method of calling the API that has been determined to be correct on a display device, and corrects the method of calling the API based on correction information input by a user. The information processing device according to claim 1 .

7. the communication interface transmits learning data including information indicating the method of calling the API that has been determined to be correct to the specification interpretation device; The information processing device according to claim 1 .

8. an information processing device transmits an inference request for an API calling method included in an API (Application Programming Interface) specification to a specification interpretation device that interprets the API specification by natural language processing, and receives a response indicating an API calling method included in the API specification from the specification interpretation device; the information processing device determines whether the API calling method indicated by the response is correct, and if the API calling method indicated by the response is correct, generates a test scenario based on the API calling method indicated by the response. Information processing methods.

9. On the computer, transmitting a request for inferring how to call an API (Application Programming Interface) included in an API specification to a specification interpretation device that interprets the API specification by natural language processing, and receiving a response indicating how to call the API included in the API specification from the specification interpretation device; determining whether the API calling method indicated by the response is correct, and if the API calling method indicated by the response is correct, generating a test scenario based on the API calling method indicated by the response; program.

Images