Generation device, generation method, and generation program

Abstract

To automatically set an appropriate determination rule in a test case regardless of the number of combinations of attribute values.SOLUTION: The generation device includes a request extraction unit that extracts a request from a test case including the request to be transmitted to a system to be tested and a determination rule for determining pass or fail of a test, a test execution unit that executes a test on the system to be tested by transmitting the extracted request to the system to be tested and receives a response to the request, a determination rule extraction unit that extracts the determination rule from the test case, a test result determination unit that determines pass or fail of the executed test based on the request, the response, and the determination rule, and a new rule derivation unit that generates a new rule as a determination rule to be newly applied to the test case based on the request, the response, the determination rule, and the pass or fail of the test determined by the test result determination unit.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a generation device, a generation method, and a generation program. [Background technology]

[0002] Test cases for software testing describe what tests will be performed and the conditions for the results that will be obtained by executing those tests. When testing software, the tests described in the test cases are executed, and the pass / fail of the test is determined based on whether the execution results meet the conditions. In particular, in the case of functional testing of Application Programming Interfaces (APIs), the pass / fail of the test is determined by determining whether the API request sent to the API being tested and the API response returned from the API being tested meet pre-set judgment rules.

[0003] However, depending on the software development environment, the conditions written in test cases may be insufficient or may not be what the developer truly intended. For example, while the number of digits in phone numbers and credit card numbers is fixed, test cases may only specify whether the number is an arbitrary string due to undefined API requirements. Another example is a case where a certain attribute value was initially expected to be 10-digit alphanumeric characters, but the requirements were later changed to require only 10-digit numbers. However, the test case remains unchanged, and 10-digit alphanumeric characters are still written as the condition. Such test cases may not perform boundary value testing properly, potentially failing to detect software defects. However, manually examining the conditions of a massive number of test cases requires a significant amount of effort, so technology is needed to automatically set appropriate judgment rules for test cases.

[0004] Patent Document 1 describes a method for generating discrimination conditions for positive and negative examples by processing case data consisting of multiple attribute values ​​and discrimination results (positive or negative examples) based on the attribute values, setting the number of positive and negative examples, and setting whether or not a combination of attribute values ​​is necessary. This makes it possible to provide a method for generating discrimination conditions that allows discrimination conditions to be easily generated by applying learning from case data in the field, performs discrimination processing using the discrimination conditions, and can immediately correct the discrimination conditions if an error occurs in the discrimination. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 6-68066 Summary of the Invention [Problem to be solved by the invention]

[0006] However, when there are a huge number of combinations of multiple attribute values, there is a problem in that the method disclosed in Patent Document 1 cannot be applied. For example, in API function testing, there are often a huge number of combinations of attribute values ​​for API requests and API responses.

[0007] The present invention has been made in consideration of these circumstances, and its purpose is to provide a generation device, a generation method, and a generation program that can automatically set appropriate judgment rules for test cases regardless of the number of combinations of attribute values. [Means for solving the problem]

[0008] One aspect of the present invention for solving the above problem is a generation device comprising: a request extraction unit that extracts a request from a test case that includes a request to be sent to a system under test and a judgment rule for judging whether the test has passed or failed; a test execution unit that executes a test on the system under test by sending the extracted request to the system under test and receives a response to the request; a judgment rule extraction unit that extracts the judgment rule from the test case; a test result judgment unit that judges whether the executed test has passed or failed based on the extracted request, the received response, and the extracted judgment rule; and a new rule derivation unit that generates a new rule as a judgment rule to be newly applied to the test case based on the extracted request, the received response, the extracted judgment rule, and the pass or fail of the test judged by the test result judgment unit. [Effects of the Invention]

[0009] According to the present invention, it is possible to automatically set appropriate judgment rules for test cases regardless of the number of combinations of attribute values. Configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 2 illustrates an example of a hardware configuration of a software testing device. [Figure 2] 1 is a diagram showing functions of a software testing device, data stored in the software testing device, and inputs and outputs to and from the software testing device in the first embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of a test history table. [Figure 4] 10 is a flowchart illustrating an outline of an execution result recording process executed by the software testing device. [Figure 5] 10 is a flowchart illustrating an example of an API request extraction process. [Figure 6] 10 is a flowchart illustrating an example of a test execution process. [Figure 7] 10 is a flowchart illustrating an example of a determination rule extraction process. [Figure 8] 10 is a flowchart illustrating an example of a test execution process. [Figure 9] 10 is a flowchart illustrating an example of a new rule derivation process in the first embodiment. [Figure 10] 10 is a flowchart illustrating an example of an adoption rule determination process. [Figure 11] 10A to 10C are diagrams illustrating an example of screen transitions in the adoption rule determination process. [Figure 12] FIG. 10 is a diagram illustrating an example of a new rule list screen. [Figure 13] FIG. 10 is a diagram illustrating an example of a new rule confirmation screen. [Figure 14] FIG. 10 is a diagram illustrating an example of a new rule editing screen. [Figure 15] 10 is a flowchart illustrating an example of a test case generation process. [Figure 16] FIG. 10 is a diagram showing functions of a software testing device, data stored in the software testing device, and inputs and outputs to and from the software testing device in the second embodiment. [Figure 17] 10 is a flowchart illustrating an example of a new rule derivation process in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. The following description and drawings are examples for explaining the present invention, and some omissions and simplifications have been made as appropriate for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural. In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings. In the following explanation, various types of information may be described using expressions such as "table," "list," and "queue," but the various types of information may also be expressed using data structures other than these. To indicate that the information is not dependent on the data structure, "XX table," "XX list," etc. may be referred to as "XX information." When describing identification information, expressions such as "identification information," "identifier," "name," "ID," and "number" are used, but these are interchangeable. When there are multiple components with the same or similar functions, they may be described using the same reference numeral with different subscripts. However, when there is no need to distinguish between these multiple components, the subscripts may be omitted. Furthermore, in the following description, an example of processing performed by executing a program may be described, but the program is executed by a processor (e.g., a CPU or a GPU) to perform the specified processing while appropriately using storage resources (e.g., a memory) and / or interface devices (e.g., a communication port), and therefore the subject of the processing may be the processor. Similarly, the subject of the processing performed by executing a program may be a controller, device, system, computer, or node having a processor. The subject of the processing performed by executing a program may be any computing unit, and may include a dedicated circuit (e.g., an FPGA or an ASIC) that performs a specific processing. A program may be installed on a device such as a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. If the program source is a program distribution server, the program distribution server may include a processor and storage resources for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. Also, in the following description, two or more programs may be realized as one program, and one program may be realized as two or more programs.

[0012] [First embodiment] First, a software testing device (generation device), a software testing method (generation method), and a software testing program (generation program) according to a first embodiment of the present invention will be described.

[0013] <Software testing equipment> 1 is a diagram showing an example of the hardware configuration of a software testing device 100. The software testing device 100 is an information processing device used for the development or maintenance of a software system. The software testing device 100 may be realized virtually, such as a cloud server provided by a cloud system.

[0014] 1, the software testing device 100 includes a processor 101, a main memory device 102, an auxiliary memory device 103, an input device 104, an output device 105, and a communication device 106. These devices are capable of communicating with each other via a communication means such as a bus (not shown).

[0015] The processor 101 is configured using, for example, a central processing unit (CPU) or a micro processing unit (MPU). The processor 101 reads and executes a program stored in the main memory device 102, thereby realizing various functions of the software testing device 100.

[0016] The main memory device 102 is a device that stores programs and data, and is configured using, for example, a random access memory (RAM), a read only memory (ROM), or a non-volatile semiconductor memory (Non-Volatile RAM (NVRAM)).

[0017] The auxiliary storage device 103 is a device for reading and writing recording media or a storage area of ​​a cloud server, and is configured using, for example, a Hard Disc Drive (HDD), a Solid State Drive (SSD), a storage system, an Integrated Circuit (IC) card, a Secure Digital (SD) memory card, or an optical recording medium (such as a Compact Disc (CD) or a Digital Versatile Disc (DVD)).

[0018] The input device 104 is a user interface through which the user inputs various information to the device, and is configured using, for example, a keyboard, a mouse, a touch panel, a card reader, and a voice input device.

[0019] The output device 105 is a user interface that provides the user with various information such as processing progress and processing results, and is composed of a screen display device (e.g., a Liquid Crustal Display (LCD) or a Head Mounted Display (HMD)), an audio output device, and a printing device.

[0020] The software testing device 100 may be configured to input and output information to and from other devices via the communication device 106.

[0021] The communication device 106 is a wired or wireless communication interface that enables communication with other devices via communication means such as a local area network (LAN) or the Internet, and is configured using, for example, a network interface card (NIC), a wireless communication module, a universal serial interface (USB) module, or a serial communication module.

[0022] 2 is a diagram showing the functions of the software testing device 100 in the first embodiment, the data stored in the software testing device 100, and input / output to / from the software testing device 100. The software testing device 100 is capable of communicating with a database that stores one or more test cases 208 and an API 210 to be tested via a communication device 106.

[0023] The API under test 210 is the API that is the subject of an API test (software test) and the external system in which the API is implemented. The API test is, for example, a test that verifies the functionality of the API. The API under test 210 has the ability to receive an API request 209 and return an API response 211 in response to the received API request 209.

[0024] The API type and protocol may be any type including gPRC and SOAP, but in this embodiment, a REST (Representational State Transfer) API using HyperText Transfer Protocol (HTTP) will be described as an example.

[0025] The API request 209 is data that describes header and payload information and its history, and is sent to the test target API 210 when the test target API 210 is called.

[0026] The API response 211 is data that is returned from the test target API 210 when the test target API 210 is called, and that stores information about the header and payload, as well as the history of the information.

[0027] Test case 208 is data that contains the contents of an API test for test target API 210, which is executed by software test device 100. Test case 208 includes API request 209 to be sent to test target API 210, and determination rule 212 that describes the conditions for determining whether the test is passed or failed.

[0028] The determination rule 212 is data describing the conditions used to determine whether the API request 209 and the API response 211 pass the test case 208 .

[0029] Note that the test case 208 may include multiple API requests 209. For example, the test case 208 may include multiple API requests 209 for the same API, but with different values ​​of variables included in the API requests 209.

[0030] The software testing device 100 includes an API request extraction unit 201, a test execution unit 202, a determination rule extraction unit 203, a test result determination unit 204, a new rule derivation unit 205, a test case generation unit 206, an information storage unit 207, and an adoption rule determination unit 216.

[0031] These functions are realized, for example, by the processor 101 reading and executing a program stored in the main memory device 102 or the auxiliary memory device 103. These functions may also be realized, for example, by hardware (such as an Application Specific Integrated Circuit (ASIC)) included in the software testing device 100.

[0032] The API request extraction unit 201 receives an input of a test case 208 and extracts an API request 209 from the received test case 208 .

[0033] The test execution unit 202 calls the test target API 210 by sending the extracted API request 209 to the test target API 210 and receives an API response 211 from the test target API 210 .

[0034] The determination rule extraction unit 203 receives the input of the test case 208 and extracts the determination rule 212 from the received test case 208 .

[0035] The test result determination unit 204 generates a test result 213 based on the API request 209 , the API response 211 , and the determination rule 212 .

[0036] The test result 213 is data generated by the test result determination unit 204 and stores the result of determining whether the API request 209 and the API response 211 pass or fail the test case 208 .

[0037] The new rule derivation unit 205 generates a new rule 214 based on the API request 209 , the API response 211 , the determination rule 212 , and the test result 213 .

[0038] The new rule 214 is a new condition used when determining whether the API request 209 and the API response 211 pass the test case 208. For example, the new rule 214 describes a condition different from the condition described in the determination rule 212.

[0039] The adopted rule determination unit 216 receives from the user of the software testing device 100 whether the new rule 214 is adopted or not, and updates the judgment rule 212 to the adopted new rule 214. For example, the user refers to the new rule 214 on a screen or the like displayed by the adopted rule determination unit 216 and determines the new rule 214 to be adopted.

[0040] The test case generation unit 206 receives an input of the threshold 215 and generates a new test case 208 based on the threshold 215 , the API request 209 , the API response 211 , and the judgment rule 212 .

[0041] The threshold value 215 is data that stores a value used when determining the number of newly created test cases in the test case generation unit 206. For example, the threshold value 215 may be set to an upper limit of "50" for the number of test cases or a lower limit of "10" for the number of test cases.

[0042] The programs that realize the functions of the API request extraction unit 201, test execution unit 202, judgment rule extraction unit 203, test result judgment unit 204, new rule derivation unit 205, test case generation unit 206, and adopted rule determination unit 216 may be pre-recorded on an external storage medium, or may be installed when necessary via a predetermined communication network. Also, the programs that realize the functions of the API request extraction unit 201, test execution unit 202, judgment rule extraction unit 203, test result judgment unit 204, new rule derivation unit 205, test case generation unit 206, and adopted rule determination unit 216 may be recorded on, for example, a portable or fixed recording medium and distributed.

[0043] The information storage unit 207 stores information that is referenced or generated as appropriate by the API request extraction unit 201, test execution unit 202, judgment rule extraction unit 203, test result judgment unit 204, new rule derivation unit 205, test case generation unit 206, or adoption rule determination unit 216. The information storage unit 207 manages the information stored in the auxiliary storage device 103, for example, by using a file system or a database management system (DBMS). The information storage unit 207 stores, for example, a test history table that accumulates the history of execution results of tests.

[0044] (Test History Table) FIG. 3 is a diagram showing an example of a test history table. The test history table 300 includes a first item 301 in which the identification information of the test case 208 is set, a second item 302 in which the execution date and time of the test is set, a third item 303 in which the API request 209 used in the test is set, a fourth item 304 in which the API response 211 for the API request 209 is set, a fifth item 305 in which the determination rule 212 of the test case 208 is set, and a database having each data item of a sixth item 306 in which the test result 213 is set.

[0045] In the data example shown in FIG. 3, the history of the execution results of executing N test cases of test case #1 to test case #N once a week is accumulated in the test history table 300.

[0046] Next, the processing performed by the software test apparatus 100 will be described.

[0047] <Execution result recording process> FIG. 4 is a flowchart for explaining an outline of an execution result recording process executed by the software test apparatus in the first embodiment. The process shown in this figure is executed, for example, when a predetermined input is given from a user to the software test apparatus 100, or at a predetermined timing (for example, a predetermined time, a predetermined time interval). For example, the software test apparatus 100 may execute the execution result recording process periodically, such as once a week.

[0048] The software test apparatus 100 executes the following processes S2 to S5 for all the test cases 208 (S1).

[0049] That is, first, the API request extraction unit 201 executes an API request extraction process for extracting the API request 209 from the test case 208 (S2).

[0050] <API request extraction process> 5 is a flowchart illustrating an example of the API request extraction process. The process shown in this figure is a detailed description of the process in S2 described above.

[0051] First, the API request extraction unit 201 extracts the API request 209 written in the test case 208 (S201).

[0052] Next, the API request extraction unit 201 stores the extracted API request 209 in the test history table 300 in association with the test case 208 (S202). After that, the API request extraction unit 201 ends the API request extraction process.

[0053] Following the API request extraction process, the test execution unit 202 executes a test execution process to execute a test on the test target API 210 (S3).

[0054] <Test execution process> 6 is a flowchart illustrating an example of the test execution process. The process shown in this figure is a detailed description of the process in S3 described above.

[0055] The test execution unit 202 executes the following steps S302 to S304 for all API requests 209 (S301).

[0056] That is, first, the test execution unit 202 sends the API request 209 extracted in the API request extraction process to the test target API 210 (S302).

[0057] Next, the test execution unit 202 receives an API response 211 in response to the API request 209 sent in S302 from the test target API 210 (S303).

[0058] Next, the test execution unit 202 stores the API response 211 received in S303 in the test history table 300 in association with the API request 209 sent in S302 (S304).

[0059] When the test execution unit 202 has executed the processes of S302 to S304 for all API requests 209 (S305), the test execution process ends.

[0060] Following the test execution process, the determination rule extraction unit 203 executes a determination rule extraction process to extract a determination rule 212 from the test case 208 (S4).

[0061] <Decision rule extraction process> 7 is a flowchart illustrating an example of the determination rule extraction process, which shows details of the process in S4 described above.

[0062] First, the determination rule extraction unit 203 extracts the determination rule 212 written in the test case 208 (S401).

[0063] Next, the determination rule extraction unit 203 stores the extracted determination rule 212 in the test history table 300 in association with the test case 208 (S402). After that, the determination rule extraction unit 203 ends the determination rule extraction process.

[0064] Following the determination rule extraction process, the test result determination unit 204 executes a test result determination process for determining the test results (S5).

[0065] <Test result determination process> 8 is a flowchart illustrating an example of the test execution process, which shows details of the process in S5 described above.

[0066] First, the test result determination unit 204 reads the API request 209 corresponding to the test case 208 from the test history table 300 (S501).

[0067] Next, the test result determination unit 204 reads the API response 211 corresponding to the test case 208 from the test history table 300 (S502).

[0068] Next, the test result judgment unit 204 reads the judgment rule 212 corresponding to the test case 208 from the test history table 300 (S503).

[0069] Next, the test result determination unit 204 determines whether the read API request 209 and API response 211 satisfy the conditions described in the read determination rule 212 (S504).

[0070] If the condition is met (S504: Yes), the test result judging unit 204 judges the test result 213 corresponding to the test case 208 as passed and stores it in the test history table 300 (S505). After that, the test result judging unit 204 ends the test result judging process.

[0071] On the other hand, if the condition is not satisfied (S504: No), the test result determination unit 204 determines that the test result 213 corresponding to the test case 208 has failed, and stores this in the test history table 300 (S506). After that, the test result determination unit 204 ends the test result determination process.

[0072] However, the order of the above-mentioned steps S501, S502, and S503 does not matter.

[0073] The software test device 100 ends the execution result recording process when it has executed the processes of S2 to S5 (S6) for all test cases 208. However, the process of S4 described above may be executed before the process of S2 or S3.

[0074] <New rule derivation process> 9 is a flowchart illustrating an example of a new rule derivation process in the first embodiment. The process shown in this drawing is executed after the execution result recording process has been executed at least once.

[0075] The new rule derivation unit 205 executes the following processes of S702 to S710 for all test cases 208 (S701).

[0076] That is, first, the new rule derivation unit 205 reads the history of the API request 209 corresponding to the test case 208 from the test history table 300 (S702).

[0077] Next, the new rule derivation unit 205 reads the history of the API response 211 corresponding to the test case 208 from the test history table 300 (S703).

[0078] Next, the new rule derivation unit 205 reads the judgment rule 212 corresponding to the test case 208 from the test history table 300 (S704).

[0079] Next, the new rule derivation unit 205 reads the history of the test result 213 corresponding to the test case 208 from the test history table 300 (S705).

[0080] Next, the new rule derivation unit 205 extracts test results 213 that are pass from the history of the read test results 213 (S706).

[0081] Next, the new rule derivation unit 205 extracts the API request 209 and the API response 211 corresponding to the extracted test result 213 (S707).

[0082] Next, the new rule derivation unit 205 derives conditions that are satisfied for all API requests 209 extracted in S707 (S708). Any method may be used to derive the conditions, and examples include generating regular expressions using pattern analysis. One example of a condition is a condition for the value of a variable included in the API response 211. For example, a condition (possible range or value pattern (regularity)) is derived, such as that the value of a predetermined variable A is always a fixed value (a value that should match every time) or that varies but falls within a certain range (e.g., six digits, odd numbers, even numbers, etc.). That is, the new rule derivation unit 205 analyzes the value of the API request 209 to estimate the possible range (acceptable range) or value (expected value) pattern of the value of the API request 209.

[0083] Next, the new rule derivation unit 205 derives conditions that are satisfied for all of the API responses 211 extracted in S707 (S709). Any method may be used to derive the conditions, and examples include generating regular expressions through pattern analysis. As with the API request 209, the conditions may be conditions on the values ​​of variables included in the API response 211. The new rule derivation unit 205 may also derive conditions by comparing the values ​​of variables in the API request 209 with the values ​​of variables in the API response 211. For example, a condition such as "the value of a predetermined variable B is the same as the value of variable B included in the API request 209" is derived. In other words, the new rule derivation unit 205 analyzes the values ​​of the API response 211 to estimate a possible range (allowable range) or a pattern of values ​​(expected values) that the values ​​of the API response 211 can take. At this time, the new rule derivation unit 205 may estimate a possible range (acceptable range) or a pattern of values ​​(expected values) of the API response 211 for each value or range of the API request 209, or may estimate a possible range (acceptable range) or a pattern of values ​​(expected values) of the API response 211 for all values ​​of the API request 209.

[0084] Next, the new rule derivation unit 205 changes the parts of the judgment rule 212 that relate to the conditions for the API request 209 and the conditions for the API response 211 to the conditions for the API request 209 generated in the processing of S708 and the conditions for the API response 211 generated in the processing of S709, respectively, and generates a new rule 214 (S710).

[0085] The new rule derivation unit 205 completes the new rule derivation process when it has executed the processes of S702 to S710 for all test cases 208 (S711). However, the order of the processes of S702, S703, S704, and S705 does not matter. Also, the order of the processes of S708 and S709 does not matter.

[0086] In the above-described flow, the new rule deriving unit 205 generates the new rule 214 for one test case 208 based on the history of that test case 208. However, this is not limiting, and the new rule deriving unit 205 may also generate the new rule 214 by referring to the history of other test cases 208. For example, the new rule deriving unit 205 may generate the new rule 214 for the first test case 208 by including the history of a second test case 208 that includes a second API request 209 that has a variable with the same attribute as a first API request 209 included in the first test case 208 but with a different value for that variable. For example, it is assumed that the first API request 209 and the second API request 209 have a variable A with the same attribute, and the value of the variable A in the first API request 209 is "1" and the value of the variable A in the second API request 209 is "2."

[0087] <Recruitment rule determination process> 10 is a flowchart illustrating an example of the adopted rule determination process, which is executed at least after the new rule derivation process.

[0088] The adopted rule determination unit 216 executes the following processes of S902 to S905 for all new rules 214 (S901).

[0089] That is, first, the adoption rule determination unit 216 displays a screen for confirming the content of the new rule N (S902). The screen may be displayed on a display device provided in the software testing device 100, or on a user terminal used by a user who manages the test cases (the user of the software testing device 100). The user confirms the content of the new rule 214 based on the displayed screen.

[0090] The adopted rule determination unit 216 receives an input as to whether or not editing (modification) of the new rule 214 is necessary (S903). The user determines whether or not the content of the new rule 214 is what the software developer intended. If editing of the new rule 214 is not necessary (S903: No), the adopted rule determination unit 216 proceeds to the processing of S905.

[0091] On the other hand, if the new rule 214 needs to be edited (S904: Yes), the adopted rule determination unit 216 accepts input for editing the new rule 214 (S904). The user edits the content of the new rule 214. The adopted rule determination unit 216 updates the new rule 214 to the content edited by the user.

[0092] Next, the adopted rule determination unit 216 changes the determination rule 212 corresponding to the new rule 214 to the new rule 214 (S905).

[0093] When the adopted rule determination unit 216 has executed the processes of S902 to S905 for all the new rules 214 (S906), it ends the adopted rule determination process.

[0094] (Screen transition) 11 is a diagram showing an example of screen transitions in the adopted rule determination process. The screen configuration in the adopted rule determination process consists of a new rule list screen 1001, a new rule confirmation screen 1002, and a new rule edit screen 1003. Bidirectional transitions are possible between the new rule list screen 1001 and the new rule confirmation screen 1002. Bidirectional transitions are also possible between the new rule confirmation screen 1002 and the new rule edit screen 1003. It is also possible to transition from the new rule edit screen 1003 to the new rule list screen 1001.

[0095] (New rules list screen) 12 is a diagram showing an example of a new rule list screen 1001. The new rule list screen 1001 is configured with a new rule list tab 1101. The new rule list tab 1101 displays, in a tabular format, a list of test cases 208 corresponding to the new rule 214, data indicating whether the new rule 214 has been edited ("edited" or "unedited"), and a "view details" button 1102. When the user presses the "view details" button 1102, the adopted rule determination unit 216 transitions the display to the new rule confirmation screen 1002.

[0096] (New rule confirmation screen) 13 is a diagram showing an example of a new rule confirmation screen 1002. The new rule confirmation screen 1002 is composed of a confirmation target rule tab 1201, a start editing button 1202, and a return to list button 1203. The confirmation target rule tab 1201 includes a request confirmation table 1211 that displays a list of new rules for the API request 209, and a response confirmation table 1212 that displays a list of new rules for the API response 211. The request confirmation table 1211 displays a list of the request content of the API request 209, the current determination rule 212, and the new rule 214 in a tabular format. The request content is, for example, the properties (e.g., attributes) of each variable included in the API request 209, and the determination rule 212 and the new rule 214 are property conditions. The response confirmation table 1212 displays a list of the response content of the API response 211, the current determination rule 212, and the new rule 214 in a tabular format. The response content is, for example, the properties (e.g., attributes) of the variables included in the API response 211, and the determination rule 212 and new rule 214 are property conditions. When the user presses a button 1202 to start editing, the adopted rule determination unit 216 transitions the display to a new rule editing screen 1003. When the user presses a button 1203 to return to list, the adopted rule determination unit 216 transitions the display to a new rule list screen 1001.

[0097] (New rule editing screen) 14 is a diagram showing an example of the new rule editing screen 1003. The new rule editing screen 1003 is composed of a rule to be edited tab 1301, a button 1302 for ending editing, and a return button 1303 for returning without editing. The rule to be edited tab 1301 includes a request editing table 1311 for accepting editing of the judgment rule 212 of the API request 209, and a response editing table 1312 for accepting editing of the judgment rule of the API response 211. The request editing table 1311 displays, in tabular form, the request content, the current judgment rule 212, the new rule 214, and a text box 1304 for editing the condition (determination rule 212) of the API request 209. The user can enter any condition for the corresponding property in the text box 1304. The response editing table 1312 displays in tabular form a list of the response content, the current judgment rule 212, the new rule 214, and a text box 1304 for editing the conditions (judgment rule 212) of the API response 211. When the user presses a button 1302 for ending editing, the adopted rule determination unit 216 transitions the display to the new rule list screen 1001 and changes the new rule 214 and the judgment rule 212 to the values ​​entered in the text box 1304. When the user presses a return without editing button 1303, the adopted rule determination unit 216 transitions the display to the new rule confirmation screen 1002 without storing the values ​​entered in the text box 1304.

[0098] Such a user interface can realize input and output of the adoption rule determination process in the first embodiment and the second embodiment described later, allowing the user to check and modify the processing contents of the software testing device 100.

[0099] <Test case generation process> 15 is a flowchart illustrating an example of the test case generation process. The process shown in this figure is executed at least after the adopted rule determination process. In other words, the process shown in this figure is executed after the judgment rule 212 of the test case 208 is updated to the new rule 214.

[0100] The test case generation unit 206 executes the following processes of S802 to S808 for all test cases T (S801).

[0101] That is, first, the test case generation unit 206 reads the threshold 215 (S802).

[0102] Next, the test case generation unit 206 calculates the number X of test cases to be newly created based on the read threshold value 215 (S802). Any method for calculating X may be used.

[0103] Next, the test case generation unit 206 repeats the following processes of S805 to S807 X times (S804).

[0104] That is, first, the test case generation unit 206 duplicates the test case 208 (S805).

[0105] Next, the test case generation unit 206 changes some of the variable values ​​of the API request 209 included in the copied test case 208 (S806). As a method for changing some of the values ​​of the API request 209, for example, a technique used in mutation analysis is known. For example, the test case generation unit 206 may refer to the updated determination rule 212 (new rule 214) to generate an API request 209 that can be subjected to boundary value testing.

[0106] Next, the test case generation unit 206 stores the test case 208 with the modified API request 209 as a new test case in the database that stores the test case 208 (S807).

[0107] The test case generation unit 206 repeats the processes of S805 to S807 X times (S808).

[0108] Then, when the test case generation unit 206 has executed the processes of S802 to S808 for all test cases 208 (S809), it ends the test case generation process.

[0109] In this way, by generating new test cases 208 based on the updated decision rules 212, it is possible to automatically generate test cases 208 that are capable of boundary value testing, without manual intervention. By generating test cases 208 that are capable of boundary value testing, it is possible to perform tests that provide more depth to the boundary portions.

[0110] As described above, the software testing device 100 of this embodiment includes an API request extraction unit 201 that extracts a request from a test case 208 that includes a request (API request 209) to be sent to a system under test (API under test 210) and a judgment rule 212 for judging whether the test passes or fails; a test execution unit 202 that executes a test on the system under test by sending the request to the system under test and receives a response to the request (API response 211); a judgment rule extraction unit 203 that extracts the judgment rule 212 from the test case 208; a test result judgment unit 204 that judges whether the test passes or fails based on the request, the response, and the judgment rule 212; and a new rule derivation unit 205 that generates a new rule 214 as a judgment rule to be newly applied to the test case 208 based on the request, the response, the judgment rule, and the judged test result 213.

[0111] That is, the software testing device 100 of this embodiment automatically generates a new rule 214 for a test case 208 based on test results 213 of an actual test performed on an API under test 210. This allows the new rule 214 to be generated based on the values ​​of an API request 209 and an API response 211 obtained as a result of actually executing the test. Therefore, it is possible to automatically set an appropriate judgment rule for a test case regardless of the number of combinations of attribute values.

[0112] Furthermore, the software testing device 100 of this embodiment analyzes the response value to estimate the range or value pattern that the response value can take, and generates a new rule that represents the estimated range or value pattern.

[0113] This allows the expected or acceptable range of response values ​​to be efficiently estimated from the test results.

[0114] Furthermore, the software testing device 100 of this embodiment estimates the range or value pattern that a response value can take for each request value or range, and generates a new rule that represents the estimated range or value pattern.

[0115] This allows for efficient estimation of expected values ​​or acceptable ranges of response values ​​appropriate for each request value or range.

[0116] In addition, the software testing device 100 of this embodiment estimates the range or value pattern of possible response values ​​for all request values ​​in the history of multiple executions of test case tests, and generates new rules that represent the estimated range or value pattern.

[0117] This allows for an efficient estimation of expected or acceptable ranges of matching response values ​​for all requests.

[0118] Furthermore, the software testing device 100 of this embodiment extracts requests and responses for which the test results are pass from the history of multiple executions of the test case tests, and generates new rules 214 based on the extracted requests and responses.

[0119] That is, the software testing device 100 of this embodiment generates the new rule 214 based only on the values ​​of the API request 209 and the API response 211 for which the test result is a pass after executing the test multiple times, and therefore can generate the new rule 214 that is more suited to the implementation of the API 210 to be tested.

[0120] In addition, the software testing device 100 of this embodiment estimates the conditions for the variable values ​​included in a request or response based on the variable values ​​of a request or response whose test result is pass, and generates a new rule that includes the estimated conditions.

[0121] This allows the conditions for the variable values ​​included in the API request 209 or the API response 211 to be set to be more suited to the implementation of the API 210 under test.

[0122] Furthermore, the software testing device 100 of this embodiment generates a new test case 208 by partially changing the variable values ​​of the request of the test case 208 based on the new rule 214 .

[0123] This makes it possible to generate new test cases 208 that comply with the new rules 214. For example, it is possible to generate test cases 208 that allow boundary value testing.

[0124] In addition, the software testing device 100 of this embodiment displays the generated new rule 214, accepts modifications to the displayed new rule 214 from the user, and uses the modified new rule 214 as a new judgment rule to be applied to the test case 208.

[0125] This allows the user to inspect and modify the new rules 214 that are applied to the test cases 208 .

[0126] [Second embodiment] Next, a software testing device, a software testing method, and a software testing program according to a second embodiment of the present invention will be described.

[0127] 16 is a diagram showing functions of the software testing device 100A according to the second embodiment, data stored in the software testing device 100A, and input / output to / from the software testing device 100A. It is a diagram showing an example of the functional configuration of the software testing device 100A. In this diagram, the same components as those in the software testing device 100 according to the first embodiment shown in FIG. 1 are denoted by the same reference numerals, and a description thereof will be omitted. Furthermore, the hardware configuration of the software testing device 100A is the same as that of the first embodiment, and a description thereof will be omitted.

[0128] This embodiment differs from the first embodiment in that a knowledge database 1401 is used in new rule derivation unit 205A to narrow down the conditions of new rules 214 related to API requests 209 and API responses 211.

[0129] As shown in FIG. 16, the new rule derivation unit 205A receives as input the API request 209, the API response 211, the decision rule 212, the test result 213, and also information stored in the knowledge database 1401.

[0130] The knowledge database 1401 is a database that stores various attribute names and knowledge about the conditions that the attributes must satisfy. For example, the knowledge database 1401 can store information about knowledge such as that postal codes in Japan are seven-digit numbers, that dates are numbers between 1 and 31, that currency codes used in financial systems and the like are three-letter uppercase English letters, or that financial institution codes assigned to financial institutions are four-digit numbers. Note that the knowledge database 1401 may also be a database of domain knowledge specialized for a specific domain, such as transportation or finance.

[0131] <New rule derivation process> Fig. 17 is a flowchart illustrating an example of a new rule derivation process in the second embodiment. In this figure, the same processes as those in the new rule derivation process in the first embodiment shown in Fig. 9 are denoted by the same reference numerals, and their description will be omitted.

[0132] The new rule derivation unit 205 executes the processes of S702 to S710 (S701) for all test cases 208. The processes of S702 to S707 are the same as those in the first embodiment, and therefore a description thereof will be omitted.

[0133] Following the process of S707, the new rule derivation unit 205A determines whether the attribute name of each variable (property) included in the API request 209 or the API response 211 is registered in the knowledge database 1401 (S1501).

[0134] If the attribute names of the variables included in the API request 209 or the API response 211 are registered in the knowledge database 1401 (S1501: Yes), the new rule derivation unit 205A reads the conditions of the API request and the API response from the knowledge database 1401 (S1502). Then, the process proceeds to S710.

[0135] On the other hand, if the attribute name of each property included in the API request 209 or the API response 211 is not registered in the knowledge database 1401 (S1502: No), the new rule derivation unit 205A executes the processes of S708 and S709, and then proceeds to the process of S710.

[0136] The processing of S710 is the same as in the first embodiment, and therefore a description thereof will be omitted.

[0137] When the new rule derivation unit 205A has executed the processes of S702 to S710 for all test cases T (S711), it ends the new rule derivation process. However, the order of the processes of S702, S703, S704, and S705 does not matter. Also, the order of the processes of S708 and S709 does not matter.

[0138] In addition, when the adopted rule determination unit 216 generates a new rule 214 using conditions registered in the knowledge database 1401, it may display the name (attribute name) of the adopted knowledge, such as "postal code," "date," "currency code," or "financial institution code," on the new rule confirmation screen 1002 shown in Figure 13.

[0139] In this way, by using the conditions registered in the knowledge database 1401 as the conditions that the API request 209 and the API response 211 must satisfy, a more appropriate new rule 214 can be generated.

[0140] Other processes executed by the software test device 100 are the same as those in the first embodiment, and therefore, description thereof will be omitted.

[0141] As described above, when a condition related to a variable included in a request or response is stored in the knowledge database 1401, the software testing device 100 of this embodiment generates a new rule 214 including the condition related to the variable stored in the knowledge database 1401.

[0142] This allows the conditions for attributes whose requirements are generally defined, such as "postal code," "date," "currency code," or "financial institution code," to be accurately reflected in the determination rules 212.

[0143] The present invention is not limited to the above-described embodiments, and can be implemented using any components within the scope of the present invention. The above-described embodiments and modifications are merely examples, and the present invention is not limited to these contents as long as the characteristics of the invention are not impaired. Furthermore, although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other aspects conceivable within the scope of the technical idea of ​​the present invention are also included within the scope of the present invention.

[0144] For example, part of the hardware provided in each device of this embodiment may be provided in another device.

[0145] Furthermore, each program of the software testing device 100 may be provided in another device, a program may consist of multiple programs, or multiple programs may be integrated into one program. [Explanation of symbols]

[0146] 100 Software Test Equipment 101 processors 102 Main storage 103 Auxiliary storage device 104 Input Device 105 Output Device 106 Communication equipment 201 API request extraction unit 202 Test Execution Department 203 Decision Rule Extraction Unit 204 Test result judgment unit 205 New Rule Derivation 206 Test Case Generation Unit 207 Information storage section 208 test cases 209 API Request 210 Test target API 211 API response 212 Judgment Rules 213 Test Results 214 New Rules 215 Threshold 216 Recruitment Rules Decision Department 1001 New rule list screen 1002 New rule confirmation screen 1003 New rule editing screen 1101 New Rules List Tab 1102 View details button 1201 Check Rules Tab 1202 Start Editing Button 1203 Back to list button 1301 Editable Rule Tab 1302 End Editing Button 1303 Back without editing button 1304 Text Box 1401 Knowledge Database

Claims

1. a processor and a memory; a request extraction unit that extracts a request from a test case that includes a request to be sent to a system under test and a judgment rule for judging whether the test passes or fails; a test execution unit that executes a test on the system under test by transmitting the extracted request to the system under test and receives a response to the request; a determination rule extraction unit that extracts the determination rules from the test cases; a test result determination unit that determines whether the executed test has passed or failed based on the extracted request, the received response, and the extracted determination rule; a new rule derivation unit that generates a new rule as a judgment rule to be newly applied to the test case based on the extracted request, the received response, the extracted judgment rule, and the pass / fail of the test judged by the test result judgment unit; A generating device comprising:

2. the new rule derivation unit analyzes the response value to estimate a possible range or value pattern of the response value, and generates a new rule representing the estimated range or value pattern. The generating device of claim 1 .

3. the new rule derivation unit estimates a possible range or value pattern of the response value for each value or range of the request, and generates a new rule representing the estimated range or value pattern. The generating device of claim 2 .

4. the new rule derivation unit estimates a range or value pattern of the response value for all of the request values ​​in the history of multiple executions of the test case, and generates a new rule that represents the estimated range or value pattern. The generating device of claim 2 .

5. the new rule derivation unit extracts the requests and responses for which the test results are pass from a history of multiple executions of the test case, and generates the new rule based on the extracted requests and responses. The generating device of claim 1 .

6. the new rule derivation unit infers conditions for the values ​​of variables included in the request or the response by analyzing the values ​​of the variables in the request or the response for which the test result is a pass, and generates the new rule including the inferred conditions. The generating device of claim 5 .

7. When a condition related to a variable included in the request or the response is stored in a predetermined database, the new rule derivation unit generates the new rule including the condition related to the variable stored in the database. The generating device of claim 6.

8. a test case generation unit that generates a new test case by partially changing variable values ​​of the request of the test case based on the new rule; The generating device of claim 1 further comprising:

9. and a rule adoption unit that displays the generated new rule, accepts a modification of the displayed new rule from a user, and sets the modified new rule as a new judgment rule to be applied to the test case. The generating device of claim 1 .

10. The information processing device a request extraction process for extracting a request from a test case including a request to be transmitted to a system under test and a judgment rule for judging whether the test passes or fails; a test execution process that executes a test on the system under test by sending the extracted request to the system under test and receives a response to the request; a determination rule extraction process for extracting the determination rules from the test cases; a test result determination process for determining whether the executed test has passed or failed based on the extracted request, the received response, and the extracted determination rule; a new rule derivation process for generating a new rule as a judgment rule to be newly applied to the test case based on the extracted request, the received response, the extracted judgment rule, and the pass / fail of the judged test; A generation method that performs the following.

11. In the information processing device, a request extraction process for extracting a request from a test case including a request to be transmitted to a system under test and a judgment rule for judging whether the test passes or fails; a test execution process that executes a test on the system under test by sending the extracted request to the system under test and receives a response to the request; a determination rule extraction process for extracting the determination rules from the test cases; a test result determination process for determining whether the executed test has passed or failed based on the extracted request, the received response, and the extracted determination rule; a new rule derivation process for generating a new rule as a judgment rule to be newly applied to the test case based on the extracted request, the received response, the extracted judgment rule, and the pass / fail of the judged test; A generator program that executes the

Images