Test case generation device and test case generation method

Abstract

To provide a test case generation apparatus and method for generating test items focusing on parameter values with bugs.SOLUTION: A method includes: receiving an input of a result of executing a program to be tested with test items including one or more parameter values as input items; calculating importance which is a degree of probability of a bug factor, for each parameter value, on the basis of a bug generation rate obtained from the execution result; determining, when the calculated importance satisfies a first predetermined condition, a parameter value corresponding to the importance, to be reflected to a test viewpoint that indicates a point of focus or / and a point of non-focus of a test; generating test items on the basis of the reflected test viewpoint; determining, when the number of generated test items exceeds a threshold, a parameter value corresponding to the importance satisfying a second predetermined condition; reflecting the determined parameter value to the point of non-focus of the test viewpoint to suppress the number of test items; and outputting the suppressed number of test items.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a test case generation device and a test case generation method. [Background technology]

[0002] As a prior art in this technical field, there is JP 2013-77124 A (Patent Document 1). This publication describes "a software test case generation device that generates test cases for factor coverage testing of software testing and generates a list of the factor coverage testing, comprising: a storage unit that stores the generated test cases; a test execution history storage unit that stores execution history of software testing according to the generated test cases; a judgment criterion storage unit that stores judgment conditions for caution levels of factors and levels that should be tested with priority; a caution level combination analysis unit that judges combinations of factors and levels that require caution according to the judgment conditions for the caution level; a caution level combination storage unit that stores caution level combinations that are combinations of factors and levels that have been judged to be at a caution level; and an effective test case generation unit that generates efficient test cases based on the software test execution history and the caution level combinations" (see abstract). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-77124 A Summary of the Invention [Problem to be solved by the invention]

[0004] The above-mentioned Patent Document 1 describes a test case generation method for improving the efficiency of bug detection. However, the test case generation method of Patent Document 1 cannot generate test items related to parameter values ​​where bugs occur in a focused manner according to the frequency of bug occurrence.

[0005] An object of one aspect of the present invention is to provide a technique capable of generating test items related to parameter values ​​in which bugs have occurred in a focused manner according to the frequency with which the bugs occur. [Means for solving the problem]

[0006] A test case generation device according to one aspect of the present invention is a test case generation device that generates test cases for processing performed by a computer, and is characterized in having a calculation device that performs the following: a test result input process that receives an input of the result of execution of a test target program by providing test items including one or more parameter values ​​as input items to the program; a parameter value importance analysis process that calculates an importance, which is the likelihood of a bug cause for each parameter value, based on a bug occurrence rate obtained from the result of the execution; a test perspective determination process that, if the calculated importance satisfies a first predetermined condition, determines a parameter value corresponding to the importance and reflects the parameter value in a test perspective that indicates a focus point and / or a non-focus point of a test; a test item generation process that generates test items based on the reflected test perspective; and a test item suppression process that, if the number of generated test items exceeds a threshold value, determines a parameter value corresponding to the importance that satisfies a second predetermined condition and reflects the determined parameter value in a non-focus point of the test perspective to suppress the number of test items; and an output device that performs a test item output process that outputs the suppressed number of test items. Effect of the Invention

[0007] According to one aspect of the present invention, test items related to parameter values ​​in which bugs occur can be generated in a focused manner according to the frequency with which the bugs occur. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of the configuration of a test case generation device; [Diagram 2] 11 is a flowchart illustrating a process of the test case generation device. [Diagram 3] 1 is an explanatory example of parameters and parameter values. [Figure 4] 13 is an example of how to enter test items and test execution results. [Diagram 5] 13 is an explanatory example of a parameter value importance analysis process. [Figure 6] 13 is a flowchart of a parameter value importance analysis process. [Figure 7] 13 is an explanatory example of a test viewpoint determination process. [Figure 8] 13 is an example of test items generated in the test item generation process. [Figure 9] 13 is a flowchart of a test item suppression process. [Figure 10] 13 is an example of test items obtained after test item regeneration in the test item suppression process. [Figure 11] 11 is a flowchart illustrating a process of a test case generation device in the second embodiment. [Figure 12] FIG. 1 is a diagram illustrating an example of a schematic configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An example of a test case generation device according to an embodiment of the present invention will be described below with reference to the drawings. The following description and drawings are examples for explaining the present invention, and some omissions and simplifications have been made as appropriate for clarity of explanation. The present invention can be embodied in various other forms. Unless otherwise specified, each component may be singular or plural.

[0010] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0011] In the following explanation, various information may be described using expressions such as "database," "table," and "list," but the various information may be expressed in other data structures. To indicate independence of data structure, "XX table," "XX list," and the like may be referred to as "XX information." When describing identification information, when expressions such as "identification information," "identifier," "name," "ID," and "number" are used, these are interchangeable.

[0012] When there are multiple components having the same or similar functions, they may be described by using the same reference numerals with different subscripts. However, when there is no need to distinguish between these multiple components, the subscripts may be omitted.

[0013] In addition, in the following description, processing performed by executing a program may be described, but the program is executed by a processor (e.g., a CPU (Central Processing Unit), a GPU (Graphics Processing Unit)) to perform a defined process using storage resources (e.g., a memory) and / or an interface device (e.g., a communication port) as appropriate, so the subject of the processing may be the processor. Similarly, the subject of the processing performed by executing a program may be a controller, device, system, computer, or node having a processor. The subject of the processing performed by executing a program may be a calculation unit, and may include a dedicated circuit that performs specific processing (e.g., an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)).

[0014] A program may be installed in a device such as a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource for storing a program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In the following description, two or more programs may be realized as one program, and one program may be realized as two or more programs. EXAMPLES

[0015] FIG. 1 is a diagram illustrating an example of the configuration of a test case generation device according to the present embodiment.

[0016] As shown in FIG. 1 , the test case generation device 100 has a processing unit that executes a test result input process 101, a processing unit that executes a user judgment input process 102, a processing unit that executes a parameter value importance analysis process 103, a processing unit that executes a test viewpoint determination process 104, a processing unit that executes a test item generation process 105, a processing unit that executes a test item suppression process 106, a parameter value importance holding unit 107, a user judgment holding unit 108, a test viewpoint holding unit 109, and processing units that execute a test item output process 110.

[0017] The test case generation device 100 shown in Figure 1 can be realized, for example, by a general computer 1600 as shown in Figure 12 (schematic computer diagram), which has a CPU 1601, a memory 1602, an external storage device 1603 such as an HDD (Hard Disk Drive), a reading / writing device 1607 for reading and writing information from a portable storage medium 1608 such as a CD (Compact Disk) or a USB memory, an input device 1606 such as a keyboard or a mouse, an output device 1605 such as a display, a communication device 1604 such as a NIC (Network Interface Card) for connecting to a communication network, and an internal communication line (called a system bus) 1609 such as a system bus that connects these together.

[0018] Various data stored in the test case generation device 100 or used for processing (e.g., data stored in the parameter value importance storage unit 107, the user decision storage unit 108, and the test viewpoint storage unit 109) can be realized by the CPU 1601 reading them from the memory 1602 or the external storage device 1603 and using them. Also, each processing unit of the test case generation device 100 (e.g., each processing unit executing the test result input process 101, the user decision input process 102, the parameter value importance analysis process 103, the test viewpoint determination process 104, the test item generation process 105, the test item suppression process 106, and the test item output process 110) can be realized by the CPU 1601 loading a predetermined program stored in the external storage device 1603 into the memory 1602 and executing it.

[0019] The above-mentioned predetermined program may be stored (downloaded) in the external storage device 1603 from the storage medium 1608 via the reading / writing device 1607 or from the network via the communication device 1604, and then loaded onto the memory 1602 and executed by the CPU 1601. Alternatively, the program may be directly loaded onto the memory 1602 from the storage medium 1608 via the reading / writing device 1607 or from the network via the communication device 1604, and executed by the CPU 1601.

[0020] In the following, we will explain an example in which the test case generation device 100 is configured by a single computer, but all or part of these functions may be distributed across one or more computers, such as a cloud, and similar functions may be realized by communicating with each other via a network.

[0021] The test result input process 101 receives unexecuted test items and records the results of the executed tests.

[0022] The user decision input process 102 receives parameters or parameter values ​​that the user has decided to have high or low importance.

[0023] The parameter value importance analysis process 103 analyzes the possibility of a bug cause based on the test execution result, and determines the importance of the parameter value.

[0024] A test viewpoint determination process 104 determines a test viewpoint using a parameter value that is determined to have high importance based on the test result and a user's judgment.

[0025] The test item generation process 105 generates test items based on the test viewpoints.

[0026] If the generated test items exceed a threshold, the test item suppression process 106 suppresses the test items by reflecting the parameter values ​​that are determined to have low importance based on the test results and user judgment in the test viewpoint.

[0027] The parameter value importance storage unit 107 stores the importance of the parameter value determined in the parameter value importance analysis process 103 .

[0028] The user decision holding unit 108 holds parameters or parameter values ​​with high importance and parameters or parameter values ​​with low importance input by the user in the user decision input process 102 .

[0029] The test viewpoint holding unit 109 holds the test viewpoint determined in the test viewpoint determination process 104 .

[0030] The test item output process 110 outputs the test items determined as a result of the test item generation process 105 and the test item suppression process 106 .

[0031] FIG. 2 is a flowchart 150 for explaining the processing of the test case generation device 100.

[0032] Step 151 accepts input of initial test items prepared in advance.

[0033] Step 152 performs the test result input process 101 .

[0034] Step 153 judges whether the test is terminated. The number of test items and the number of bugs are used as the criteria. For the number of test items, the input result of step 151 is used the first time, and the output result of step 159 is used from the second time onwards. For the number of bugs, the result of step 152 is used.

[0035] In this embodiment, if it is determined that a sufficient number of tests have been performed and a sufficient number of bugs have been detected, that is, if the number of test items and the number of bugs exceed the designated number, it is determined that the end condition is met (step 153; Yes), and the flow chart 150 is terminated. If not (step 153; No), the process proceeds to step 154.

[0036] Step 154 ​​performs the parameter value importance analysis process 103, step 155 performs the user decision input process 102, and step 156 performs the test viewpoint determination process 104. Also, step 157 performs the test item generation process 105, step 158 performs the test item suppression process 106, and step 159 performs the test item output process 110. After step 159 is completed, the process returns to step 153 and is repeated until the end condition is satisfied.

[0037] 3 is an explanatory diagram of a test target example 200 of parameters and parameter values. A parameter is a data item input to a program from the outside and affects the operation of the program. A parameter value refers to the possible values ​​of each parameter.

[0038] The test target example 200 of parameters and parameter values ​​is composed of test target parameters 201 and test target parameter values ​​202. In the case of Fig. 3, the test target parameters 201 include three parameters A, B, and C. The test target parameter values ​​202 include parameter values ​​a1 and a2 related to A, parameter values ​​b1, b2, and b3 related to B, and parameter values ​​c1, c2, c3, and c4 related to C.

[0039] 4 is an explanatory diagram of an example 300 of test items and their execution results in the processing of the test result input process 101. A test item is prepared to check the operation of a program, and generally indicates execution conditions, data, etc. In this embodiment, it indicates one parameter value corresponding to a parameter, or a combination of multiple parameter values.

[0040] An example 300 of a test item and its execution result includes a test item number 301 , a test item 302 to be executed, and a test execution result 303 .

[0041] For example, in the case of FIG. 4, the test item 302 to be executed is composed of a combination of parameter values ​​A, B, and C, which are the test target parameters 201, and one of the test target parameter values ​​202 is assigned to each of the parameters A, B, and C.

[0042] The test execution result 303 indicates whether or not a bug has been detected. If a bug has been detected, a value of 1 is entered in the test execution result 303, and if no bug has been detected, a value of 0 is entered in the test execution result 303.

[0043] FIG. 5 is an explanatory diagram of an analysis example 400 of the importance of a parameter value in the parameter value importance analysis process 103. As shown in FIG.

[0044] An analysis example 400 of parameter value importance includes an analysis target parameter / parameter value 401, a bug occurrence rate 402, and a parameter value importance 403, and in Fig. 5, the analysis target parameter / parameter value 401 are displayed in descending order of bug occurrence rate 402. The parameter value importance 403 indicates the importance of the parameter value, which is determined according to the bug occurrence rate of the parameter value.

[0045] The analyzed parameter / parameter value 401 is composed of a combination of the tested parameter 201 and the tested parameter value 202 .

[0046] The bug occurrence rate 402 is the proportion of test items that detected a bug (test items 302 to be executed that had a test execution result 303 of 1) among the test items (test items 302 to be executed) that include the analysis target parameter / parameter value 401.

[0047] FIG. 6 is a flowchart 500 of the importance analysis process, illustrating the parameter value importance analysis process 103.

[0048] In step 501, the analysis target parameters and parameter values ​​401 are read.

[0049] In step 502, for each parameter value read in step 501, the number of occurrences in the test items 302 to be executed included in FIG. 4 is counted.

[0050] In step 503, for each parameter value read in step 501, the number of test execution results 303 that are 1, that is, the number of test items 302 to be executed that have detected a bug, is counted.

[0051] In step 504 , the bug occurrence rate 402 is calculated for each parameter value read in step 501 .

[0052] In step 505, parameter value importance 403 is calculated according to bug occurrence rate 402. In this embodiment, the higher the bug occurrence rate 402, the more likely the corresponding analysis target parameter / parameter value 401 is determined to be a bug cause, and the value of parameter value importance 403 is increased. The larger the value of parameter value importance 403, the more important the parameter value is, and the same value is set as the number of parameter values ​​included in test target parameter value 202. Note that the parameter value importance 403 of analysis target parameters / parameter values ​​401 with the same bug occurrence rate 402 is the same.

[0053] In the following, the process of deriving the bug incidence rate 402 and the parameter value importance 403 of the parameter value importance analysis example 400 in FIG. 5 will be described using the test items and the execution result example 300 in FIG. 4 as an example.

[0054] First, consider the case of calculating the bug incidence rate 402 of A=a1 among the analysis target parameters and parameter values ​​401 in FIG. 5. In step 502, the number of test items including a1 among the test items 302 to be executed is counted as six, namely #1, #2, #5, #9, #10, and #12. Among them, in step 503, the number of test items 302 to be executed whose test execution result 303 is 1 is five, namely #2, #5, #9, #10, and #12. Therefore, in step 504, the bug incidence rate 402 of A=a1 is calculated as 5 / 6*100 = 83%. Similarly, the bug incidence rate 402 of the analysis target parameters and parameter values ​​401 other than A=a1 is as shown in FIG. 5.

[0055] Next, consider the case of calculating parameter value importance 403 for each analysis target parameter / parameter value 401 in Fig. 5. Since the total number of analysis target parameters / parameter values ​​401 in Fig. 5 is 9, the parameter value importance 403 of the analysis target parameter / parameter value 401 with the highest importance is 9. As a result of calculating bug occurrence rate 402, it is determined that the analysis target parameter / parameter value 401 with the highest bug occurrence rate 402 is A=a1, and the value of parameter value importance 403 is 9, which is the highest importance.

[0056] On the other hand, the analyzed parameter / parameter value 401 with the lowest bug incidence rate 402 is C=c1, and the value of the parameter value importance 403 is determined to be 1, the lowest importance.

[0057] The combination of the analysis target parameter / parameter value 401 thus obtained and the corresponding parameter value importance 403 is stored in the parameter value importance storage unit 107 .

[0058] FIG. 7 is an explanatory diagram of a test viewpoint determination example 600 in the test viewpoint determination process 104. As shown in FIG.

[0059] The test viewpoint generally refers to a summary of the test viewpoints required to confirm the function and performance of the test target. In particular, in this embodiment, it refers to the points that show the focus and non-focus of the test. This allows you to target points that are likely to be bug causes. It also allows you to suppress points that are unlikely to be bug causes.

[0060] There are two examples of test viewpoints 601: one showing a key point 602 and the other showing a non-key point 603. In the case of the key point 602, the test viewpoint 601 is described as "Focus on testing for...", and in the case of the non-key point 603, the test viewpoint 601 is described as "Suppress test items related to...".

[0061] The "..." is substituted with one of the test target parameter values ​​202. The test target parameter value 202 is the analysis target parameter value 401 that is determined to have the highest parameter value importance 403 as a result of the parameter value importance analysis process 103.

[0062] For example, in the case of FIG. 5, since it is determined that the analyzed parameter / parameter value 401 having the highest parameter value importance 403 is A=a1, the test viewpoint 601 is described as "Focus on testing A=a1" based on the focus point 602.

[0063] The user judgment input process 102 receives parameters or parameter values ​​indicating the key points 602, which are input from the user. However, this input is not essential. When a parameter value is input, the input parameter or parameter value is reflected in the test viewpoint 601 in priority over the result of the parameter value importance analysis process 103. When a parameter is input, the same process (for example, the process after step 154 ​​in FIG. 2) is performed on all test target parameter values ​​202 corresponding to that parameter.

[0064] This allows the determination of a test viewpoint 601 other than the parameter value importance 403 determined from the bug occurrence rate 402, making the method of determining the test viewpoint 601 flexible.

[0065] FIG. 8 is an explanatory diagram of a test item generation example 700 in the processing of the test item generation process 105.

[0066] The test item generation example 700 includes a test item generation number 701 and a test item to be generated 702. The test item generation process 105 generates the test item generation example 700 based on the test viewpoint 601 described in the test viewpoint determination process 104.

[0067] For example, in the case of FIG. 8, as a result of the test viewpoint determination process 104, the test viewpoint 601 based on the focus point 602 is determined to be “Focus on testing A=a1”, so test items that include A=a1 in the combination are generated, and six test item generation numbers 701 are generated, from #13 to #18.

[0068] However, the test items that already exist (in this case, the test items in the example 300 of the test items and their execution results in FIG. 4) are not subject to generation.

[0069] FIG. 9 is a suppression process flow chart 800 illustrating the test item suppression process 106.

[0070] In step 801 , the test item generation example 700 generated in the test item generation process 105 is read.

[0071] In step 802, the number of test items to be generated read in step 801 is compared with a preset upper limit. If the number of generated items does not exceed the upper limit (step 802; No), the flow chart 800 ends without performing the suppression processing in the following steps 803, 804, and 805. If not (step 802; Yes), the process proceeds to step 803.

[0072] In step 803 , the analysis target parameter / parameter value 401 having the lowest parameter value importance 403 is selected as the parameter value to be suppressed in the non-important portion 603 from among the parameter values ​​not previously selected as the suppression target.

[0073] The user judgment input process 102 also receives parameters or parameter values ​​indicating non-important points 603, which are input from the user. However, this input is not essential. If there is an input, the input parameters or parameter values ​​are selected as suppression targets in priority over the results of the parameter value importance analysis process 103.

[0074] In step 804, the parameter values ​​to be suppressed selected in step 803 are reflected and corrected in the test viewpoint 601. At this time, the corresponding parameter values ​​are inserted into "...", and the statement "Test items related to... are suppressed" regarding the non-emphasis points 603 is entered in the test viewpoint 601.

[0075] In step 805, test items are suppressed based on the test viewpoint 601 modified in step 804, and the number of test items to be generated is adjusted. At this time, the test items are suppressed so that all combinations other than the parameter values ​​to be suppressed selected in step 803 can be covered.

[0076] 10 is an explanatory diagram of a test item suppression example 900 in the processing of the test item suppression process 106. The test item suppression example 900 shows the test items after suppression as a result of the test item suppression process 106, out of the test item generation example 700 in FIG.

[0077] For example, in step 802 in FIG. 9, if the total number of test items after the test item generation process 105 is 18 in total for FIGS.

[0078] 4, as a result of the processing of the parameter value importance analysis process 103, it is determined that the analysis target parameter / parameter value 401 having the lowest parameter value importance 403 is C=c1, and therefore C=c1 is selected as the parameter value with the lowest importance in step 803. If a parameter or parameter value to be suppressed is not input in the user judgment input process 102, C=c1 becomes the parameter value to be suppressed.

[0079] In step 804 , based on the result of step 803 , the statement “suppress test items related to C=c1” regarding the non-emphasis point 603 is written in the test viewpoint 601 .

[0080] Then, in step 805, test items related to the parameter value C=c1 to be suppressed are suppressed based on the result of step 804. As a result, in the test item generation example 700 in Fig. 8, the test items with test item generation numbers 701 #13 and #15 are targeted for deletion. Since combinations other than C=c1, (A,B)=(a1,b1), (a1,b2), also exist in #14 and #16, these two test items #13 and #15 are deleted.

[0081] The process proceeds again to step 802, and since the total number of test items in FIG. 4 and FIG. 10 is 16, the suppression process flowchart 800 ends.

[0082] In this manner, in this embodiment, a test case generation device 100 for generating test cases for processing performed by a computer includes a test result input process 101 for receiving input of the results of execution of a test item including one or more parameter values ​​as input items given to a program under test, a parameter value importance analysis process 103 for calculating the importance, which is the likelihood of a bug being the cause of a bug, for each parameter value based on a bug occurrence rate (e.g., bug occurrence rate 402) obtained from the execution result, and a parameter value importance analysis process 104 for calculating the importance of a parameter value corresponding to the importance when the calculated importance satisfies a first predetermined condition (e.g., being the highest importance). The system includes a calculation device that performs a test viewpoint determination process 104 that determines a parameter value corresponding to the importance that satisfies a second predetermined condition (e.g., the lowest importance) and reflects it in a test viewpoint that indicates a test focus point or / and a non-focus point, a test item generation process 105 that generates test items based on the reflected test viewpoint, a test item suppression process 106 that determines a parameter value corresponding to the importance that satisfies a second predetermined condition (e.g., the lowest importance) when the number of generated test items exceeds a threshold, and reflects the determined parameter value in a non-focus point of the test viewpoint to suppress the number of test items, and an output device that performs a test item output process 110 that outputs the suppressed number of test items. Therefore, by quantifying the possibility of a bug cause for each parameter value based on the test execution result and reflecting it in a test viewpoint that indicates a test focus point / non-focus point, it is possible to generate test items related to parameter values ​​that are likely to be bug causes with a focus. In addition, since the test viewpoint is used to add or suppress test items focusing on a specific parameter value, it is possible to generate test items taking into account the importance of each parameter value, and it is expected that the test efficiency in bug detection will be improved.

[0083] The computing device also performs a user judgment input process 102 for receiving input from a user of a parameter or parameter value satisfying the first predetermined condition, a parameter or parameter value satisfying the second predetermined condition, or both, and in the test viewpoint determination process, determines the parameter value to be reflected in the test viewpoint based on the importance updated based on the parameter or parameter value input in the user judgment input process. Thus, the user can input a parameter or parameter value that he or she judges to be of high or low importance, and adjust the parameter or parameter value. EXAMPLES

[0084] In this embodiment, an example of a test case generation device that accepts input of data other than the immediately preceding test result will be described.

[0085] In such a case, the test result input process 101 receives the test item for which the test was just executed and other test items previously stored in the storage device, and writes down the results of the test execution.

[0086] Fig. 11 is a flowchart 250 for explaining the processing of the test case generation device 100 in the second embodiment. In Fig. 11, input of a prepared initial test item and test result input processing 101 is performed by steps (steps 151-1 and 152-1) performed for the test item that was tested immediately before, and steps (steps 151-2 and 152-2) performed for the other test items previously stored in the storage device, and the end condition for each step is determined in step 153. The other processing is the same as in the first embodiment.

[0087] In this manner, in this embodiment, the computing device accepts input of the most recent test result and past test results in the test result input process, and calculates the importance level based on the bug occurrence rate obtained from the most recent test result and the past test results in the test viewpoint determination process. Therefore, the importance level can be determined taking into account past test results in addition to the current test result. EXAMPLES

[0088] In this embodiment, an example of a test case generation device that accepts the number of parameters or parameter values ​​with high importance and the number of parameters or parameter values ​​with low importance will be described.

[0089] In such a case, the user judgment input process 102 accepts the number (M) of parameters or parameter values ​​with high importance and the number (N) of parameters or parameter values ​​with low importance. The number (M) of parameters or parameter values ​​with high importance and the number (N) of parameters or parameter values ​​with low importance can be determined as follows. For example, the number of parameters or parameter values ​​decreasing from the most important parameter or parameter value can be determined as M, and the number of parameters or parameter values ​​increasing from the least important parameter or parameter value can be determined as N.

[0090] Furthermore, the test viewpoint determination process 104 reflects M analysis target parameters / parameter values ​​401 in descending order of the parameter value importance 403 determined to be the highest as a result of the parameter value importance analysis process 103, in the test viewpoint 601. While Fig. 7 illustrates an example in which the test viewpoint 601 is described in one record, in this embodiment, M records of test viewpoints 601 are described.

[0091] Furthermore, with regard to the test item suppression process 106, step 803 selects N analyzed parameters / parameter values ​​401 in ascending order of parameter value importance 403 from among parameter values ​​not previously selected as suppression targets, as parameter values ​​to be suppressed in non-important areas 603. Thereafter, step 804 reflects the N analyzed parameters / parameter values ​​401 selected in step 803 in the test viewpoint 601. While FIG. 7 illustrates an example in which the test viewpoint 601 is described in one record, in this embodiment, N records of test viewpoints 601 are described. Processing other than the above is the same as in embodiment 1.

[0092] Thus, in this embodiment, the arithmetic device accepts an input of a number of parameters or parameter values ​​(e.g., the M number) having a certain threshold or more and a high importance as the first predetermined condition in the user judgment input process, accepts an input of a number of parameters or parameter values ​​having a certain threshold or more and a low importance as the second predetermined condition (e.g., the N number), and determines the parameter value to be reflected in the test viewpoint in the test viewpoint determination process based on the importance updated based on the number of parameters or parameter values ​​input in the user judgment input process. Therefore, a plurality of parameters or parameter values ​​can be reflected in the test viewpoint in order from the most important one or the least important one. EXAMPLES

[0093] In this embodiment, an example of a test case generation device that considers multiple combinations of parameter values ​​in determining the parameter value importance will be described.

[0094] In such a case, in the parameter value importance analysis process 103, the analysis target parameter / parameter value 401 is configured using a plurality of combinations of the test target parameter 201 and the test target parameter value 202. The bug occurrence rate 402 and the parameter value importance 403 are derived in the same manner as in the first embodiment. For example, the bug occurrence rate 402 and the parameter value importance 403 may be calculated for a combination of the analysis target parameter / parameter value where the analysis target parameter / parameter value 401 is A=a1 and B=b1. The other processes are the same as in the first embodiment.

[0095] Thus, in this embodiment, the arithmetic device calculates the importance of a combination (e.g., the combination of "A=a1" and "B=b1") of one parameter value (e.g., "A=a1" in FIG. 5) and another parameter value (e.g., "B=b1" in FIG. 5) in the parameter value importance analysis process. For example, the importance is calculated using the average value of the bug incidence rate of "83%" for "A=a1" and the bug incidence rate of "75%" for "B=b1". This makes it possible to calculate the importance after leveling out the bug incidence rate using a plurality of parameter values. EXAMPLES

[0096] In this embodiment, an example of a test case generation device in which source code is input as input from the user, rather than input of parameters or parameter values, will be described.

[0097] In such a case, the user decision input process 102 accepts the source code as an input from the user. By analyzing the path of the source code, etc., the importance of the parameter or the parameter value is automatically derived. The other processes are the same as those in the first embodiment.

[0098] Thus, in this embodiment, in the parameter value importance analysis process, the arithmetic device analyzes the source code of the test target program and determines the importance of the parameter value based on the analysis result. Therefore, the importance can be determined without the user having to input parameters or parameter values. Even if the source code of the test target program is frequently modified, the importance of the parameter value can be automatically determined by simply reading and analyzing the source code without having to input parameters or parameter values ​​corresponding to the modifications. EXAMPLES

[0099] In this embodiment, an example of a test case generation device that determines the importance of parameter values ​​based on the coverage of tests in addition to the possibility of bug causes will be described. This increases the number of test cases by increasing the importance of parameter values ​​that appear less frequently.

[0100] In such a case, the parameter value importance analysis example 400 in the processing of the parameter value importance analysis process 103 may further include the coverage rate in addition to the bug occurrence rate 402 .

[0101] The coverage rate is the ratio of the number of test cases input to the test result input process 101 to the number of all possible test cases when focusing on a particular parameter value.

[0102] For example, consider the case of obtaining the coverage rate of A=a1 among the analysis target parameters and parameter values ​​401. Considering the combination of possible values ​​of the test target parameters B and C, the total number of test cases including A=a1 is 1*3*4=12. Of these, the number of test cases including A=a1 in FIG. 4 is six, namely #1, #2, #5, #9, #10, and #12. Therefore, the coverage rate of A=a1 is 6 / 12*100=50%. In the same manner, the coverage rates of other analysis target parameters and parameter values ​​401 can be calculated. In addition, the parameter value importance 403 of the analysis target parameters and parameter values ​​401 with low coverage rates is increased. For example, in this embodiment, the importance can be determined such that the greater the value obtained by multiplying the bug occurrence rate 402 and the above coverage rate, the higher the importance is, while the smaller the value is, the lower the importance is. Other processing is the same as in the first embodiment.

[0103] In this manner, in the present embodiment, the arithmetic device increases the importance of a parameter value having a low test coverage in the parameter value importance analysis process, thereby making it possible to determine the importance of a parameter value in consideration of the test coverage.

[0104] The present invention is not limited to the above-described embodiments, and includes various modified examples. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the configurations described. It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

[0105] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in hardware by designing some or all of them as an integrated circuit, for example. In addition, the above-mentioned configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0106] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0107] 100 Test Case Generator 101 Test result input processing 102 User decision input processing 103 Parameter Value Importance Analysis Processing 104 Test viewpoint decision process 105 Test item generation process 106 Test Item Suppression Processing 107 Parameter value importance storage unit 108 User decision holding unit 109 Test Viewpoint Holder 110 Test item output processing

Claims

1. A test case generation device for generating test cases for a process performed by a computer, comprising: a test result input process for receiving input of a result of executing a test item including one or more parameter values ​​as input items by providing the test item to a program under test; a parameter value importance analysis process for calculating an importance level, which is a likelihood of a bug being a cause, for each of the parameter values ​​based on a bug occurrence rate obtained from the execution result; a test viewpoint determination process for determining a parameter value corresponding to the calculated importance level when the calculated importance level satisfies a first predetermined condition, and reflecting the determined parameter value in a test viewpoint indicating a key point and / or a non-key point of the test; A test item generation process for generating test items based on the reflected test viewpoints; a calculation device that performs a test item suppression process of determining the parameter value corresponding to the importance that satisfies a second predetermined condition when the number of the generated test items exceeds a threshold, and suppressing the number of the test items by reflecting the determined parameter value in a non-important point of the test viewpoint; and an output device that performs a test item output process for outputting the suppressed number of the test items; A test case generation device comprising:

2. the arithmetic device performs a user decision input process of accepting input of a parameter or a parameter value that satisfies the first predetermined condition, a parameter or a parameter value that satisfies the second predetermined condition, or both of them, from a user; In the test viewpoint determination process, the parameter value to be reflected in the test viewpoint is determined based on the importance updated based on the parameter or parameter value input in the user judgment input process.

2. The test case generation device according to claim 1.

3. the computing device accepts input of a previous test result and a past test result in the test result input process; In the test viewpoint determination process, the importance is calculated based on a bug occurrence rate obtained from the immediately preceding test result and the past test results.

2. The test case generation device according to claim 1.

4. the arithmetic device, in the user judgment input process, accepts an input of a number of parameters or parameter values ​​having high importance that is equal to or greater than a certain threshold as the first predetermined condition, and accepts an input of a number of parameters or parameter values ​​having low importance that is equal to or greater than a certain threshold as the second predetermined condition; In the test viewpoint determination process, the parameter value to be reflected in the test viewpoint is determined based on the importance updated based on the number of parameters or parameter values ​​input in the user judgment input process.

3. The test case generation device according to claim 2.

5. the arithmetic device calculates the importance of a combination of one parameter value and another parameter value in the parameter value importance analysis process; 2. The test case generation device according to claim 1.

6. the arithmetic device, in the parameter value importance analysis process, analyzes a source code of the program to be tested, and determines the importance of the parameter value based on a result of the analysis; 2. The test case generation device according to claim 1.

7. the arithmetic device increases the importance of a parameter value having a low test coverage in the parameter value importance analysis process.

2. The test case generation device according to claim 1.

8. 1. A method for generating test cases for a process performed by a computer, comprising the steps of: receiving input of the result of executing a test item including one or more parameter values ​​as input items to a program under test; calculating a degree of importance, which is a degree of likelihood of a bug cause, for each of the parameter values ​​based on a bug occurrence rate obtained from the execution result; When the calculated importance satisfies a first predetermined condition, a parameter value corresponding to the importance is determined, and the determined parameter value is reflected in a test perspective indicating a focus point and / or a non-focus point of the test; generating test items based on the reflected test viewpoints; When the number of the generated test items exceeds a threshold, the parameter value corresponding to the importance that satisfies a second predetermined condition is determined, and the determined parameter value is reflected in a non-important point of the test viewpoint to suppress the number of the test items; outputting a suppressed number of said test items; A method for generating test cases comprising:

Images