Test case generation device, test case generation method and computer-readable recording medium

Abstract

When the (i-1)th test case as a test case from step 1 to step (i-1) is stored, the generation control unit (130) selects the first generation method (M1) for generating the i-th test case as a test case from step 1 to step i using the (i-1)th test case. The test generation unit (170) generates a test case for executing the i-th step in accordance with the first generation method (M1) while maintaining an internal state at the end time of step (i-1) based on the execution of the (i-1)th test case. The test generation unit (170) generates the i-th test case by linking the (i-1)th test case and the test case of the i-th step.

Description

Technical Field

[0001] The present invention relates to a test case generating device, a test case generating method and a computer-readable recording medium, and more particularly to a test case generating device, a test case generating method and a computer-readable recording medium for automatically generating test cases for embedded software. Background Art

[0002] Control devices such as automobiles and industrial robots are equipped with embedded real-time software for various control functions. If this software malfunctions, it can lead to life-threatening accidents. Therefore, thorough testing is required before shipment. For example, ISO 26262, the functional safety standard for automobiles, and DO-178C, the functional safety standard for aircraft, require that test methods be determined for each test process and that testing be performed in accordance with these test requirements. As an example of a test requirement for unit testing, test cases are generated based on equivalence partitioning or boundary value analysis, thereby executing the test in a manner that includes the requirements. Furthermore, the test cases are required to include the source code structure of the test target software. The source code structure is based on the command lines that describe the processing, or branch locations such as if statements and for statements. The benchmark used varies depending on the test target software. This benchmark is called a structure inclusion benchmark. When the command line is used as the structure inclusion benchmark, the command inclusion rate is measured, which indicates how much of the code included in the test target software is executed by the test case. When branches such as if statements are used as structural inclusion criteria, the branch inclusion rate is measured, which indicates whether all branch destinations included in the code have been executed. If the instruction inclusion rate or branch inclusion rate is sufficiently high, it is determined that the test is sufficient.

[0003] Embedded real-time software typically has multiple states, including initial state, execution state, and termination preparation state. In such software, simply entering a test case once does not capture the source code structure. Sometimes, multiple test case entries are necessary. Here, the execution unit of the test software is called a step. For example, entering a test case once is considered a step.

[0004] In recent years, the source code of embedded real-time software has become larger and more complex. Consequently, manually generating multi-step test cases that meet the software's specifications and include source code structure has become a problem.

[0005] Patent Document 1 discloses a method for automatically generating multi-step test cases for unit testing using finite model checking to address the increase in man-hours.

[0006] Patent Document 2 discloses a method using non-symbolic execution such as white-box fuzzing in addition to a method using symbolic execution for input value generation for vulnerability analysis.

[0007] Patent Document 3 discloses a method of performing symbolic execution path analysis on each module and then obtaining a composite execution path of a plurality of modules using information on each module.

[0008] Prior art literature

[0009] Patent Literature

[0010] Patent Document 1: Japanese Patent Application Laid-Open No. 2014-063415

[0011] Patent Document 2: Japanese Patent Application Laid-Open No. 2016-167262

[0012] Patent Document 3: Japanese Patent Application Laid-Open No. 2017-204164 Summary of the Invention

[0013] Problems to be solved by the invention

[0014] The method in Patent Document 1 uses finite model checking to analyze whether test cases can be generated that sequentially satisfy test requirements from the first time until a predetermined maximum number of steps is reached. Furthermore, if there are requirements or branches that are not included, the number of steps is increased, and the test case generation analysis is repeated. However, this method has the following problem: increasing the number of steps increases the number of states subject to finite model checking, potentially preventing the test case generation time from converging to a practical time.

[0015] The method in Patent Document 2 combines symbolic execution-based test case generation, such as finite model checking, with non-symbolic execution-based test case generation, such as white-box fuzz testing. This allows the use of a high-speed non-symbolic execution-based generation method to include as many included objects as possible, while also using a reliable symbolic execution-based generation method to include remaining included objects, thus reducing overall test case generation time. However, since the analysis and generation of steps increase as the number of steps increases, the issue of increasing test case generation time remains.

[0016] The method in Patent Document 3 extracts the execution path for each step and then associates the information of the variables stored across those steps. This reduces the time required to extract the execution path across all steps when generating a test case. However, the size of the execution path analyzed for test case generation increases by an amount corresponding to the number of steps. Therefore, as with other methods, the test case generation time may not be within a practical time.

[0017] Thus, conventional methods cannot be applied to the generation of test cases involving multiple steps for software whose input changes between steps, such as embedded real-time software.

[0018] An object of the present invention is to reduce the time required for generating test cases involving multiple steps.

[0019] Means for solving problems

[0020] A test case generation device of the present invention generates a test case for a program that executes one step of processing each time a value is input, wherein the test case is composed of values ​​of the number of test case inputs input to each of i (i is an integer greater than or equal to 2) steps of processing executed sequentially. The test case generation device comprises:

[0021] a memory that stores data;

[0022] a generation control unit that, when the memory stores an (i-1)th test case as a test case from step 1 to step (i-1), selects a first generation method of generating an i-th test case as a test case from step 1 to step i using the (i-1)th test case; and

[0023] A test generation unit generates a test case for executing the i-th step in accordance with the first generation method while maintaining an internal state at the end time of the (i-1) step based on the execution of the (i-1)-th test case, and generates the i-th test case by linking the (i-1)-th test case and the test case of the i-th step and stores it in the memory.

[0024] Effects of the Invention

[0025] In the test case generation device of the present invention, even when the number of steps increases, only one step of test case generation using non-symbolic execution or symbolic execution is performed first. This reduces the time required to generate test cases involving multiple steps using non-symbolic execution or symbolic execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a block diagram showing the structure of the test case generation device according to the first embodiment.

[0027] Figure 2 This is a flowchart showing the operation of the test case generation device according to the first embodiment.

[0028] Figure 3 This is a flowchart showing the operation of the test case generation device according to the first embodiment.

[0029] Figure 4 This is a flowchart showing the operation of the test case generation device according to the first embodiment.

[0030] Figure 5 This is a diagram showing a program of the test target software according to the first embodiment.

[0031] Figure 6 This is a diagram showing the included parts included in the test target software program according to the first embodiment.

[0032] Figure 7 This is a diagram showing a test case generation function for generating a single-step test case generated by the function generation unit of the test case generation device according to the first embodiment.

[0033] Figure 8 This is a diagram showing a test case generation function for generating a test case of a second step using a test case of a first step generated by a function generation unit of the test case generation device of the first embodiment.

[0034] Figure 9 This is a diagram showing a test case generation function for generating a two-step test case generated by the function generation unit of the test case generation device according to the first embodiment.

[0035] Figure 10 This is a diagram showing a test case generation function for generating a test case of the third step using the test case of the second step generated by the function generation unit of the test case generation device of the first embodiment.

[0036] Figure 11 This is a block diagram showing the structure of a test case generation device according to a modified example of the first embodiment. DETAILED DESCRIPTION

[0037] Hereinafter, embodiments of the present invention will be described using the accompanying drawings. In the drawings, identical or corresponding parts are denoted by the same reference numerals. In the description of the embodiments, identical or corresponding parts are appropriately omitted or simplified.

[0038] Implementation Method 1

[0039] ***Description of the structure***

[0040] Reference Figure 1 , describing the structure of the test case generation device 100 of this embodiment.

[0041] The test case generation device 100 is a computer. The test case generation device 100 includes a processor 910 and other hardware such as a memory 920 and an input / output device 930. The processor 910 is connected to the other hardware via a signal line and controls the other hardware.

[0042] The test case generation device 100 includes, as functional elements, a program acquisition unit 110, an included portion extraction unit 120, a generation control unit 130, a function generation unit 140, a non-symbolic generation unit 150, and a symbolic generation unit 160. The non-symbolic generation unit 150 and the symbolic generation unit 160 are also referred to as a test generation unit 170. Each functional element is implemented by software.

[0043] Processor 910 is a device that executes the test case generation program. The test case generation program implements the functions of program acquisition unit 110, part extraction unit 120, generation control unit 130, function generation unit 140, non-symbol generation unit 150, and symbol generation unit 160. Processor 910 is, for example, a CPU. "CPU" stands for Central Processing Unit.

[0044] Memory 920 is a device that stores the test case generation program. Memory 920 may be, for example, RAM, flash memory, or a combination thereof. "RAM" stands for Random Access Memory. Memory 920 also stores information such as the test target software program, its specifications, and the generated test cases.

[0045] The input / output device 930 is connected to an input device operated by a user to input data into the test case generation program, and a display that displays data output from the test case generation program on a screen. Examples of the input device include a mouse, keyboard, touch panel, or a combination of several or all of these. Examples of the display include an LCD (Liquid Crystal Display). "LCD" stands for Liquid Crystal Display. The display is specifically used to display test cases generated by the test generation unit 170.

[0046] The test case generation program is read from memory 920 by processor 910 and executed by processor 910. Memory 920 stores not only the test case generation program but also the operating system (OS). "OS" stands for Operating System. Processor 910 executes the test case generation program while executing the OS. Alternatively, part or all of the test case generation program may be incorporated into the OS.

[0047] The test case generation program and the OS may also be stored in an auxiliary storage device. Examples of the auxiliary storage device include an HDD, a flash memory, or a combination thereof. "HDD" stands for Hard Disk Drive. When the test case generation program and the OS are stored in the auxiliary storage device, they are loaded into the memory 920 and executed by the processor 910.

[0048] The test case generation device 100 may include a plurality of processors 910 instead of the processor 910. These plurality of processors 910 share the responsibility of executing the test case generation program. Each processor 910 is, for example, a CPU.

[0049] The data, information, signal values, and variable values ​​used, processed, or output by the test case generation program are stored in the memory 920 , an auxiliary storage device, or a register or cache memory within the processor 910 .

[0050] The "unit" in each of the program acquisition unit 110, the component extraction unit 120, the generation control unit 130, the function generation unit 140, the non-symbol generation unit 150, the symbol generation unit 160, and the test generation unit 170 can also be rewritten as "process," "procedure," or "step." Furthermore, the "process" in the program acquisition process, the component extraction process, the generation control process, the function generation process, the non-symbol generation process, the symbol generation process, and the test generation process can also be rewritten as "program," "program product," or "computer-readable recording medium storing the program."

[0051] The test case generation program causes a computer to execute each process, procedure, or step by rewriting the "part" of each part into a "process," "procedure," or "step." Furthermore, the test case generation method is performed by executing the test case generation program by the test case generation device 100.

[0052] The test case generation program may be provided by being stored in a computer-readable recording medium or provided as a program product.

[0053] The test case generation device 100 may be composed of a single computer or multiple computers. If the test case generation device 100 is composed of multiple computers, the functions of the program acquisition unit 110, the included portion extraction unit 120, the generation control unit 130, the function generation unit 140, the non-symbol generation unit 150, and the symbol generation unit 160 are distributed and implemented in each computer.

[0054] ***Function Description***

[0055] Here, refer to Figure 1 The operation outline of the test case generation device 100 of this embodiment will be described. The operation of the test case generation device 100 corresponds to the test case generation method of this embodiment.

[0056] Test case generation device 100 generates test case 23 for program 20, which executes one step of processing each time a value is input. Test case 23 is composed of values ​​corresponding to the number of test case inputs input to each of i (i is an integer greater than or equal to 2) steps of processing to be executed sequentially. Furthermore, this embodiment is applicable even if the test case consists of only one step (i.e., i = 1).

[0057] The program acquisition unit 110 reads the program 20 of the test target software, specification information, and information on the maximum number of steps from the outside via the memory 920 or the input / output device 930 .

[0058] The inclusion site extraction unit 120 extracts the inclusion site 21 included by the test from the source code of the program 20 .

[0059] The generation control unit 130 determines whether the memory 920 stores the (i-1)th test case, which is a test case from step 1 to step (i-1). If the (i-1)th test case is stored, the generation control unit 130 selects a first generation method M1 for generating the i-th test case, which is a test case from step 1 to step i, using the (i-1)th test case. If the memory 920 does not store the (i-1)th test case, the generation control unit 130 selects a second generation method M2 for generating each test case from step 1 to step i.

[0060] In this way, the generation control unit 130 determines a test case generation target step and controls the test case generation process by selecting a test case generation method in the step.

[0061] Function generation unit 140 maintains the state between steps and generates a test case generation function 22 that calls the program of the test target software, based on the test case generation method determined by generation control unit 130. Function generation unit 140 is also referred to as a test case generation function generation unit. Function generation unit 140 generates a test case generation function 22 that includes test case generation conditions and program calls, so that the i-th test case includes the included portion, i.e., the non-included portion, not included in the first through (i-1)th test cases.

[0062] The function generation unit 140 generates the test case generation function 22 using the (i-1)th test case according to the first generation method M1. The test case generation function 22 generates the i-th test case as a test case from step 1 to step i.

[0063] In addition, the function generation unit 140 generates a test case generation function 22 in accordance with the second generation method M2 while maintaining the internal state at the end time of the (i-1) step based on the execution of the (i-1) test case. The test case generation function 22 generates test cases from the 1st step to the i-th step respectively.

[0064] In this description, "test case at step i" or "test case generated in step i" refers to the test case input values ​​from step 1 to step i. Furthermore, the test case input values ​​from step 1 to step i are also referred to as the i-th test case. Similarly, the test case input values ​​from step 1 to step (i-1) are also referred to as the (i-1)-th test case.

[0065] Furthermore, "test case of step i" refers to the input value of a test case of step i.

[0066] The test generation unit 170 generates a test case for executing the i-th step according to the first generation method M1, while maintaining the internal state at the end time of the (i-1)th step based on the execution of the (i-1)th test case. The test generation unit 170 then generates the i-th test case by linking the (i-1)th test case with the test case for the i-th step, and stores it in the memory 920. Alternatively, the test generation unit 170 generates each test case from the first step to the i-th step using the test case generation function according to the second generation method M2. The test generation unit 170 then generates the i-th test case by linking the test cases from the first step to the i-th step, and stores it in the memory 920. In this way, the test generation unit 170 generates the i-th test case using the test case generation function 22.

[0067] Furthermore, the test generation unit 170 includes a non-symbolic generation unit 150 that generates test cases using non-symbolic execution, and a symbolic generation unit that generates test cases using symbolic execution. When the first generation method M1 is selected, the test generation unit 170 generates the i-th test case using the non-symbolic generation unit 150. The test generation unit 170 then determines whether any unincluded parts exist by executing a test using the i-th test case generated by the non-symbolic generation unit 150. If any unincluded parts exist, the test generation unit 170 generates the i-th test case using the symbolic generation unit 160 so as to include at least one of the unincluded parts. The test generation unit 170 then stores the i-th test case generated by the symbolic generation unit 160 in the memory 920. Furthermore, if the test generation unit 170 generates the i-th test case using the (i-1)th test case according to the first generation method M1, the (i-1)th test case used is discarded.

[0068] The non-symbolic generation unit 150 randomly generates input values ​​as test case candidates based on the specification information read by the program acquisition unit 110. The function generation unit 140 then uses the generated function to maintain the state between steps and extract inputs containing included objects, determining them as test cases. The non-symbolic generation unit 150 is also called a test generation unit based on non-symbolic execution.

[0069] Furthermore, the symbolic generator 160 generates test cases using the finite model checking method by using the function generated by the function generator 140 to maintain the state between steps based on the specification information read by the program acquisition unit 110. The symbolic generator 160 is also called a test generator based on symbolic execution.

[0070] ***Description of the action***

[0071] Next, refer to Figures 2 to 4 The operation of the test case generation device 100 according to this embodiment will be described in detail.

[0072] In step S1, the program acquisition unit 110 reads the program 20 of the test target software, its specifications, and its maximum number of steps. The process then proceeds to step S2. The specifications and maximum number of steps can be manually set by the user based on the software specifications. Alternatively, the specifications and maximum number of steps can be automatically set based on specifications written in a specific format. The specifications information includes information about the input variables of the program 20 of the test target software.

[0073] In step S2, the inclusion site extraction unit 120 extracts the inclusion sites within the test target software program according to the inclusion criteria to be used, sets these inclusion sites as not included, and proceeds to step S3. When using branches as the inclusion criteria during test execution to measure the branch inclusion rate, the inclusion site extraction unit 120 extracts branch sites. For example, in the case of an if statement, two sites are extracted as inclusion targets: those where the statement evaluates to "true" and those where it evaluates to "false." Since there is no dependency between steps S2 and S3, they can be executed in reverse order or in parallel.

[0074] In step S3 , the generation control unit 130 sets the variable i to 1, and proceeds to step S4 .

[0075] In step S4, the generation control unit 130 determines whether the value of the variable i is greater than the maximum number of steps or whether there are any unincluded locations. If the value of the variable i is greater than the maximum number of steps, the target test case generation step has ended, and the generation control unit 130 terminates the process. If the value of the variable i is less than the maximum number of steps but there are no unincluded locations, the generation control unit 130 determines that test case generation is unnecessary and terminates the process. In all other cases, the process proceeds to step S5.

[0076] In step S5, the generation control unit 130 checks whether the test case generated in step (i-1) exists. If so, the process proceeds to step S6. If not, the process proceeds to step S17.

[0077] In step S6, the non-symbol generator 150 links the test case generated in step (i-1) with the test case candidate for step i randomly generated based on the specification information read in step S1. The non-symbol generator 150 then sets the linked test case as the test case candidate for step i, and the process proceeds to step S7.

[0078] In step S7, the function generation unit 140 takes as input the i-step test case candidate generated by the non-symbol generation unit 150 and generates a test case generation function for executing the i-step test case. This function is used to execute the next step of the test target software program, maintaining the internal state at the end of each step.

[0079] In step S8, the non-symbolic generation unit 150 takes the i-step test case candidate generated in step S6 as input, maintains the inter-step state, and executes the i-step test target software program. The non-symbolic generation unit 150 then checks whether the i-step test case candidate is included. If the i-step test case is included, the non-symbolic generation unit 150 stores the i-step test case candidate as the i-step test case in the memory 920, and the process proceeds to step S9.

[0080] In step S9 , the non-symbol generating unit 150 excludes the portion included in the test case of step i determined in step S8 from the included portion, and the process proceeds to step S10 .

[0081] In step S10, the generation control unit 130 determines whether there are any unincluded locations. If there are no unincluded locations, the generation control unit 130 determines that test case generation is unnecessary and ends the process. Otherwise, the process proceeds to step S11.

[0082] In step S11, the function generation unit 140 receives the test case generated in step (i-1) as input for steps 1 through (i-1) of the program of the test target software. The function generation unit 140 then uses the test case generated in step (i-1) to generate a test case generation function for finding a test case at step i that includes the unincluded included portion through finite model checking, and then proceeds to step S12.

[0083] In step S12 , the symbol generation unit 160 uses the test case generation function generated in step S11 to maintain the state between steps and perform finite model checking to try to generate a test case for the i-th step, and then proceeds to step S13 .

[0084] In step S13, if the symbol generation unit 160 has generated a test case in step S12, it links the test case generated in step (i-1) used in step S11 with the test case of the i-th step generated in step S12. The symbol generation unit 160 then stores the linked test case in the memory 920 as the test case of the i-th step, and the process proceeds to step S14.

[0085] In step S14 , the symbol generation unit 160 excludes the locations included in the test case generated in step S12 from the included locations, and the process proceeds to step S15 .

[0086] In step S15 , when the test case of i steps is stored in the memory 920 in step S8 or step S13 , the generation control unit 130 excludes the test case of (i−1) steps used for generating the test case from the memory 920 .

[0087] In step S16, the generation control unit 130 determines whether there are any unincluded locations. If there are no unincluded locations, the generation control unit 130 determines that test case generation is unnecessary and ends the process. Otherwise, the process proceeds to step S17.

[0088] In step S17 , the non-symbol generation unit 150 randomly generates i-step test case candidates based on the specification information read in step S1 , and the process proceeds to step S18 .

[0089] In step S18, the function generation unit 140 obtains the i-step test case candidate generated by the non-symbol generation unit 150. The function generation unit 140 then generates a test case generation function for executing the next step of the test target software program of the i-step test case while retaining the internal state at the end of each step.

[0090] In step S19, the non-symbolic generation unit 150 obtains the i-step test case candidate generated in step S17. Taking the i-step test case candidate as input, the non-symbolic generation unit 150 maintains the state between steps and executes the i-step test target software program to confirm whether it contains any unincluded inclusions. If the unincluded inclusions are included, the i-step test case candidate is stored in the memory 920 as the i-step test case, and the process proceeds to step S20.

[0091] In step S20 , the non-symbol generating unit 150 excludes the portion already included in the test case of step i determined in step S19 from the included portion, and the process proceeds to step S21 .

[0092] In step S21, the generation control unit 130 determines whether there are any unincluded locations. If there are no unincluded locations, the generation control unit 130 determines that test case generation is unnecessary and ends the process. Otherwise, the process proceeds to step S22.

[0093] In step S22 , the function generation unit 140 generates a test case generation function for obtaining a test case including i steps of the included object not included in the program of the test target software by finite model checking, and the process proceeds to step S23 .

[0094] In step S23 , the symbol generation unit 160 uses the test case generation function generated in step S22 to maintain the state between steps and perform finite model checking to try to generate a test case for i steps, and then proceeds to step S24 .

[0095] In step S24 , when the symbol generation unit 160 has generated a test case in step S23 , the symbol generation unit 160 stores the test case in the memory 920 as a test case of step i, and the process proceeds to step S25 .

[0096] In step S25 , the symbol generation unit 160 excludes the portion included in the test case generated in step S23 from the included portion, and the process proceeds to step S26 .

[0097] In step S26 , the generation control unit 130 increments the variable i by 1 and proceeds to step S4 .

[0098] In the above test case generation process, the processes of steps S6 to S15 are examples of the first generation method M1 of generating the i-th test case using the (i-1)-th test case when the (i-1)-th test case exists.

[0099] The processing of steps S6 to S9 is a process for generating the i-th test case by non-symbolic execution. The processing of steps S11 to S15 is a process for generating the i-th test case by symbolic execution performed when there are also parts not included in the processing of steps S6 to S9.

[0100] Furthermore, the processing of steps S17 to S25 is an example of a second generation method M2 for generating an i-th test case by generating test cases from step 1 to step i when there is no (i-1)-th test case.

[0101] The processes of steps S17 to S20 generate the i-th test case by non-symbolic execution. The processes of steps S22 to S25 generate the i-th test case by symbolic execution performed when there are also parts not included in the processes of steps S17 to S20.

[0102] Furthermore, the series of processes from steps S6 to S26 is for the case where the i-th test case generated using the (i-1)-th test case according to the first generation method also contains an unincluded portion. In this case, the process from step S17 onwards becomes a process of generating test cases for all steps according to the second generation method and generating the i-th test case again.

[0103] If there are multiple test cases generated in step (i-1), steps S7 to S15 are performed for each of the test cases generated in step (i-1). These steps may be performed sequentially or in parallel. Furthermore, if multiple i-step test cases generated using the multiple test cases generated in step (i-1) only contain the same inclusion locations, any one of these test cases may be stored in memory 920 as the i-step test case.

[0104] The non-symbol generation unit 150 may generate a plurality of i-th step test case candidates for one (i-1)-step test case as a plurality of i-step test case candidates.

[0105] The generation control unit 130 may also execute steps S5 to S25 after previously determining in which step the included region can be included. For example, between steps S4 and S5, another method may be used to extract the included regions that can be included in step i from the included regions extracted in step S2. Subsequently, in steps S10, S16, and S21, it may be determined whether there are any unincluded regions among the included regions that can be included in step i.

[0106] In this embodiment, the symbolic generation unit 160 uses finite model checking to tentatively generate test cases. However, the symbolic generation unit 160 may also use other symbolic execution-based test case generation methods such as collaborative testing.

[0107] In this embodiment, the non-symbolic generation unit 150 randomly generates values ​​as test case candidates. However, the non-symbolic generation unit 150 may also use a test case generation method other than symbolic execution, such as a genetic algorithm or other machine learning method. Furthermore, this generation method may use the value of the (i-1)-step test case.

[0108] Next, the operation of the test case generation device 100 will be described using a specific example.

[0109] Figure 5 This is a diagram showing a portion of the test target software program 20 of the present embodiment 1. In the following example, Figure 5 The part of the program of the test object software shown in the figure. In addition, branch inclusion is used as the structural inclusion basis when executing the test. In addition, in the following description, the steps in brackets are the same as Figures 2 to 4 The steps correspond to .

[0110] The program acquisition unit 110 reads the program, specification information, and maximum number of steps of the test target software (step S1). In this example, only one function func is read, but a collection of multiple functions or files may also be read. The specification information includes information about the input variable sigA of the test target software program. The maximum number of steps is set to 4. The number of input variables depends on the test target software program and may be multiple.

[0111] After reading this information, the included part extraction unit 120 searches for the "true" and "false" branches of each if statement in the program of the test target software as branch parts to be included, and extracts a total of seven parts (step S2). Furthermore, it is assumed that the included part extraction unit 120 does not include the extracted parts. Figure 6 P1 to P7 are used to represent Figure 5 The indicated program should contain branch locations.

[0112] The generation control unit 130 sets the variable i to 1 (step S3). Since the value of variable i is 1, which is less than the maximum number of steps (4), and since the inclusion points P1 to P7 are not included, the generation control unit 130 determines that a test case needs to be generated (step S4). Next, the generation control unit 130 checks whether a test case generated in the previous step exists. Since the previous step does not exist in step 1, the generation control unit 130 determines that there is no test case (step S5).

[0113] The non-sign generation unit 150 generates a random value 50 as a test case candidate for one step for the input variable sigA based on the specification information (step S17 ).

[0114] The function generation unit 140 uses the value 50 as a test case candidate as an input of sigA, and generates a test case generation function for executing a program of the test target software in one step (step S18 ).

[0115] The non-symbol generator 150 executes the test case generation function to check whether the missing inclusion locations are included (step S19). Since P1 and P7 are included, 50 is stored as a single-step test case in the memory 920. The included inclusion locations P1 and P7 are then excluded from the included locations (step S20).

[0116] Since P2 to P6 exist as included parts, the generation control unit 130 proceeds to the next process (step S21 ).

[0117] The function generation unit 140 generates a test case generation function for finding a test case of one step including the non-included inclusion portion 21 of the program of the test target software through finite model checking (step S22 ).

[0118] Figure 7 This is an example of the generated test case generation function 22.

[0119] The symbol generation unit 160 uses Figure 7 The function performs finite model checking and tries to generate a test case for one step (step S23). Here, since there is no inclusion part that can be included in the first step, no new test case for one step is generated (steps S24, S25).

[0120] Next, the generation control unit 130 proceeds to the next step, i = 2 (step S26). It then confirms that the maximum number of steps has not yet been reached, and that there are branches P2 to P6 that are not included. Therefore, the generation control unit 130 checks whether there are test cases generated in one step. Here, since there are 50 test cases generated in one step, the generation of test cases using this test case generated in one step proceeds (step S5).

[0121] The non-symbolic generation unit 150 randomly generates a second-step test case candidate value of 50 through non-symbolic execution (step S6). Using the first-step test case 50, the non-symbolic generation unit 150 sets the first-step value of the input variable sigA to 50, and concatenates the value 50, which served as the second-step test case candidate, to create a second-step test case candidate (50, 50).

[0122] The function generation unit 140 generates a test case generation function for executing the program of the test target software in two steps by taking the test case candidate (50, 50) as the input of sigA (step S7).

[0123] The non-symbol generator 150 uses a test case generation function to maintain the state between steps and execute the second step after completing the first step. The non-symbol generator 150 then checks whether the excluded inclusion locations P2 to P6 are included (step S8). As a result, in step 2, the value is 50, and inclusion location P2 is included. Therefore, (50, 50) is stored in memory 920 as a test case for step 2, and P2 is excluded from the inclusion locations (step S9).

[0124] Since P3 to P6 exist as included parts, the generation control unit 130 proceeds to the next process (step S10 ).

[0125] The function generation unit 140 uses the value 50 of the test case of step 1 as the test case of step 1 and generates a test case generation function for obtaining the test case of step 2 including the unincluded included portion through finite model checking (step S11 ).

[0126] Figure 8This is an example of the generated test case generation function 22.

[0127] The symbol generation unit 160 uses Figure 8 The function performs finite model checking while maintaining the state of step 1 and checking step 2, attempting to generate a test case for step 2 (step S12). As a result, a value less than 10 (e.g., 0) is generated as the test case for step 2, i.e., the value of the input variable sigA for step 2, as shown in the example of the inclusion location P3. Therefore, the symbol generation unit 160 stores (50, 0) as a test case for step 2 in the memory 920 and excludes P3 from the inclusion location (steps S13 and S14).

[0128] Generation control unit 130 removes the one-step test case 50 used to generate the two-step test case from memory (step S15). As a result, only the two-step test cases (50, 50) and (50, 0) remain in memory 920 as test cases. Since inclusion points P4 to P6 exist, generation control unit 130 proceeds to the next step (step S16).

[0129] The non-symbol generation unit 150 generates random values ​​(0, 0) as two-step test case candidates (step S17 ).

[0130] The function generation unit 140 uses the test case candidate (0, 0) as an input of sigA to generate a test case generation function for executing the program of the test target software in two steps (step S18).

[0131] The non-symbol generator 150 uses the test case generation function to maintain the state between steps and execute the second step after completing the first step (step S19). The non-symbol generator 150 checks whether the unincluded inclusion portion is included. However, in this case, since the unincluded inclusion portion is not included, the test case candidate (0, 0) is discarded (steps S20 and S21).

[0132] Since the included parts P4 to P6 still exist, the generation control unit 130 proceeds to the next process.

[0133] The function generation unit 140 generates a test case generation function for obtaining a two-step test case including an unincluded included portion through finite model checking (step S22 ).

[0134] Figure 9 This is an example of the generated test case generation function 22.

[0135] The symbol generation unit 160 uses Figure 9The function performs finite model checking by maintaining the state of step 1 and checking step 2, attempting to generate a two-step test case (step S23). As a result, for example, (150, 0) is generated as a two-step test case that includes inclusion location P6. Therefore, symbol generation unit 160 stores (150, 0) in memory 920 and excludes P6 from the inclusion location (steps S24 and S25).

[0136] Next, the generation control unit 130 proceeds to the next step, i = 3 (step S26). It then confirms that the maximum number of steps has not yet been reached, and that branches P4 and P5 are not included. Therefore, it checks whether a test case generated in step 2 exists. Here, since (50, 50), (50, 0), and (150, 0) exist as test cases generated in step 2, the generation control unit 130 proceeds to test case generation using these (50, 50), (50, 0), and (150, 0) (steps S4 and S5).

[0137] The non-symbol generation unit 150 uses the two-step test cases (50, 50), (50, 0), and (150, 0) to randomly generate test case candidates for the third step, similar to the case of the second step. The non-symbol generation unit 150 then uses the test case candidates for the third step to generate test case candidates for the third step (step S6). For example, the test case candidates for the third step are (50, 50, 50), (50, 0, 50), and (150, 0, 50).

[0138] The function generation unit 140 uses the test case candidates (50, 50, 50), (50, 0, 50), and (150, 0, 50) as inputs of sigA to generate a test case generation function for executing the three-step test target software program (step S7).

[0139] The non-symbol generator 150 uses a test case generation function to maintain the state between steps and execute the test (step S8). Specifically, upon completing one step, the non-symbol generator 150 maintains the state and executes the second step; upon completing two steps, the non-symbol generator 150 maintains the state and executes the third step. The non-symbol generator 150 then checks whether the excluded inclusion points P4 and P5 are included. While both are capable of including P4, one of them (50, 50, 50) is stored as a three-step test case. The non-symbol generator 150 then excludes P4 from the list of included points (steps S8 and S9).

[0140] Since P5 exists as an included part, the generation control unit 130 proceeds to the next process (step S10 ).

[0141] The function generator 140 generates a test case generation function using the two-step test case as the test cases for the first and second steps (step S11). The function generator 140 generates a test case generation function for determining a test case for the third step that includes the unincluded inclusion portion P5 through finite model checking. Here, the two-step test cases are (50, 50), (50, 0), and (150, 0).

[0142] Figure 10 This is an example of the test case generation function 22 using a two-step test case (50, 50).

[0143] For example, in Figure 10 In the test case generation function shown, after executing step 1, step 2 is called without initialization. Therefore, after executing step 1, the values ​​of global variables or static variables used by the test target function func remain unchanged before processing of the function func in step 2. Therefore, for example, the test generation unit 170 performing symbolic execution analyzes the uninitialized function func in step 2 after analyzing the function func in step 1. This operation generates test cases while maintaining internal state.

[0144] The symbol generation unit 160 uses Figure 10 The function performs finite model checking while maintaining the states of steps 1 and 2 and checking step 3, attempting to generate a test case for step 3 (step S12). As a result, a value less than 30 (e.g., 0) is generated, such as the value of the input variable sigA in step 3, including the inclusion location P5. Therefore, the symbol generation unit 160 stores (50, 50, 0) as a test case for step 3 in the memory 920 and excludes P5 from the inclusion location (steps S13, S14, and S15).

[0145] Next, the generation control unit 130 excludes the two-step test case (50, 50) used to generate the three-step test case from the memory 920. As a result, the test cases (50, 0), (150, 0), (50, 50, 50), and (50, 50, 0) exist in the memory 920. Since there are no more included locations, the generation control unit 130 determines that test case generation is unnecessary and ends the process (step S16).

[0146] As described above, the test case generation device 100 of this embodiment analyzes software that executes one step of processing each time a value is input, and generates a sequence of values ​​input for testing the software as a test case.

[0147] If a test case for step (i-1) exists, the generation control unit 130 selects a method for generating a test case using the test case for step (i-1). At this time, the generation control unit selects a method for generating a test case for step i while maintaining the internal state at the end of step (i-1) based on the execution of the test case. If a test case for step (i-1) does not exist, the generation control unit selects a test case generation method for generating test cases from step 1 to step i.

[0148] The function generation unit 140 generates a test case generation function including a test case generation condition and a call to a program of the test target software, according to the test case generation method selected by the generation control unit.

[0149] The test generation unit 170 generates test cases by combining symbolic execution or non-symbolic execution based on the test case generation method selected by the generation control unit 130, maintaining inter-step states using a test case generation function. If a test case for step (i-1) is generated using a test case for step (i-1), the used test case for step (i-1) is discarded.

[0150] ***Explanation of the Effects of This Implementation***

[0151] In the test case generation device of this embodiment, when test case generation involving multiple steps is required, the test case generation results of the previous step are first used to test non-symbolic execution and symbolic execution test case generation. Consequently, in step 3, the test case generation device of this embodiment performs a single-step analysis using only the test cases generated in step 2 while maintaining the internal state at the end of step 2. This has the following effect: Specifically, it is possible to include one inclusion site in both non-symbolic execution and symbolic execution test case generation, and terminate test case generation. In the specific example above, the internal state at the end of step 2 is mode = 1 and sub_mode = 2. That is, by performing a single-step analysis while maintaining mode = 1 and sub_mode = 2, it is possible to include one inclusion site in both non-symbolic execution and symbolic execution test case generation, and terminate test case generation.

[0152] Furthermore, without using the method of this embodiment, test cases must be generated experimentally for all three-step paths. Therefore, in non-symbolic execution-based test case generation, the number of test case candidates becomes enormous, requiring time to generate. Furthermore, in symbolic execution-based test case generation, analyzing the three-step paths and extracting the values ​​that satisfy these three-step path analyses takes time.

[0153] Furthermore, in the test case generation device of this embodiment, when i = 2, after test case generation using the test case generation results of step 1, an unincluded inclusion location P6 remains, even though it can actually be included in step 2. In contrast, the test case generation device of this embodiment generates a test case that can include P6 by performing symbolic execution-based test case generation from step 1 to step 2. In this way, by combining the method of generating a test case for step i using the test case of step (i-1) and the method of generating test cases covering all steps i, it is possible to reliably generate a test case that includes the location that can be included in step i.

[0154] Furthermore, in the test case generation device of this embodiment, when i = 2, a test case is generated using a one-step test case. The inclusion of the inclusion object is confirmed and stored in memory 920 as a two-step test case. Furthermore, the test case is stored in memory 920 as a test case, and the used one-step test case is deleted. This allows for the deletion of redundant test cases including the inclusion object, reducing the total number of test cases and reducing testing man-hours such as the time required to examine the expected output value and the test execution time.

[0155] ***Other structures***

[0156] <Variation 1>

[0157] In this embodiment, the functions of the program acquisition unit 110, the included portion extraction unit 120, the generation control unit 130, the function generation unit 140, the non-symbol generation unit 150, and the symbol generation unit 160 are implemented by software. As a modified example, the functions of the program acquisition unit 110, the included portion extraction unit 120, the generation control unit 130, the function generation unit 140, the non-symbol generation unit 150, and the symbol generation unit 160 may also be implemented by hardware.

[0158] Figure 11 It is a structural diagram of a test case generation device 100 according to a modified example of the present embodiment.

[0159] The test case generating device 100 includes hardware such as an electronic circuit 909 , a memory 920 , and an input / output device 930 .

[0160] The electronic circuit 909 is a dedicated electronic circuit that realizes the functions of the program acquisition unit 110 , the part extraction unit 120 , the generation control unit 130 , the function generation unit 140 , the non-symbol generation unit 150 , and the symbol generation unit 160 .

[0161] Specifically, electronic circuit 909 is a single circuit, a complex circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA, an ASIC, or an FPGA. GA is the abbreviation for Gate Array. ASIC is the abbreviation for Application Specific Integrated Circuit. FPGA is the abbreviation for Field-Programmable Gate Array.

[0162] The functions of the program acquisition unit 110 , the included portion extraction unit 120 , the generation control unit 130 , the function generation unit 140 , the non-symbol generation unit 150 , and the symbol generation unit 160 may be implemented by a single electronic circuit or distributed across a plurality of electronic circuits.

[0163] As another variation, some functions of the program acquisition unit 110, the included portion extraction unit 120, the generation control unit 130, the function generation unit 140, the non-symbol generation unit 150, and the symbol generation unit 160 may be implemented by electronic circuits, and the remaining functions may be implemented by software.

[0164] As another modification, some or all of the functions of the program acquisition unit 110 , the included portion extraction unit 120 , the generation control unit 130 , the function generation unit 140 , the non-symbol generation unit 150 , and the symbol generation unit 160 may be implemented by firmware.

[0165] Processors and electronic circuits are also referred to as processing circuits. Specifically, in test case generation device 100, the functions of program acquisition unit 110, inclusion portion extraction unit 120, generation control unit 130, function generation unit 140, non-symbol generation unit 150, and symbol generation unit 160 are implemented by processing circuits.

[0166] In the first embodiment above, the various components of the test case generation device are described as independent functional blocks. However, the structure of the test case generation device does not need to be the same as that of the above embodiment. The functional blocks of the test case generation device can have any structure as long as they can achieve the functions described in the above embodiment. Furthermore, the test case generation device can also be a system composed of multiple devices rather than a single device.

[0167] Furthermore, a plurality of parts in Embodiment 1 may be implemented in combination. Alternatively, a portion of these embodiments may be implemented. Furthermore, these embodiments may be implemented in any combination as a whole or in part.

[0168] That is, in the first embodiment, it is possible to freely combine the respective embodiments, modify any constituent elements of the respective embodiments, or omit any constituent elements of the respective embodiments.

[0169] The above embodiments are merely preferred examples in nature and are not intended to limit the scope of the present invention, the scope of its applications, or the scope of its uses.

[0170] Label Description

[0171] 20: Program; 21: Inclusion part; 22: Test case generation function; 23: Test case; 100: Test case generation device; 110: Program acquisition unit; 120: Inclusion part extraction unit; 130: Generation control unit; 140: Function generation unit; 150: Non-symbol generation unit; 160: Symbol generation unit; 170: Test generation unit; 909: Electronic circuit; 910: Processor; 920: Memory; 930: Input / output device; M1: First generation method; M2: Second generation method.

Claims

1. A test case generating device, the test case generating device generating a test case that is input to a program that executes one step of processing each time a value is input, the test case being composed of the values ​​of the number of test case inputs input to each of the i-step processings executed sequentially, i being an integer greater than 2, wherein, The test case generating device has: a memory that stores data; a generation control unit, which, when the memory stores an (i-1)th test case as a test case from step 1 to step (i-1), selects a first generation method of using the (i-1)th test case to generate an i-th test case as a test case from step 1 to step i; as well as A test generation unit, which generates a test case for executing the i-th step in accordance with the first generation method while maintaining an internal state at the end time of the (i-1) step based on the execution of the (i-1) test case, and generates the i-th test case and stores it in the memory by connecting the (i-1) test case and the test case of the i-th step.

2. The test case generating device according to claim 1, wherein: The test case generating device comprises: an inclusion site extraction unit that extracts an inclusion site included by the test from the source code of the program; a function generating unit for generating a test case generating function including a test case generating condition and calling the program in such a manner that the i-th test case includes an included portion not included by the (i-1)-th test case, that is, a non-included portion; and The test generation unit generates the i-th test case using the test case generation function.

3. The test case generating device according to claim 2, wherein: When the memory does not store the (i-1)th test case, the generation control unit selects a second generation method for generating each test case from step 1 to step i, The function generation unit generates a test case generation function for generating test cases from the first step to the i-th step respectively in accordance with the second generation method while maintaining an internal state at the end time of the (i-1)th step based on the execution of the (i-1)th test case. The test generation unit generates each test case from step 1 to step i using the test case generation function, and generates the i-th test case by linking the test cases from step 1 to step i and stores the test case in the memory.

4. The test case generating device according to claim 3, wherein: The test generation unit includes a non-symbolic generation unit for generating test cases by non-symbolic execution and a symbolic generation unit for generating test cases by symbolic execution. When the first generation method is selected, the non-symbolic generation unit generates the i-th test case, and determines whether the unincluded part exists by executing a test using the i-th test case generated by the non-symbolic generation unit. When the unincluded part exists, the symbolic generation unit generates the i-th test case in a manner that includes at least any one of the unincluded parts, and stores the i-th test case generated by the symbolic generation unit in the memory.

5. The test case generating device according to any one of claims 1 to 4, wherein: When the test generation unit generates the i-th test case by using the (i-1)-th test case according to the first generation method, the test generation unit discards the used (i-1)-th test case.

6. A test case generation method of a test case generation device, the test case generation device generating a test case that is input to a program that executes one step of processing each time a value is input, the test case being composed of the values ​​of the number of test case inputs input to each of i steps of processing executed sequentially, i being an integer greater than 2, wherein: The test case generating device has a memory for storing data. When the memory stores an (i-1)th test case as a test case from step 1 to step (i-1), the generation control unit selects a first generation method of using the (i-1)th test case to generate an i-th test case as a test case from step 1 to step i, The test generation unit generates a test case for executing the i-th step in accordance with the first generation method while maintaining an internal state at the end time of the (i-1) step based on the execution of the (i-1) test case, and generates the i-th test case and stores it in the memory by connecting the (i-1) test case and the test case of the i-th step.

7. A computer-readable recording medium having a test case generation program of a test case generation device recorded thereon, the test case generation device generating a test case that is input into a program that executes one step of processing each time a value is input, the test case being composed of the values ​​of the number of test case inputs input into each of i steps of processing executed sequentially, i being an integer greater than 2, wherein: The test case generating device has a memory for storing data. The test case generation program causes the test case generation device as a computer to perform the following processing: generation control processing, when the memory stores an (i-1)th test case as a test case from step 1 to step (i-1), selecting a first generation method of using the (i-1)th test case to generate an i-th test case as a test case from step 1 to step i; as well as The test generation process generates a test case for executing the i-th step in accordance with the first generation method while maintaining the internal state at the end time of the (i-1) step based on the execution of the (i-1) test case, and generates the i-th test case and stores it in the memory by linking the (i-1) test case and the test case of the i-th step.

Images