Automatic test scenario generator

Abstract

The objective of the present disclosure is automatic generation of regression test scenarios which reduces time required in a case of manual work. An automatic test scenario generation device (101) according to the present disclosure comprises: a scenario generation unit (13) that generates a regression test scenario; a command execution unit (11) that outputs, to a test object (16), a first command group formed of a plurality of first commands and the regression test scenario and obtains a response result from the test object (16); and a recording unit (12) that records a command log. The scenario generation unit (13) generates, on the basis of the command log of the first command group, a first regression test scenario in which a plurality of second commands identical to the plurality of first commands are arranged in the same order and with the same wait times as those of the plurality of first commands in the first command group, and reduces the wait time for at least one second command in a second regression test scenario compared to the first regression test scenario to such an extent that the response result is consistent with that of the first regression test scenario.

Description

[Technical field]

[0001] The present disclosure relates to an automatic test scenario generation device. [Background technology]

[0002] Technologies for automating and streamlining software testing include a technology for recording previous test scenarios and reusing them in regression testing, and a technology for automating testing by sequentially executing test scenarios generated by a user and automatically generated test scenarios.

[0003] Patent Document 1 discloses an automatic regression test execution system that records an operation log during testing of a GUI application and automatically executes regression testing by replaying the log. The automatic regression test execution system of Patent Document 1 manages buttons and the like as resources and updates the coordinates of each resource before testing, so that even if a coordinate change occurs, testing can be performed using past operation logs.

[0004] Patent Document 2 discloses a regression test scenario generating device that automatically generates regression test scenarios by recording operation logs for GUI applications. This regression test scenario generating device records operations only when the window to be tested is active, thereby preventing unnecessary operations from being recorded. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 5314438 [Patent Document 2] Patent No. 6312559 Summary of the Invention [Problem to be solved by the invention]

[0006] According to the regression test automatic execution system of Patent Document 1, even if the resource allocation of a GUI application changes due to a specification change, it is possible to reuse regression test scenarios used in the past, but it is not possible to reduce the time required for regression testing.

[0007] According to the regression test scenario generation device of Patent Document 2, unnecessary operations are not recorded when recording a regression test scenario, so the time required for regression testing can be reduced. However, there is room for reducing waste caused by manual work in operations that need to be recorded. For example, there is the time spent visually checking the results, the time spent moving the mouse cursor when operating the GUI, and the delay in work caused by checking the procedure.

[0008] The present disclosure has been made to solve the above problems, and aims to automatically generate regression test scenarios while reducing the time required for manual work. [Means for solving the problem]

[0009] The automatic test scenario generation device disclosed herein is an automatic test scenario generation device that generates regression test scenarios for a test target, and includes a scenario generation unit that generates the regression test scenarios, a command execution unit that outputs a first command group consisting of a plurality of first commands sequentially input from outside the automatic test scenario generation device, and the regression test scenario to the test target and acquires a response result from the test target, and a recording unit that records a command log including an execution time and a response result from the test target for each command included in the first command group and the regression test scenario, wherein the regression test scenario includes a first regression test scenario and a second regression test scenario, and the scenario generation unit generates a first regression test scenario based on the command log of the first command group, in which a plurality of second commands identical to the plurality of first commands are arranged in the same order and wait times as the plurality of first commands in the first command group, and reduces the wait time of at least one second command in the second regression test scenario compared to the first regression test scenario, to the extent that the response result matches the first regression test scenario, based on the command logs of the first regression test scenario and the second regression test scenario. Effect of the Invention

[0010] According to the automatic test scenario generation device of the present disclosure, it is possible to automatically generate regression test scenarios while reducing the time required for manual work. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 illustrates a configuration of an automatic test scenario generation device. [Diagram 2] FIG. 11 is a diagram illustrating an example of a command log. [Diagram 3] 11 is a diagram illustrating an example of a threshold value used for determining whether a response result to a command matches; [Figure 4] 11A and 11B are diagrams illustrating examples of threshold candidates used for determining whether a response result to a command matches; [Diagram 5]It is a diagram illustrating commands or combinations of commands and waiting times preset in the test scenario automatic generation device. [Figure 6] It is a diagram showing the hardware configuration of the test scenario automatic generation device. [Figure 7] It is a diagram showing the hardware configuration of the test scenario automatic generation device.

Mode for Carrying Out the Invention

[0012] <A. Embodiment 1> <A-1. Configuration> FIG. 1 is a configuration diagram of a test scenario automatic generation device 101 according to Embodiment 1. The test scenario automatic generation device 101 includes a command execution unit 11, a recording unit 12, a scenario generation unit 13, and a playback unit 14. Outside the test scenario automatic generation device 101, there are a test target 16 and an operator 15 who tests it.

[0013] The test scenario automatic generation device 101 and the test target 16 are not limited to either S / W or H / W. The test scenario automatic generation device 101 and the test target 16 may be, for example, S / W installed on a personal computer, or H / W such as an automobile, a motor, or an elevator. Also, the operator 15 is not limited to a person, and may be S / W on a personal computer that automatically tests the test target 16, or H / W such as a controller that controls a motor.

[0014] <A-2. Operation> Hereinafter, the automatic generation process of the test scenario by the test scenario automatic generation device 101 will be described.

[0015] First, the command execution unit 11 executes the command input from the operator 15. Specifically, the command execution unit 11 outputs the command to the test target 16 and acquires a command response from the test target 16.

[0016] The recording unit 12 records a command log, which is the result of command execution by the command execution unit 11, in the recording unit 12. The command log includes the content of the command, an execution timestamp which is the time when the command was sent from the command execution unit 11 to the test subject 16, a response timestamp which is the time when the command execution unit 11 received a command response from the test subject 16, and the command response. Note that the execution timestamp and the response timestamp are expressed as elapsed time when the recording start time of the recording unit 12 is set to 0.

[0017] An example of a command log is shown in Fig. 2. The command log shown in Fig. 2 includes the command number, the recording time (seconds), the command content, the response result, and the difference in time (seconds) from the previous command. Here, the recording time represents the time when the command was sent from the command execution unit 11 to the test subject 16.

[0018] Hereinafter, a command inputted to command execution unit 11 by operator 15 will be referred to as a first command, and a command group consisting of a plurality of first commands will be referred to as a first command group.

[0019] Next, the scenario generating unit 13 generates a first regression test scenario that reproduces the order and execution timing of the multiple first commands in the first command group based on the command log of the first command group. That is, the first regression test scenario is a scenario in which multiple second commands are executed in order, but the multiple second commands are the same as the multiple first commands, and the order of the multiple second commands in the first regression test scenario is the same as the order of the multiple first commands in the first command group. Also, the waiting time of the multiple second commands in the first regression test scenario is the same as the waiting time of the multiple first commands in the first command group. Here, the waiting time of a command refers to the difference between the execution time of the command and the execution time of the previous command. Also, a first command that is the same as a second command is called a first command corresponding to the second command.

[0020] The first regression test scenario is output from the scenario generation unit 13 to the playback unit 14. The playback unit 14 plays back the first regression test scenario to perform a regression test. That is, a plurality of second commands are input from the playback unit 14 to the command execution unit 11 in accordance with the order and waiting time defined in the first regression test scenario, and are executed by the command execution unit 11.

[0021] The recording unit 12 records in the recording unit 12 a command log of the second command, which is the execution result of the first regression test scenario.

[0022] In this way, the command log of the first command, which is the execution result of the first command group, and the command log of the second command, which is the execution result of the first regression test scenario, are recorded in the recording unit 12. The scenario generating unit 13 refers to both command logs and checks whether the response result of the second command matches the response result of the corresponding first command.

[0023] If at least one response result of the second command does not match the response result of the corresponding first command, the scenario generator 13 saves the first regression test scenario as a failure scenario and ends the process.

[0024] On the other hand, if the response results of all the second commands match the response results of the corresponding first commands, the scenario generator 13 stores the first regression test scenario as a successful scenario.

[0025] The scenario generating unit 13 generates a regression test scenario that is based on the successful scenario and reduces the waiting time of the second command as a second regression test scenario. The process of generating the second regression test scenario will be described below.

[0026] First, the scenario generating unit 13 creates a second regression test scenario by setting the initial value of the waiting time of each second command to 0, that is, to the minimum, without changing the execution order of each second command in the successful scenario.

[0027] The second regression test scenario is output from the scenario generation unit 13 to the playback unit 14. The playback unit 14 plays back the second regression test scenario to perform a regression test. That is, a plurality of second commands are input from the playback unit 14 to the command execution unit 11 in accordance with the order and waiting time defined in the second regression test scenario, and are executed by the command execution unit 11.

[0028] The recording unit 12 records in the recording unit 12 a command log of the second command, which is the execution result of the second regression test scenario.

[0029] In this way, the recording unit 12 records the command log of the first command which is the execution result of the first command group, the command log of the second command which is the execution result of the first regression test scenario, and the command log of the second command which is the execution result of the second regression test scenario. The scenario generating unit 13 compares the command log of the first command which is the execution result of the first command group with the command log of the second command which is the execution result of the second regression test scenario, and checks whether the response result of the second command in the second regression test scenario matches the response result of the corresponding first command. Note that here, the scenario generating unit 13 may compare the command log of the second command which is the execution result of the first regression test scenario with the command log of the second command which is the execution result of the second regression test scenario, and check whether the response result of the second command in the second regression test scenario matches the response result of the second command in the first regression test scenario.

[0030] When the response result of a second command in the second regression test scenario does not match the response result of the corresponding first command, the scenario generation unit 13 sets a wait time between the mismatched second command and the second command immediately preceding it, similar to that of the corresponding first command, based on the command log. That is, the scenario generation unit 13 changes the wait time of the second command in the second regression test scenario, the response result of which does not match that of the corresponding first command, to the same value as the wait time of the corresponding first command.

[0031] Then, the scenario generation unit 13 resets the test target 16 and retries the regression test using the second regression test scenario. If the response result of the second command in the second regression test scenario does not match the response result of the corresponding first command again, the scenario generation unit 13 sets a waiting time similar to that of the corresponding first command based on the command log between the second command immediately preceding the non-matching second command and the second command two before that. In this way, the scenario generation unit 13 sets the waiting times of the second commands before the non-matching second command to the same values as the waiting times of the corresponding first commands in order until the response result of the second command in the second regression test scenario matches the response result of the corresponding first command.

[0032] If the response result of each second command in the second regression test scenario matches the response result of the corresponding first command, the scenario generation unit 13 saves the second regression test scenario as an improved successful scenario. In this case, the waiting time between the second commands in the second regression test scenario is reduced compared to the first regression test scenario.

[0033] <A-3. Variant 1> When the same command is continuously input from the operator 15 to the command execution unit 11, that is, when a plurality of the same first commands are consecutive in the first command group. In this case, the scenario generation unit 13 may delete the second commands other than the last one among the plurality of consecutive same second commands corresponding to the plurality of consecutive same first commands from the second regression test scenario.

[0034] Thereby, in the second regression test scenario, it is possible to eliminate the waste of issuing the same command multiple times to check the status while polling the status of the test target 16, and it is possible to shorten the generation time of the second regression test scenario.

[0035] <A-4. Variant 2> The recording unit 12 may be set in advance with commands for which recording of command logs is unnecessary. That is, the recording unit 12 records command logs only for predetermined commands. As a result, the scenario generation unit 13 can reduce the generation time of the second regression test scenario. Note that the recording unit 12 may be set such that recording of command logs is unnecessary not only for individual commands but also for combinations of a plurality of commands.

[0036] <A-5. Effect> As described above, the test scenario automatic generation device 101 according to Embodiment 1 includes a scenario generation unit 13 that generates a regression test scenario, a first command group including a plurality of first commands sequentially input from outside the test scenario automatic generation device 101, and a command execution unit 11 that outputs the regression test scenario to a test target and acquires a response result of the test target, and a recording unit 12 that records a command log including an execution time and a response result of the test target for each command included in the first command group and the regression test scenario. The regression test scenario includes a first regression test scenario and a second regression test scenario. The scenario generation unit 13 generates a first regression test scenario in which a plurality of second commands identical to the plurality of first commands are arranged in the same order and waiting time as the plurality of first commands in the first command group based on the command log of the first command group, and based on the command logs of the first regression test scenario and the second regression test scenario, reduces the waiting time of at least one second command in the second regression test scenario compared to the first regression test scenario within a range where the response result matches the first regression test scenario.

[0037] Therefore, according to the test scenario automatic generation device 101, the first regression test scenario can be automatically generated from the command log recording the operations of the operator 15, and further, the second regression test scenario that saves unnecessary time caused by manual work can be automatically generated.

[0038] <B. Embodiment 2> <B-1. Configuration> The configuration of the test scenario automatic generation device 102 according to Embodiment 2 is as shown in FIG. 1, and is the same as that of the test scenario automatic generation device 101 according to Embodiment 1.

[0039] <B-2. Operation> In the scenario generation unit 13 of Embodiment 1, in the second regression test scenario, the waiting time of the second command whose response result does not match was changed from the initial value of 0 to the same value as the waiting time of the corresponding first command.

[0040] In contrast, the scenario generation unit 13 of Embodiment 2 sets a unit waiting time shorter than the waiting time of the first command, and in the second regression test scenario, finely adjusts the waiting time of the second command whose response result does not match by one unit waiting time at a time.

[0041] The scenario generation unit 13 of Embodiment 2 adjusts the waiting time of the command whose response result does not match by any of the following methods.

[0042] (1) The scenario generation unit 13 sets the initial value of the waiting time of each second command in the second regression test scenario to 0. Then, the scenario generation unit 13 adds the unit waiting time one by one from the initial value to the waiting time of the second command until the response result matches the corresponding first command. For example, for command No. 4 shown in FIG. 2, if the unit waiting time is 5 seconds, the waiting time becomes 5 seconds after the first mismatch. Hereinafter, every time the mismatch is repeated, 5 seconds of the unit waiting time is added, and the waiting time increases to 10 seconds, 15 seconds, 20 seconds... In this way, the scenario generation unit 13 finds the minimum waiting time for which the response results match.

[0043] (2) The scenario generation unit 13 sets the initial value of the waiting time of each second command in the second regression test scenario to the same value as the waiting time of the corresponding first command. Then, the scenario generation unit 13 subtracts the unit waiting time from the waiting time of each second command one by one until the response results do not match between the corresponding first command and the second command. For example, for command No. 4 shown in FIG. 2, if the unit waiting time is 5 seconds, after the first mismatch, the waiting time becomes 20 seconds - 5 seconds = 15 seconds. Subsequently, each time the mismatch is repeated, 5 seconds of the unit waiting time is subtracted, and the waiting time decreases to 10 seconds, 5 seconds, and 0 seconds. In this way, the scenario generation unit 13 finds the minimum waiting time at which the response results match.

[0044] The unit waiting time can be obtained, for example, by multiplying the waiting time coefficient determined in advance by the operator 15, who is the user, by the waiting time of the corresponding first command. Assuming that the waiting time coefficient is 25% for command No. 4 shown in FIG. 2, the unit waiting time is (the differential time of 20 seconds from the previous command) × (the waiting time coefficient of 25%) = 5 seconds.

[0045] Alternatively, the unit waiting time is set in advance by the operator 15.

[0046] <B-3. Effect> According to the test scenario automatic generation device 102 of the second embodiment, since the scenario generation unit 13 adjusts the waiting time of the second command in the second regression test scenario by the unit waiting time shorter than the waiting time of the corresponding first command, it is possible to automatically generate a second regression test scenario with a shorter test time than in the first embodiment.

[0047] <C. Third Embodiment> <C-1. Configuration> The configuration of the test scenario automatic generation device 103 according to the third embodiment is as shown in FIG. 1, which is the same as the test scenario automatic generation device 101 according to the first embodiment.

[0048] <C-2. Operation> In the third embodiment, a necessary waiting time is set in advance for a single command or a combination of commands. That is, the scenario generating unit 13 sets a predetermined waiting time for a predetermined second command or a second or subsequent second command in a plurality of predetermined consecutive second commands in the second regression test scenario. For a second command or a combination of second commands for which a waiting time is not set in advance, the scenario generating unit 13 sets a waiting time in the second regression test scenario using the method described in the first and second embodiments. Note that the scenario generating unit 13 may also adjust and set a waiting time for a second command or a combination of second commands for which a waiting time is set in advance using the method described in the first and second embodiments. It is assumed that an operator can set in advance for each command or combination of commands whether or not to adjust the waiting time.

[0049] If the regression test using the second regression test scenario does not produce results similar to those of the regression test using the first regression test scenario, the test scenario automatic generation device 103 determines that there is a problem with the preset waiting time, notifies the operator of this, and terminates the regression test using the second regression test scenario.

[0050] In addition, if the waiting time preset in the second regression test scenario is longer than the waiting time in the first regression test scenario, the scenario generation unit 13 may notify the operator whether or not to replace the waiting time setting, allowing the operator to select which waiting time to use.

[0051] In the example of FIG. 2, the command following the "Set A" command of Command No. 1 is the "Get A" command of Command No. 2. Here, assuming that according to the specification of the object under test 16, it takes 1 second until the result of the "Set A" command is reflected in the result of the "Get A command". In this case, the scenario generation unit 13 sets the waiting time between the "Set A" command and the "Get A" command to 1 second. Then, when the scenario generation unit 13 modifies the first regression test scenario to generate the second regression test scenario, the waiting time between the "Set A" command and the "Get A" command is set to 1 second.

[0052] <C-3. Effect> In the test scenario automatic generation device 103 according to Embodiment 3, the scenario generation unit 13 sets a predetermined waiting time for the second command determined in advance, or for the second and subsequent second commands in a plurality of consecutive second commands determined in advance, in the second regression test scenario. Therefore, according to the test scenario automatic generation device 101 according to Embodiment 3, the time required for automatically generating the second regression test scenario can be shortened.

[0053] <D. Embodiment 4> <D-1. Configuration> The configuration of the test scenario automatic generation device 104 according to Embodiment 4 is as shown in FIG. 1, and is the same as that of the test scenario automatic generation device 101 according to Embodiment 1. The test scenario automatic generation device 104 according to Embodiment 4 is such that in the test scenario automatic generation devices 101-103 according to any one of Embodiments 1-3, the scenario generation unit 13 determines the consistency of the response results of the commands as follows.

[0054] <D-2. Operation> When the scenario generation unit 13 determines whether the response results of the commands match between the first command group and the first regression test scenario, and between the first command group and the second regression test scenario, it determines that they match not only when they are exactly the same, but also when the difference in the response results is within the range of the threshold value.

[0055] The threshold for determining whether a match occurs is set by one of the following methods (1) to (3).

[0056] (1) A user who is the operator 15 sets a threshold value for determining whether or not a match is made in advance.

[0057] (2) Regression tests are performed multiple times using the automatically generated regression test scenarios, and a threshold value for the regression test scenarios is automatically calculated from the results of the regression tests.

[0058] (3) Use both (1) and (2).

[0059] Method (1) will be described with reference to FIG. 3. FIG. 3 illustrates an example of the thresholds for determining whether each command matches, which are set in the scenario generating unit 13 by the operator 15. FIG. 2 shows the command log of the first command group or the first regression test scenario. The response result of the "Get A" command in the first command group or the first regression test scenario is 12.3, and the threshold for determining whether the command matches is ±3. Therefore, when the response result of the "Get A" command in the second regression test scenario is 12.3±3, that is, between 9.3 and 15.3, the scenario generating unit 13 determines that the response result matches the response result of the "Get A" command in the first regression test scenario.

[0060] For commands for which no threshold for determining whether a match is set, such as the "Get C" command in Figure 3 or the "Set" command in Figure 2, the scenario generation unit 13 determines that the two match if the response result in the second regression test scenario completely matches the response result in the first regression test scenario.

[0061] The method of (2) will be described taking Fig. 4 as an example. In this method, the first regression test scenario is executed as many times as previously set by the operator 15, and threshold candidates used for the coincidence determination are calculated from the response results of the first regression test scenario. The test scenario automatic generation device 101 calculates the minimum value, the maximum value, the average value, the median value, and the mode value from the response results of the first regression test scenario as threshold candidates. Before proceeding to the flow of obtaining the improved regression test scenario, the operator 15 sets in advance which threshold candidate to select as the threshold. Note that the following constraints A - D are provided for the selection of the threshold.

[0062] A. Select at least one or more threshold candidates as the threshold.

[0063] B. When multiple threshold candidates are selected, the range with the minimum value among the selected multiple threshold candidates as the lower limit and the maximum value as the upper limit is defined as the "threshold". For example, in the results of Fig. 4, when the minimum value and the average value are selected, the threshold is 10 or more and 12.25 or less.

[0064] C. When there are multiple mode values, all mode values are used as the threshold.

[0065] D. Set the number of significant digits for each threshold candidate. For example, when the average value is set to "up to the first decimal place", 12.2 becomes the threshold.

[0066] In the method of (3), both of the above - mentioned methods of (1) and (2) are applied. For example, in the results of Figs. 3 and 4, when the minimum value is selected as the threshold, 10 ± 3, that is, 7 or more and 13 or less becomes the threshold.

[0067] <D - 3. Effect> In the test scenario automatic generation device 104 according to Embodiment 4, when the difference between the response result of each second command in the second regression test scenario and the response result of the corresponding second command in the first regression test scenario is less than the threshold value, it is determined that they match. Thus, according to the test scenario automatic generation device 104, it is possible to automatically generate a second regression test scenario even for a test target 16 including a function that requires a range in the response result, such as a sensor that returns an analog value.

[0068] <E. Embodiment 5> <E-1. Configuration> The configuration of the test scenario automatic generation device 105 according to Embodiment 5 is as shown in FIG. 1 and is the same as that of the test scenario automatic generation device 101 according to Embodiment 1. The test scenario automatic generation device 105 according to Embodiment 5 has a function of verifying in advance the standby time of a preset command or combination of commands in the test scenario automatic generation devices 101-104 according to any of Embodiments 1-4.

[0069] <E-2. Operation> FIG. 5 is a diagram illustrating a preset command or combination of commands and the standby time in the test scenario automatic generation device 105. The scenario generation unit 13 verifies the standby time according to the following procedure for a combination assumed to cause a change in the response result among the commands with a preset standby time. Combinations assumed to cause a change in the response result are, for example, No. 1 or No. 2 in FIG. 5. It is assumed that the verification of the standby time can be started by inputting a dedicated command from the operator.

[0070] (1) The scenario generation unit 13 resets the test target 16.

[0071] (2) The scenario generation unit 13 selects one combination of commands with a set waiting time that has not yet been verified. If all combinations of target commands have been verified, the verification of the waiting time is terminated.

[0072] (3) In the selected combination of commands, the scenario generation unit 13 outputs only the commands for obtaining a response from the test target 16, such as No. 1 Get A in FIG. 5, to the test target 16 and records the response result.

[0073] (4) The scenario generation unit 13 outputs commands to the test target 16 according to the selected combination of commands and the waiting time, and records the response result.

[0074] (5) Compare the response results of (3) and (4), and execute either (a) or (b) below according to the comparison result.

[0075] (a) When the response results match, the test scenario automatic generation device 105 notifies the operator that there is a defect in the waiting time between the registered commands.

[0076] (b) When the response results do not match, the test scenario automatic generation device 105 notifies the operator that there is no problem with the waiting time between the registered commands.

[0077] <E-3. Effect> In the test scenario automatic generation device 105 according to the fifth embodiment, the scenario generation unit 13 verifies a predetermined second command or the second and subsequent second commands in a plurality of consecutive predetermined second commands, and notifies the user of the verification result. Therefore, according to the test scenario automatic generation device 105 according to the fifth embodiment, it is possible to detect before the second regression test that there is a defect in the waiting time in the combination of pre-registered commands for some reason, and it is possible to shorten the time required for automatically generating the second regression test scenario.

[0078] <F. Hardware Configuration> In the test scenario automatic generation devices 101 - 104 described above, the command execution unit 11, the recording unit 12, the scenario generation unit 13, and the playback unit 14 are realized by the processing circuit 81 shown in FIG. 6. That is, the processing circuit 81 includes the command execution unit 11, the recording unit 12, the scenario generation unit 13, and the playback unit 14 (hereinafter referred to as "the command execution unit 11 etc."). For the processing circuit 81, dedicated hardware may be applied, or a processor that executes a program stored in the memory may be applied. The processor is, for example, a central processing unit, a processing device, an arithmetic device, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), or the like.

[0079] When the processing circuit 81 is dedicated hardware, the processing circuit 81 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof. The functions of each part such as the command execution unit 11 etc. may be realized by a plurality of processing circuits 81, or the functions of each part may be realized together by one processing circuit.

[0080] When the processing circuit 81 is a processor, the functions of the command execution unit 11 and the like are realized by a combination of software, etc. (software, firmware, or software and firmware). The software, etc. are written as a program and stored in a memory. As shown in FIG. 7, the processor 82 applied to the processing circuit 81 realizes the functions of each unit by reading and executing a program stored in a memory 83. In other words, this program can be said to cause a computer to execute the procedure or method of the command execution unit 11 and the like. Here, the memory 83 may be, for example, a non-volatile or volatile semiconductor memory such as a random access memory (RAM), a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM), or an electrically erasable programmable read only memory (EEPROM), a hard disk drive (HDD), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a digital versatile disk (DVD) and its drive device, or any storage medium to be used in the future.

[0081] Although command output and command response are shown as the communication between the command execution unit 11 and the test subject 16, the communication format between the two is not limited to a specific packet format, signal transmission method, or standard. For example, in addition to digital data communication such as UART or Ethernet (registered trademark), interrupt signals using high / low digital input / output, or input / output of analog values ​​assuming a sensor element may be used.

[0082] The above describes a configuration in which each function of the command execution unit 11 is realized either by hardware or software, etc. However, the present invention is not limited to this, and a configuration in which part of the command execution unit 11, etc. is realized by dedicated hardware and another part is realized by software, etc. For example, the function of the command execution unit 11 can be realized by a processing circuit as dedicated hardware, and the other functions can be realized by the processing circuit 81 as the processor 82 reading and executing a program stored in the memory 83.

[0083] As described above, the processing circuitry can realize the above-mentioned functions by hardware, software, or a combination of these.

[0084] It should be noted that the embodiments can be freely combined, and each embodiment can be modified or omitted as appropriate. [Explanation of symbols]

[0085] 11 command execution unit, 12 recording unit, 13 scenario generation unit, 14 playback unit, 15 operator, 16 test subject, 81 processing circuit, 82 processor, 83 memory, 101-104 test scenario automatic generation device.

Claims

1. An automatic test scenario generation device for generating a regression test scenario for a test target, comprising: a scenario generation unit for generating the regression test scenario; a command execution unit that outputs a first command group consisting of a plurality of first commands sequentially input from outside the test scenario automatic generation device and the regression test scenario to the test subject and acquires a response result from the test subject; a recording unit configured to record a command log including an execution time and a response result of the test subject for each command included in the first command group and the regression test scenario, The regression test scenarios include a first regression test scenario and a second regression test scenario; The scenario generation unit includes: generating the first regression test scenario based on the command log of the first command group, in which a plurality of second commands identical to the plurality of first commands are arranged in the same order and with the same waiting time as the order and waiting time of the plurality of first commands in the first command group; reducing a waiting time of at least one of the second commands in the second regression test scenario compared to the first regression test scenario, based on the command logs of the first regression test scenario and the second regression test scenario, to an extent that the response result coincides with the first regression test scenario; Automatic test scenario generator.

2. The scenario generation unit includes: An initial value of a waiting time of each of the second commands in the second regression test scenario is set to 0; changing a waiting time of the second command of the second regression test scenario in which the response result does not match with the corresponding first command from the initial value to a value equal to the waiting time of the corresponding first command; The test scenario automatic generation device according to claim 1.

3. the scenario generation unit, when a plurality of identical first commands are consecutive in the first command group, deletes from the second regression test scenario the second commands other than a last one of the consecutive identical second commands corresponding to the consecutive plurality of identical first commands; 3. The test scenario automatic generation device according to claim 1.

4. The recording unit records the command log only for predetermined commands.

3. The test scenario automatic generation device according to claim 1.

5. The scenario generation unit includes: An initial value of a waiting time of each of the second commands in the second regression test scenario is set to 0; adding a unit wait time shorter than the wait time of the corresponding first command from the initial value at a time until the response result coincides with the corresponding first command; The test scenario automatic generation device according to claim 1.

6. The scenario generation unit includes: setting an initial value of a waiting time of each of the second commands of the second regression test scenario to a value equal to a waiting time of each of the corresponding first commands; subtracting from the initial value the waiting time of each of the second commands by a unit waiting time shorter than the waiting time of each of the corresponding first commands until the response results do not match between the corresponding first commands; The test scenario automatic generation device according to claim 1.

7. The unit wait time is calculated by multiplying a wait time coefficient defined by a user by a wait time of the corresponding first command.

7. The test scenario automatic generation device according to claim 5.

8. The unit waiting time is determined by a user.

7. The test scenario automatic generation device according to claim 5.

9. the scenario generation unit sets a predetermined waiting time for a predetermined second command or for a second or subsequent second command among a predetermined number of consecutive second commands in the second regression test scenario; 3. The test scenario automatic generation device according to claim 1.

10. the scenario generation unit determines that the response results of each of the second commands in the second regression test scenario match each other when a difference between the response results of each of the second commands in the second regression test scenario and the corresponding response results of each of the second commands in the first regression test scenario is less than a threshold value.

3. The test scenario automatic generation device according to claim 1.

11. The threshold value is predetermined by a user. The test scenario automatic generation device according to claim 10.

12. The threshold value is calculated from the response results of the first regression test scenario performed multiple times. The test scenario automatic generation device according to claim 10.

13. the scenario generation unit verifies a predetermined waiting time for the predetermined second command or for a second or subsequent second command among a predetermined number of consecutive second commands, and notifies a user of a result of the verification.

3. The test scenario automatic generation device according to claim 1.

Images