Test case classification device and test case classification program

Abstract

This invention provides a test case classification device and a test case classification program that can classify test cases in a way that allows testing to be completed within the test period. [Solution] The test case classification device 3 includes a memory 35 and a processor 4. The processor includes a test case classification unit 49 that classifies a single test case by comparing a manual test execution time, which indicates the time required to manually execute a test based on one test case TC that describes the test contents of the system software, with an automatic test execution time, which indicates the time required to automatically execute the test, to set the execution mode of the single test case, and a result output unit 51 that outputs the classification result from the test case classification unit.

Description

Technical Field

[0001] The present invention relates to a test case classification device and a test case classification program.

Background Art

[0002] Conventionally, a software development system capable of determining whether to adopt automatic testing or manual testing when a specification is changed is known (see, for example, Patent Document 1). The software development system described in Patent Document 1 includes a test determination system, a test system, a version management system, a release management system, and an ICT system. When the specification of software is changed, the test determination system predicts the man-hours of automatic testing and the man-hours of manual testing, and presents the one with fewer man-hours. More specifically, for each test case with a usage change, the test determination system acquires the number of test executions and the number of steps, calculates the man-hours of automatic testing and the man-hours of manual testing, and determines the test form with fewer man-hours.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The software development system described in Patent Document 1 distinguishes between test cases that should be performed manually and those that should be performed automatically based on the amount of effort required. Therefore, even if the effort required for a single test case is small, if the time required to complete each process is relatively long, there is a problem in that the time required to perform a test case cannot necessarily be determined by the amount of effort. In addition, manual testing and automated testing have their advantages and disadvantages, and there are test cases that can be completed in less time when performed manually compared to when performed automatically. For this reason, the configuration described in Patent Document 1 may not necessarily shorten the time required to perform tests based on all test cases that should be performed, and there is a possibility that the tests may not be completed within the test period. Therefore, there has been a demand for a configuration that allows test cases to be categorized in a way that enables them to be completed within the test period. [Means for solving the problem]

[0005] A test case classification device according to a first aspect of the present invention includes a memory and a processor, the processor having a test case classification unit that classifies a single test case by comparing a manual test execution time, which indicates the time required to manually execute a test based on one test case from among test cases describing the test contents of system software, with an automatic test execution time, which indicates the time required to automatically execute the test, to set the execution mode of the single test case, and a result output unit that outputs the classification result by the test case classification unit.

[0006] A test case classification program according to a second aspect of the present invention causes a computer having memory and a processor to function as a test case classification unit that classifies a single test case by comparing a manual test execution time, which indicates the time required when manually executing a test based on one test case from among test cases describing the test contents of system software, with an automatic test execution time, which indicates the time required when the test is executed automatically, in order to set the execution mode of the single test case, and a result output unit that outputs the classification result by the test case classification unit. [Brief explanation of the drawing]

[0007] [Figure 1] A schematic diagram showing the configuration of a test case classification system in one embodiment. [Figure 2] A diagram showing an example of a set of test cases in one embodiment. [Figure 3] A block diagram showing the configuration of a test case classification device in one embodiment. [Figure 4] A block diagram showing the functional configuration of a processor in one embodiment. [Figure 5] A flowchart illustrating the test case classification process in one embodiment. [Figure 6] A flowchart illustrating the first classification process in one embodiment. [Figure 7] A flowchart illustrating the second classification process in one embodiment. [Modes for carrying out the invention]

[0008] One embodiment of the present invention will be described below with reference to the drawings. [Outline of the test case classification system] Figure 1 is a schematic diagram showing the configuration of the test case classification system 1 according to this embodiment. As shown in Figure 1, the test case classification system 1 according to this embodiment comprises a test case management server 2 for managing system software test cases and a test case classification device 3. The test case classification device 3 sets the execution mode of the test cases acquired from the test case management server 2 and classifies the test cases. Specifically, as will be described in more detail later, the test case classification device 3 classifies the acquired test cases into test cases to be executed by manual testing and test cases to be executed by automated testing. The following details each component of Test Case Classification System 1.

[0009] [Test Case Management Server Configuration] Figure 2 shows an example of a test case group TG. Test case management server 2 holds the test case management database DB1. The test case management database DB1 manages a set of test cases, including test cases for the system software being tested. For example, the test case management database DB1 manages the test case set TG, which is shown as an example in Figure 2.

[0010] [Structure of the test case group] Here, we will explain the test case group TG, which includes multiple test cases TC. The test case group TG shown in Figure 2 is an example of a collection of test cases TC that describe the content of the tests applied to the system software under test. In other words, as shown in Figure 2, the test case group TG is a table in which at least one test case TC is described on a row-by-row basis. A test case group TG includes a test ID field TG1, a procedure field TG2, an expected value field TG3, a manual test execution time field TG4, and an automated test execution time field TG5. That is, one test case TC includes a test ID, procedure, expected value, manual test execution time, and automated test execution time. A test case group TG may further include input fields where test results are entered.

[0011] The Test ID field TG1 is set to the Test Case ID that identifies Test Case TC. The procedure field TG2 is where the test procedure is set. The expected value is set in the Expected Value column TG3. The expected value is the correct test result when the test is performed according to the procedure described in the Procedure column TG2 of the same test case TC. In other words, the expected value is determined by the specifications of the system software under test and is the test result when the test is performed according to that procedure. The manual test execution time field TG4 is where the time required to manually execute the test based on test case TC is set. The Automated Test Execution Time field TG5 is set to indicate the time required to automatically execute tests based on test case TC. Note that for test case TC where automated testing is not possible, the Automated Test Execution Time field TG5 will be set to "Automation Not Possible".

[0012] [Test Case Classification Device Configuration] Figure 3 is a block diagram showing the configuration of the test case classification device 3. The test case classification device 3 shown in Figure 1 can communicate with the test case management server 2 via network N. The test case classification device 3 obtains the test case TC to be executed from the test case management server 2, from among multiple test cases TC that describe the test content of the system software under test, and classifies the obtained test case TC as a test case that will be executed by at least one of manual or automatic means. More specifically, for each of the obtained test cases to be executed, the test case classification device 3 compares the manual test execution time and the automatic test execution time to set the execution mode for each test case, and classifies each test case into a test case to be executed by manual testing and a test case to be executed by automatic testing. As shown in Figure 3, the test case classification device 3 has an input unit 31, an output unit 32, a communication unit 33, and a control unit 34. The input unit 31, output unit 32, communication unit 33, and control unit 34 are connected to each other via a bus line BL so that they can communicate with one another.

[0013] [Configuration of Input Unit, Output Unit, and Communication Unit] The input unit 31 receives an input operation by an operator of the test case classification device 3 and outputs an operation signal corresponding to the operator's input operation to the control unit 34. That is, the input unit 31 receives an operation for inputting a test period, an operation for inputting test conditions, an operation for selecting priorities, and an operation for specifying a range of test targets. The input unit 31 can be configured by an input device such as a keyboard and a pointing device. The pointing device is, for example, a mouse and a touch panel. The output unit 32 outputs the processing result by the control unit 34. As an example of the output unit 32, a display device and a printer can be mentioned. The communication unit 33 communicates with external devices such as the test case management server 2 via the network N under the control of the control unit 34.

[0014] [Configuration of Control Unit] The control unit 34 controls the test case classification device 3. For example, the control unit 34 executes a predetermined process according to an operation signal input from the input unit 31 and outputs the execution result of the process to the output unit 32. The control unit 34 includes a memory 35 and a processor 4. The memory 35 includes a work memory of the processor 4 and a storage device that stores programs and data necessary for the operation of the test case classification device 3. The work memory temporarily stores at least a part of the program read from the storage device. Also, the work memory temporarily stores the test case TC acquired from the test case management server 2 via the communication unit 33. The storage device stores an OS (Operating System) and a test case classification program that causes the processor 4 to execute the test case classification process described later. That is, the memory 35 having the storage device is a recording medium on which the test case classification program is recorded.

[0015] [Configuration of Processor] Figure 4 is a block diagram showing the functional configuration of processor 4. Processor 4 controls the test case classification device 3 by executing the OS stored in the memory 35. Furthermore, processor 4 executes the test case classification program stored in the memory 35 to perform the test case classification process described later. Note that only one processor 4 is required in the test case classification device 3, and the number of processors 4 in the test case classification device 3 can be changed as appropriate.

[0016] As shown in Figure 4, the processor 4 includes a test condition acquisition unit 41, a test range acquisition unit 42, a test case acquisition unit 43, a priority acquisition unit 44, a priority determination unit 45, an executable time calculation unit 46, a total time calculation unit 47, a time determination unit 48, a test case classification unit 49, a required number calculation unit 50, and a result output unit 51, all of which function by executing a test case classification program.

[0017] The test condition acquisition unit 41 corresponds to the test period acquisition unit and acquires test conditions. The test conditions include, for example, a test period indicated by a start date and time and an end date and time, the number of operators who can perform manual tests, the time period during which operators can perform manual tests, the number of test execution devices that can perform automated tests, and the time period during which test execution devices can perform automated tests. Such test conditions are input to the input unit 31, for example, by the operator of the test case classification device 3. The test scope acquisition unit 42 acquires the scope of testing to be performed on the system software. For example, if the test target is the entire system software, the test scope will be "all test cases". Alternatively, if the test target is a portion of the source code in the system software, the test scope will be "test cases related to that portion of the source code". Such test scopes are input to the input unit 31, for example, by the operator of the test case classification device 3. The test case acquisition unit 43 acquires at least one test case TC from the test case management server 2 that corresponds to the scope of execution acquired by the test scope acquisition unit 42. The test case TC acquired by the test case acquisition unit 43 is the test case to be executed.

[0018] The priority acquisition unit 44 acquires priorities for testing the system software. The priorities are input or selected to the input unit 31 by, for example, the operator of the test case classification device 3. In this embodiment, the priorities are "cost" and "quality". However, the priorities are not limited to these and may include other items. The priority determination unit 45 determines the content of the priority item acquired by the priority acquisition unit 44. In this embodiment, the priority determination unit 45 determines whether or not the acquired priority item is "cost". As will be explained in more detail later, if "cost" is selected, the test case classification device 3, when classifying test cases TC into either manual or automated testing targets, prioritizes classifying test cases TC into automated testing targets, thereby lowering the overall testing cost for all test cases. On the other hand, if "quality" is selected, the test case classification device 3, when classifying test cases TC into either manual or automated testing targets, prioritizes classifying test cases TC as manual testing targets to improve test quality.

[0019] The execution time calculation unit 46 calculates the total execution time of automated tests that can be performed by the end date and time of the test period, and the total execution time of manual tests that can be performed by the end date and time of the test period. Specifically, the execution time calculation unit 46 calculates the execution time of the automated test by multiplying the number of hours defined by the time period during which the automated test can be executed between the start date and end date and time of the test period, based on the acquired test conditions, by the number of test execution devices that will perform the automated test. Furthermore, the execution time calculation unit 46 calculates the manual test execution time by multiplying the number of hours defined by the time period during which manual testing can be performed between the start date and end date and time of the test period, based on the acquired test conditions, by the number of people performing the manual test.

[0020] The total time calculation unit 47 calculates the total automated test time by summing the automated test execution times included in the acquired test cases TC to be executed. The total time calculation unit 47 also calculates the total manual test time by summing the manual test execution times included in the acquired test cases TC to be executed. In other words, the total time calculation unit 47 calculates the time required to execute all of the acquired test cases TC that are classified as targets for automated testing using automated testing as the total automated test time. The total time calculation unit 47 also calculates the time required to execute all of the acquired test cases TC that are classified as targets for manual testing using manual testing as the total manual test time.

[0021] The time determination unit 48 compares the time available for automatic test execution with the total time of automatic test execution to determine whether the time available for automatic test execution is equal to or greater than the total time of automatic test execution. The time determination unit 48 also compares the time available for manual test execution with the total time of manual test execution to determine whether the time available for manual test execution is equal to or greater than the total time of manual test execution.

[0022] The test case classification unit 49 sets the execution mode for each of the acquired test cases TC to be executed and classifies each test case TC into either a test case to be executed by manual testing or a test case to be executed by automated testing. Specifically, for each acquired test case TC, the test case classification unit 49 sets either manual testing or automated testing as the execution mode of the test based on the corresponding test case TC and classifies the test case TC into either a target for manual testing or a target for automated testing. At this time, the test case classification unit 49 may include test case TCs classified as targets for manual testing in the list of targets for manual testing, and test case TCs classified as targets for automated testing in the list of targets for automated testing. Alternatively, based on the classification result, the test case classification unit 49 may include either "manual testing" or "automated testing" as the execution mode for each test case TC.

[0023] The required number calculation unit 50 calculates the required number of test execution devices if, under the input test conditions, it is not possible to complete all of the acquired test cases TC to be executed within the test period, and if the tests based on all of the test cases TC to be executed are to be performed by automated testing within the test period. In other words, the required number calculation unit 50 calculates the required number of test execution devices to be completed by automated testing within the test period when the priority item described above is "cost". Furthermore, the required number calculation unit 50 calculates the number of operators needed to perform tests based on all of the acquired test cases TC within the test period if the input test conditions make it impossible to complete the tests based on all of the target test cases TC within the test period by manual testing. In other words, the required number calculation unit 50 calculates the number of operators needed to complete the tests based on all of the target test cases TC by manual testing within the test period when the priority item mentioned above is "quality".

[0024] The result output unit 51 outputs the classification results of the test cases TC to be executed. Specifically, the result output unit 51 notifies the operator of the test case classification device 3 of the classification results. If, under the input test conditions, it is not possible to complete all acquired test cases TC within the test period, the result output unit 51 functions as a required number output unit, outputting the required number calculated by the required number calculation unit 50. The notification by the result output unit 51 may also be made by email.

[0025] [Test Case Classification Process] Figure 5 is a flowchart showing the test case classification process performed by processor 4. The processor 4 of the test case classification device 3 reads the test case classification program stored in memory 35 and executes the test case classification process shown in Figure 5. In the test case classification process, first, the test condition acquisition unit 41 acquires the input test conditions (step S1), the test range acquisition unit 42 acquires the input test range (step S2), and the priority acquisition unit 44 acquires the input or selected priority (step S3). Note that the order of steps S1 to S3 can be changed as appropriate. Next, the test case acquisition unit 43 acquires test cases TC corresponding to the acquired test range from the test case management server 2 as test cases to be executed (step S4).

[0026] Then, the priority determination unit 45 determines whether the acquired priority is "cost" or not (step S5). If the decision process in step S5 determines that the priority is "cost" (step S5: YES), the processor 4 executes the first classification process SA. If the priority is determined not to be "cost" in the determination process of step S5 (step S5: NO), the processor 4 executes the second classification process SB. That is, if the priority determination unit 45 determines that the priority is "quality", the processor 4 executes the second classification process SB. The first classification process SA and the second classification process SB will be described in detail later. When the first classification process SA and the second classification process SB are completed, the result output unit 51 outputs the classification results, etc. (step S6). The test case classification process is now complete.

[0027] [First Classification Process] Figure 6 is a flowchart of the first classification process, SA. The first classification process, SA, is a process included in the test case classification process. The first classification process, SA, is executed when the priority is "cost," and it is a process that classifies the test cases to be executed with priority over those to be executed by automated tests, thereby increasing the proportion of automated tests in tests based on the test cases to be executed. In the first classification process SA, as shown in Figure 6, the test case classification unit 49 first classifies all test cases TC included in the test case to be executed as targets for automated testing (step SA01). In other words, the test case classification unit 49 determines that the execution mode of all test cases TC included in the test case to be executed is an automated test.

[0028] Next, the execution time calculation unit 46 calculates the execution time of the automated test based on the acquired test conditions (step SA02). For example, if the start date and time of the test period included in the test conditions is 10am on November 2nd, the end date and time of the test period is 10pm on November 4th, the number of test execution devices is 3, and the time period during which the automated test can be executed by the test execution devices is from 6am to 11pm, then the time during which one test execution device can execute the automated test is 46 hours. Therefore, the total execution time of the automated test that can be executed by 3 test execution devices is 46 hours × 3 devices = 138 hours.

[0029] Next, the total time calculation unit 47 calculates the total automated test time by summing up the automated test execution times of each test case TC included in the test case to be executed (step SA03). Next, the time determination unit 48 compares the time available for automatic test execution with the total time of automatic test execution and determines whether the time available for automatic test execution is equal to or greater than the total time of automatic test execution (step SA04).

[0030] If the determination process in step SA04 determines that the time available for automated test execution is not equal to or greater than the total time for automated tests (step SA04: NO), the test case classification unit 49 removes a predetermined number of test cases TC from the test cases included in the test cases to be executed that are subject to automated testing, and classifies them as subjects for manual testing (step SA05). The predetermined number can be changed as appropriate. In this process, the test case classification unit 49 compares the manual test execution time and the automated test execution time for each test case TC included in the test case to be executed, and classifies a predetermined number of test cases TC whose manual test execution time is less than the automated test execution time as test cases to be executed manually. Test cases TC classified as test cases to be executed manually are those in which the manual test execution time is less than the automated test execution time. Furthermore, the test case classification unit 49 may extract test cases TC whose difference between the manual test execution time and the automated test execution time is greater than a predetermined value, and classify a predetermined number of the extracted test cases TC as targets for manual test execution. The test case classification unit 49 may also classify a predetermined number of test cases TC, randomly selected from those classified as targets for automated testing, as targets for manual testing.

[0031] After step SA05, the total time calculation unit 47 recalculates the total automated test time (step SA06). At this time, the total time calculation unit 47 calculates the total automated test time by summing the automated test execution times of the remaining test cases TC, excluding the test cases TC that were designated for manual testing. In other words, the total time calculation unit 47 calculates the total automated test time by summing the automated test execution times of the test cases TC that were classified as targets for automated testing. Since the number of test cases TC classified as targets for automated testing decreases after step SA05, the total automated test time calculated in step SA06 will be smaller than the total automated test time calculated in step SA03. Once step SA06 is complete, processor 4 returns to step SA04.

[0032] In the determination process of step SA04, if the time available for automated test execution is equal to or greater than the total automated test time (step SA04: YES), the total time calculation unit 47 calculates the total manual test time by summing the manual test execution times of the remaining test cases TC, excluding the test cases TC classified as targets for automated test execution (step SA07). In other words, in step SA07, the total time calculation unit 47 calculates the total manual test time by summing the manual test execution times of the test cases TC classified as targets for manual test execution. If you proceed to step SA07 without going through steps SA05 and SA06, all test cases included in the target test cases are classified as targets for automated testing. In this case, the total manual test time calculated in step SA07 will be 0 hours.

[0033] Furthermore, the execution time calculation unit 46 calculates the manual test execution time based on the acquired test conditions (step SA08). For example, if the test period and end time included in the test conditions are as described above, the number of implementers included in the test conditions is 2, and the time slots in which implementers can perform manual tests are between 9:00 and 12:00 and between 13:00 and 17:00, then the time in the test period in which one implementer can perform manual tests is 20 hours. Therefore, the manual test execution time for manual tests that can be performed by two implementers is 20 hours × 2 people = 40 hours. Steps SA07 and SA08 may be executed simultaneously, or one of them may be executed before the other.

[0034] After steps SA07 and SA08, the time determination unit 48 compares the total manual test time calculated in step SA07 with the manual test execution time calculated in step SA08, and determines whether the manual test execution time is equal to or greater than the total manual test time (step SA09). In the determination process of step SA09, if it is determined that the time available for manual test execution is equal to or greater than the total time for manual test execution (step SA09: YES), the processor 4 determines that the tests based on the test cases to be executed can be completed within the test period by the number of test execution devices and the number of implementers included in the test conditions, and terminates the first classification process SA. In this case, in step S6 of the test case classification process shown in Figure 5, the result output unit 51 outputs that the classification of test cases TC has been completed and the classification result by the test case classification unit 49. As a result, the processor 4 terminates the test case classification process.

[0035] In the determination process of step SA09, if it is determined that the time available for manual test execution is not equal to or greater than the total time for manual test execution (step SA09: NO), the processor 4 determines that the number of test execution devices and the number of executors included in the test conditions cannot complete the tests based on the target test cases within the test period. In other words, if the time available for manual test execution is less than the total time for manual test execution, the processor 4 determines that the tests based on the target test cases cannot be completed within the test period. In this case, the required number calculation unit 50 calculates the number of test execution devices required to perform tests based on all test cases TC included in the test case to be executed using automated testing (step SA10). Specifically, the required number calculation unit 50 divides the total manual test time calculated in step SA03, that is, the total automated test time which is the sum of the automated test execution times for all test cases TC included in the test case to be executed, by the time during the test period that one test execution device can perform automated testing, and rounds up the remainder to obtain the quotient as the required number of test execution devices. After this, processor 4 terminates the first classification process SA. In this case, the result output unit 51 outputs that the classification of test case TC could not be completed in step S6 of the test case classification process shown in Figure 5, and the required number of test execution devices calculated in step SA10. As a result, processor 4 terminates the test case classification process.

[0036] [Second Classification Processing] Figure 7 is a flowchart of the second classification process SB. Next, we will explain the second classification process, SB. The second classification process (SB) is a process included in the test case classification process. The second classification process (SB) is executed when the priority is "quality," and it is a process that classifies the test cases to be executed with priority over those to be executed manually, thereby increasing the proportion of manual testing in tests based on the test cases to be executed. In the second classification process SB, as shown in Figure 7, first, the test case classification unit 49 classifies all test cases TC included in the test case to be executed as targets for manual testing (step SB01). In other words, the test case classification unit 49 determines the execution mode of all test cases TC included in the test case to be executed as manual testing.

[0037] Next, the execution time calculation unit 46 calculates the manual test execution time based on the acquired test conditions (step SB02). In step SB02, the manual test execution time is calculated by the same process as in step SA08 described above.

[0038] Next, the total time calculation unit 47 calculates the total manual test time by summing the manual test execution times of each test case TC included in the test case to be executed (step SB03). At step SB03, all test case TC included in the test case to be executed are manual tests, so in step SB03, the total time calculation unit 47 calculates the total manual test time by summing the manual test execution times of all the test case TC. Next, the time determination unit 48 compares the time available for manual testing with the total time of manual testing and determines whether the time available for manual testing is equal to or greater than the total time of manual testing (step SB04).

[0039] If the determination process in step SB04 determines that the time available for manual testing is not equal to or greater than the total time for manual testing (step SB04: NO), the test case classification unit 49 removes a predetermined number of test cases TC from the test cases included in the test cases to be executed that are subject to manual testing, and classifies them as subjects for automated testing (step SB05). The predetermined number can be changed as appropriate. In this process, the test case classification unit 49 compares the manual test execution time and the automated test execution time for each test case TC included in the test case to be executed, and classifies a predetermined number of test cases TC whose automated test execution time is less than or equal to the manual test execution time as targets for automated testing. In other words, the test case classification unit 49 classifies a predetermined number of test cases TC whose manual test execution time exceeds the automated test execution time as targets for automated testing. The test cases TC classified as targets for automated testing are test cases in which the automated test execution time is less than or equal to the manual test execution time. Furthermore, the test case classification unit 49 may extract test cases TC whose difference between manual test execution time and automated test execution time is greater than a predetermined value, and classify a predetermined number of the extracted test cases TC as targets for automated test execution. The test case classification unit 49 may also classify a predetermined number of test cases TC, randomly selected from those classified as targets for manual testing, as targets for automated testing.

[0040] After step SB05, the total time calculation unit 47 recalculates the total manual test time (step SB06). At this time, the total time calculation unit 47 calculates the total manual test time by summing the manual test execution times of the remaining test cases TC, excluding the test cases TC that are subject to automated testing. In other words, the total time calculation unit 47 calculates the total manual test time by summing the manual test execution times of the test cases TC that are classified as subject to manual testing. Since the number of test cases TC classified as subject to manual testing decreases after step SB05, the total manual test time calculated in step SB06 will be smaller than the total manual test time calculated in step SB03. Once step SB06 is complete, processor 4 returns to step SB04.

[0041] In the determination process of step SB04, if the time available for manual test execution is equal to or greater than the total time for manual test execution (step SB04: YES), the total time calculation unit 47 calculates the total time for automatic test execution by summing the automatic test execution times of the remaining test cases TC, excluding the test cases TC that are classified as targets for manual test execution (step SB07). In other words, in step SB07, the same process as in step SA06 is performed to calculate the total time for automatic test execution. If you proceed to step SB07 without going through steps SB05 and SB06, all test cases included in the target test cases will be classified as targets for manual testing. In this case, the total automated test time calculated in step SB07 will be 0 hours.

[0042] Furthermore, the execution time calculation unit 46 calculates the execution time of the automated test based on the acquired test conditions (step SB08). Specifically, in step SB08, the execution time calculation unit 46 performs the same processing as in step SA06 above to calculate the execution time of the automated test. Steps SB07 and SB08 may be executed simultaneously, or one of them may be executed before the other.

[0043] After steps SB07 and SB08, the time determination unit 48 compares the total automated test time calculated in step SB07 with the automated test execution time calculated in step SB08, and determines whether the automated test execution time is equal to or greater than the total automated test time (step SB09). In the determination process of step SB09, if it is determined that the time available for automated test execution is equal to or greater than the total time for automated tests (step SB09: YES), the processor 4 determines that the tests based on the test cases to be executed can be completed within the test period by the number of implementers and test execution devices included in the test conditions, and terminates the second classification process SB. In this case, in step S6 of the test case classification process shown in Figure 5, the result output unit 51 outputs that the classification of test cases TC has been completed and the classification result by the test case classification unit 49. As a result, the processor 4 terminates the test case classification process.

[0044] If the determination process in step SB09 determines that the available time for automated testing is not equal to or greater than the total automated testing time (step SB09: NO), then processor 4 determines that the number of executors and test execution devices included in the test conditions cannot complete the tests based on the target test cases within the test period. In other words, if the available time for automated testing is less than the total automated testing time, processor 4 determines that the tests based on the target test cases cannot be completed within the test period. In this case, the required number calculation unit 50 calculates the number of operators required to perform manual tests based on all test cases TC included in the test case to be executed (step SB10). Specifically, the required number calculation unit 50 divides the total manual test time calculated in step SB03, that is, the total manual test time obtained by summing the manual test execution times of all test cases TC included in the test case to be executed, by the amount of time one operator can perform manual tests during the test period, and rounds up the remainder to obtain the quotient as the required number of operators. After this, processor 4 terminates the second classification process SB. In this case, the result output unit 51 outputs that the classification of test case TC could not be completed in step S6 of the test case classification process shown in Figure 5, and the required number of implementers calculated in step SB10. As a result, processor 4 terminates the test case classification process.

[0045] [Effects of the Embodiment] The test case classification system 1 according to this embodiment, as described above, has the following effects. The test case classification device 3 includes a memory 35 and a processor 4. The processor that executes the test case classification program includes a test case classification unit 49 and a result output unit 51. The test case classification unit 49 compares the manual test execution time and the automated test execution time of one test case from among the test cases that describe the test content of the system software, sets the execution mode of that test case, and classifies that test case. In other words, the test case classification unit 49 sets either manual testing or automated testing as the execution mode of the test case, thereby classifying the test case as either a target for manual testing or a target for automated testing. The manual test execution time indicates the time required when manually executing the tests based on one test case, and the automated test execution time indicates the time required when automatically executing the tests based on one test case. The result output unit 51 outputs the classification results from the test case classification unit 49.

[0046] With this configuration, it is possible to determine whether a test case should be executed manually or automatically based on the manual and automated test execution times of that test case. The test case classification unit 49 can then classify, for example, test cases with a manual execution time less than the automated execution time as targets for manual testing, and test cases with a manual execution time equal to or greater than the automated execution time as targets for automated testing. This reduces the time required to execute tests based on multiple test cases. Therefore, test cases can be classified in a way that allows for test completion within the test period.

[0047] In the test case classification device 3, the test case classification unit 49, for example in step SA05, classifies test cases where the manual test execution time is less than the automated test execution time as targets for manual testing. Furthermore, the test case classification unit 49, for example in step SB05, classifies test cases where the manual test execution time exceeds the automated test execution time as targets for automated testing. With this configuration, for a given test case, it is possible to select the execution method that takes less time from among manual and automated testing. Therefore, the time required can be reduced.

[0048] In the test case classification device 3, the processor 4 includes a test condition acquisition unit 41 for acquiring the test period, a required number calculation unit 50, and a result output unit 51. The test condition acquisition unit 41 includes the functions of the test period acquisition unit. The required number calculation unit 50 calculates, for example, in step SA10, the number of test execution devices required to complete the test cases classified as targets for automated testing by the test case classification unit 49 within the test period. In step SA10, the required number calculation unit 50 calculates the number of test execution devices assuming that all test cases TC included in the target test cases are classified as targets for automated testing. The result output unit 51 corresponds to the required number output unit, and in step S6 after the first classification process SA, it outputs the number of test execution devices calculated by the required number calculation unit 50. With this configuration, for example, the operator or test manager of the test case classification device 3 can easily determine the number of test execution devices required. Therefore, by preparing the number of test execution devices indicated, automated tests based on the test cases classified as targets for automated testing can be completed within the test period.

[0049] In the test case classification device 3, the processor 4 includes a test condition acquisition unit 41 for acquiring the test period, a required number calculation unit 50, and a result output unit 51. The test condition acquisition unit 41 includes the functions of the test period acquisition unit. The required number calculation unit 50 calculates, for example, in step SB10, the number of testers required to complete the test cases classified as targets for manual testing by the test case classification unit 49 within the test period. In step SB10, the required number calculation unit 50 calculates the number of testers assuming that all test cases TC included in the target test cases are classified as targets for manual testing. The result output unit 51 corresponds to the required number output unit, and in step S6 after the second classification process SB, it outputs the number of test implementers calculated by the required number calculation unit 50. With this configuration, for example, the operator or test manager of the test case classification device 3 can easily determine the number of testers required. Therefore, by preparing the number of testers indicated, manual testing based on the test cases classified as targets for manual testing can be completed within the test period.

[0050] In the test case classification device 3, test case TC includes manual test execution time and automated test execution time. With this configuration, manual test execution time and automated test execution time can be obtained from the acquired test cases. Therefore, compared to obtaining manual and automated test execution time separately from test cases, the process of obtaining each execution time can be performed more quickly, and the acquisition of execution times for other test cases can be suppressed. Consequently, the processing time for classifying test cases TC can be reduced.

[0051] [Variations of the Embodiment] The present invention is not limited to the embodiments described above, and any modifications and improvements that can achieve the objectives of the present invention are included within the scope of the present invention. In the above embodiment, the test case classification unit 49 classifies the execution mode of each test case TC included in the target test case into either manual testing or automated testing based on the test conditions input to the input unit 31 and acquired by the test condition acquisition unit 41. However, the processor 4 having the test case classification unit 49 may classify the execution mode of test case TC based on predetermined conditions. For example, the test case classification unit 49 of the processor 4 may compare the manual test execution time and the automated test execution time included in the test case TC to determine whether the execution mode of the test case TC should be manual or automated. In this case, for example, the test case classification unit 49 may set the execution mode of test case TCs included in the target test case to automated tests if the automated test execution time is less than or equal to the manual test execution time, and set the execution mode of test case TCs where the automated test execution time is greater than the manual test execution time to manual tests. In this case, the execution mode of the tests based on the test case TCs included in the target test case can be set so that the tests can be executed in the shortest possible time.

[0052] In the above embodiment, the test conditions included a test period indicated by a start date and time and an end date and time, the number of operators capable of performing manual tests, the time period during which operators can perform manual tests, the number of test execution devices capable of performing automated tests, and the time period during which test execution devices can perform automated tests. However, the test conditions are not limited to this, and at least one of the test period, the number of operators, the time period during which manual tests can be performed, the number of test execution devices, and the time period during which automated tests can be performed may be omitted, and other items may be included.

[0053] In the above embodiment, each test case TC includes a manual test execution time, which represents the time required when testing manually, and an automated test execution time, which represents the time required when testing automatically. However, the embodiment is not limited to this, and the manual test execution time and automated test execution time do not necessarily have to be included in the test case TC, as long as they are associated with the test case TC. For example, the manual test execution time and automated test execution time may be stored separately from the test case TC in the test case management server 2 or the test case classification device 3.

[0054] In the above embodiment, the required number calculation unit 50 calculates the number of test execution devices required when performing tests based on all test cases included in the target test case using automated testing, and calculates the number of operators required when performing tests based on all test cases included in the target test case using manual testing. However, the processor 4 is not limited to this and does not need to calculate the required number of test execution devices or the required number of operators.

[0055] In the above embodiment, the processor 4 calculates the required number of test execution devices or the required number of implementers using the required number calculation unit 50 if it is not possible to complete the tests based on the target test case within the test period, and outputs and notifies the required number of test execution devices or the required number of implementers using the result output unit 51. However, it is not limited to this, and for example, when test conditions or priorities are acquired, the processor 4 may notify at least one of the required number of test execution devices and the required number of implementers. In this case, the processor 4 may cause the display device of the test case classification device 3 to display a notification including at least one of the numbers. In other words, the manner of notification by the processor 4 can be changed as appropriate.

[0056] In the above embodiment, the processor 4 executes one of the setting processes, either the first classification process SA or the second classification process SB, based on the priority information input to the input unit 31 and acquired by the priority information acquisition unit 44. Specifically, if the priority is "cost", the processor 4 executes the first classification process SA, which could also be called an automated test priority classification process, to set the execution mode of the test case TC to prioritize automated testing. If the priority is "quality", the processor 4 executes the second classification process SB, which could also be called a manual test priority classification process, to set the execution mode of the test case TC to prioritize manual testing. However, the processor 4 is not limited to this and may execute one of the classification processes, either the first classification process SA or the second classification process SB, based on other information. For example, the processor 4 may execute the first classification process SA if the priority is "automated test priority", and execute the second classification process SB if the priority is "manual test priority".

[0057] In the above embodiment, test case TC included manual test execution time and automated test execution time. However, it is not limited to this, and for example, test case TC may include levels indicating the time required for manual testing and the time required for automated testing, respectively.

[0058] In the above embodiment, the required number calculation unit 50 calculates in step SA10 the number of test execution devices capable of automatically completing tests based on all test cases TC included in the test cases to be executed within the test period. Furthermore, in step SB10, the required number calculation unit 50 calculates the number of test implementers capable of manually completing tests based on all test cases TC included in the test cases to be executed within the test period. However, the embodiment is not limited to this, for example, if it is determined in step SA04 that the time available for automatic test execution is equal to or greater than the total time of automatic test execution, the required number calculation unit 50 may calculate the number of test execution devices capable of completing automatic tests based on test cases TC classified as targets for automatic test execution within the test period based on the acquired test conditions, and the result output unit 51 may output the number of test execution devices at the time the number of test execution devices is calculated. Furthermore, if, for example, step SB04 determines that the time available for manual testing is equal to or greater than the total time for manual testing, the required number calculation unit 50 calculates the number of test implementers who can complete the manual tests based on the test cases TC classified as targets for manual testing within the test period, based on the acquired test conditions. The result output unit 51 may then output the number of test implementers at the time the number of test implementers is calculated.

[0059] [Summary of the present invention] A summary of the present invention is provided below. [1] Equipped with memory and a processor, The aforementioned processor, A test case classification unit classifies a single test case by comparing the manual test execution time, which indicates the time required to manually execute a test based on one test case, with the automated test execution time, which indicates the time required to automatically execute the said test, in order to set the execution mode for the single test case. The system includes a result output unit that outputs the classification results from the test case classification unit, A test case classification device characterized by the following features.

[0060] With this configuration, it is possible to determine whether a test case should be executed manually or automatically based on its manual and automated execution times. The test case classification unit can then classify test cases where the manual execution time is less than the automated execution time as targets for manual testing, and test cases where the manual execution time is equal to or greater than the automated execution time as targets for automated testing. This reduces the time required to execute tests based on multiple test cases. Therefore, test cases can be classified in a way that allows for test completion within the test period.

[0061] In the test case classification device described in [2][1], The aforementioned test case classification unit is Test cases in which the manual test execution time is less than the automated test execution time are classified as targets for manual testing. Test cases in which the manual test execution time exceeds the automated test execution time are classified as targets for automated testing. A test case classification device characterized by the following features. With this configuration, for a given test case, it is possible to select the execution method that takes less time from among manual and automated testing. Therefore, the time required can be reduced.

[0062] In the test case classification device described in [3][2], The aforementioned processor, The test period acquisition unit obtains the test period, A required number calculation unit calculates the number of test execution devices necessary to complete the test cases classified as targets for automated test execution by the test case classification unit within the test period, The system includes a required number output unit that outputs the number of test execution devices calculated by the required number calculation unit. A test case classification device characterized by the following features. With this configuration, for example, the operator of the test case classification device or the test manager can easily determine the number of test execution devices required. Therefore, by preparing the number of test execution devices indicated, automated tests based on the test cases classified as targets for automated testing can be completed within the test period.

[0063] In the test case classification device described in [4][2] or [3], The aforementioned processor, The test period acquisition unit obtains the test period, A required number calculation unit calculates the number of testers necessary to complete the test cases classified as targets for manual testing by the test case classification unit within the test period, The system includes a required number output unit that outputs the number of test implementers calculated by the required number calculation unit. A test case classification device characterized by the following features. With this configuration, for example, the operator of the test case classification device or the test manager can easily determine the number of implementers required. Therefore, by preparing the number of implementers indicated by the output, manual testing based on test cases classified as targets for manual testing can be completed within the test period.

[0064] In the test case classification device described in any one of [5], [1] to [4], The test case includes the manual test execution time and the automated test execution time. A test case classification device characterized by the following features. With this configuration, manual test execution time and automated test execution time can be obtained from the acquired test cases. Therefore, compared to obtaining manual test execution time and automated test execution time separately from the test cases, the process of obtaining each execution time can be performed more quickly, and the acquisition of execution times for other test cases can be suppressed. Consequently, the processing time for classifying test cases can be reduced.

[0065] [6] A computer having memory and a processor, A test case classification unit classifies a single test case by comparing the manual test execution time, which indicates the time required to manually execute a test based on one test case, with the automated test execution time, which indicates the time required to automatically execute the said test, in order to set the execution mode for the single test case. A test case classification program characterized by functioning as a result output unit that outputs the classification results from the test case classification unit. By having a computer execute such a program, the same effect as the test case classification device described above can be achieved. [Explanation of symbols]

[0066] 1...Test case classification system, 2...Test case management server, 3...Test case classification device, 31...Input unit, 32...Output unit, 33...Communication unit, 34...Control unit, 35...Memory, 4...Processor, 41...Test condition acquisition unit (test period acquisition unit), 42...Test range acquisition unit, 43...Test case acquisition unit, 44...Priority acquisition unit, 45...Priority determination unit, 46...Executable time calculation unit, 47...Total time calculation unit, 48...Time determination unit, 49...Test case classification unit, 50...Required number calculation unit, 51...Result output unit (required number output unit).

Claims

1. Equipped with memory and a processor, The aforementioned processor, A test case classification unit classifies a single test case by comparing the manual test execution time, which indicates the time required to manually execute a test based on one test case, with the automated test execution time, which indicates the time required to automatically execute the said test, in order to set the execution mode for the single test case. The system includes a result output unit that outputs the classification results from the test case classification unit, A test case classification device characterized by the following features.

2. In the test case classification device according to claim 1, The aforementioned test case classification unit is Test cases in which the manual test execution time is less than the automated test execution time are classified as targets for manual testing. Test cases in which the manual test execution time exceeds the automated test execution time are classified as targets for automated testing. A test case classification device characterized by the following features.

3. In the test case classification device according to claim 2, The aforementioned processor, The test period acquisition unit obtains the test period, A required number calculation unit calculates the number of test execution devices necessary to complete the test cases classified as targets for automated test execution by the test case classification unit within the test period, The system includes a required number output unit that outputs the number of test execution devices calculated by the required number calculation unit. A test case classification device characterized by the following features.

4. In the test case classification device according to claim 2, The aforementioned processor, The test period acquisition unit obtains the test period, A required number calculation unit calculates the number of testers necessary to complete the test cases classified as targets for manual testing by the test case classification unit within the test period, The system includes a required number output unit that outputs the number of test implementers calculated by the required number calculation unit. A test case classification device characterized by the following features.

5. In the test case classification device according to any one of claims 1 to 4, The test case includes the manual test execution time and the automated test execution time. A test case classification device characterized by the following features.

6. A computer having memory and a processor, A test case classification unit classifies a single test case by comparing the manual test execution time, which indicates the time required to manually execute a test based on one test case, with the automated test execution time, which indicates the time required to automatically execute the said test, in order to set the execution mode for the single test case. A test case classification program characterized by functioning as a result output unit that outputs the classification results from the test case classification unit.

Images