Program, method, information processing device, and system

Abstract

To efficiently administer a test.SOLUTION: A program according to the present disclosure, the program causes a computer including a processor and a memory to execute the following steps. The program causes the processor to execute the steps of: storing information relating to a test administered in a test process; selecting a test case to be preferentially tested from tested test cases based on the information relating to the test; extracting a test case similar to the selected test case from untested test cases in the test process; and setting up a priority of testing the extracted test case higher than the other untested test cases.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to a program, a method, an information processing device, and a system. [Background technology]

[0002] To ensure the quality of software, its functions are tested before it is released. Software function testing is generally performed based on test specifications, which describe the test method and the conditions for determining pass / fail for each test case.

[0003] Regarding software testing, it is important to create a test plan efficiently and with high accuracy in order to proceed with a project. In Patent Document 1, the man-hours required for unexecuted test items are predicted based on the actual man-hours of the same or similar tests that were conducted in the past. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2020-98413 Summary of the Invention [Problem to be solved by the invention]

[0005] In software development, the order in which tests are performed is an important factor in ensuring that the testing process in a project proceeds as planned. For example, if there are tests that are likely to cause defects, the later those tests are performed, the greater the risk of degradation. For this reason, performing tests that are likely to cause defects at an early stage in the testing process will allow for more efficient testing in the testing process.

[0006] The present disclosure aims to perform tests efficiently. [Means for solving the problem]

[0007] To solve the above problem, the present disclosure provides a program for causing a computer including a processor and a memory to execute the following steps: storing information about tests performed in a testing process; selecting, based on the information about the tests, a test case to be prioritized for testing from among the test cases that have been tested; extracting, in the testing process, test cases similar to the selected test case from among untested test cases; and setting the priority of testing for the extracted test case higher than that of other untested test cases. [Effects of the Invention]

[0008] According to the present disclosure, tests can be performed efficiently. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram showing the overall configuration of a system according to an embodiment of the present invention; [Figure 2] FIG. 2 is a block diagram showing the configuration of a terminal device included in the system of the present embodiment. [Figure 3] FIG. 2 is a block diagram showing the functional configuration of a server included in the system of the present embodiment. [Figure 4] FIG. 10 is a diagram illustrating a data structure of a test case DB stored in a server. [Figure 5] FIG. 10 is a diagram illustrating the operation of the system when a test is administered by a tester. [Figure 6] FIG. 10 is a schematic diagram illustrating an example of a project list displayed on a display. [Figure 7] FIG. 10 is a schematic diagram illustrating an example of a list of test suites displayed on a display. [Figure 8] FIG. 10 is a schematic diagram illustrating an example of a list of test case data displayed on a display. [Figure 9] FIG. 10 is a schematic diagram illustrating an example of a ticket input form displayed on a display. [Figure 10] FIG. 10 is a schematic diagram showing an example of the contents of a ticket displayed on a display. [Figure 11] FIG. 1 is a diagram illustrating the operation of the system when software is modified by a developer and the modified software is tested by a tester. [Figure 12] 10 is a flowchart illustrating an example of the operation of a server when setting priorities set in test suites. [Figure 13] 10 is a flowchart showing a specific example of the operation of an analysis unit. [Figure 14] FIG. 10 is a schematic diagram illustrating an example of a list of test suites displayed on a display. [Figure 15] FIG. 10 is a schematic diagram illustrating an example of a list of test case data displayed on a display. [Figure 16] FIG. 10 is a schematic diagram illustrating an example of a list of test suites displayed on a display. [Figure 17] FIG. 10 is a schematic diagram illustrating an example of a list of test case data displayed on a display. [Figure 18] 10 is a flowchart illustrating an example of the operation of a server when predicting test trends. [Figure 19] FIG. 10 is a schematic diagram illustrating an example of a graph showing test trends displayed on a display. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described with reference to the drawings. Note that the embodiments described below do not unduly limit the content of the present disclosure described in the claims. Furthermore, not all of the configurations described in the embodiments are necessarily essential components of the present disclosure.

[0011] <Summary> In this embodiment, system 1 analyzes test cases that have already been executed during the testing process of a specific project and extracts test cases that should be prioritized from among untested test cases. System 1 then prompts the tester to prioritize testing of the extracted test cases. In response to requests from testers, developers, project managers, etc., system 1 then presents the requesting source with predicted test trends resulting from changes to the test execution order.

[0012] In this embodiment, test cases are set based on a test design document. The test design document is data in which tests are designed to test the functions of the test object. The test object represents the object of the test, such as hardware or software developed in a specific project. Test objects include, for example, prototypes, intermediate products, and final products.

[0013] <1 Overall structure> Fig. 1 is a diagram showing the overall configuration of a system 1 of this embodiment. As shown in Fig. 1, the system 1 includes a terminal device 10 and a server 20. The terminal device 10 and the server 20 are connected to each other via a network 80 using a wired or wireless communication standard so as to be able to communicate with each other. The network 80 is realized by, for example, the Internet and / or a communication network provided by a telecommunications carrier. In the example shown in the figure, the system 1 includes a plurality of terminal devices 10.

[0014] Although FIG. 1 shows a case where the server 20 is one computer, the server 20 may be realized by combining a plurality of computers.

[0015] The terminal device 10 is a terminal used by a user who uses the services provided by the server 20. For example, the terminal device 10-1 is a terminal used by a tester who conducts tests. The terminal device 10-2 is a terminal used by a developer who develops software. Although there are terminal devices 10-1 and 10-2, the structures of the devices themselves are almost the same, so the following description will be given as the terminal device 10.

[0016] The terminal device 10 is realized by, for example, a desktop personal computer (PC), a laptop PC, a head-mounted display, etc. The terminal device 10 may also be a portable computer such as a smartphone or a tablet terminal.

[0017] 1, the terminal device 10 includes a communication IF (Interface) 12, an input device 13, an output device 14, a memory 15, a storage 16, and a processor 19. The communication IF 12, the input device 13, the output device 14, the memory 15, the storage 16, and the processor 19 are connected to each other so as to be able to communicate with each other, for example, via a bus.

[0018] The communication IF 12 is an interface for transmitting and receiving signals so that the terminal device 10 can communicate with external devices. The input device 13 is an input device for receiving input operations from a user (employee). The input device 13 includes, for example, a touch panel, a touch pad, a pointing device such as a mouse, a keyboard, etc. The output device 14 is an output device for presenting information to a user. The output device 14 includes, for example, a display, a speaker, etc.

[0019] The memory 15 is for temporarily storing programs and data processed by the programs, etc., and is realized by a volatile memory such as a DRAM (Dynamic Random Access Memory). The storage 16 is a storage device for saving data, and is realized by a non-volatile memory such as a flash memory or an HDD (Hard Disc Drive). The processor 19 is hardware for executing an instruction set written in a program, and is made up of an arithmetic unit, registers, peripheral circuits, etc.

[0020] The server 20 is a device that manages the implementation status and results of tests defined based on a test design document. The server 20 is, for example, a computer connected to a network 80.

[0021] 1, the server 20 includes a communication IF 22, an input / output IF 23, a memory 25, a storage 26, and a processor 29. The communication IF 22, the input / output IF 23, the memory 25, the storage 26, and the processor 29 are connected to each other so as to be able to communicate with each other, for example, via a bus.

[0022] The communication IF 22 is an interface for transmitting and receiving signals so that the server 20 can communicate with external devices. The input / output IF 23 functions as an interface with an input device for receiving input operations from a user and an output device for presenting information to the user. The memory 25 is for temporarily storing programs and data processed by the programs, etc., and is realized by a volatile memory such as a DRAM.

[0023] The storage 26 is a storage device for saving data, and is realized by a non-volatile memory such as a flash memory or a HDD. The storage 26 does not necessarily have to be realized by a single circuit. For example, the storage 26 may be realized by a plurality of storage circuits. The processor 29 is hardware for executing an instruction set written in a program, and is made up of an arithmetic unit, registers, peripheral circuits, etc.

[0024] <1.1 Terminal device configuration> 2 is a block diagram showing the configuration of the terminal device 10 included in the system 1 of this embodiment. As shown in FIG. 2, the terminal device 10 includes a communication unit 121, an input device 13, an output device 14, a storage unit 170, and a control unit 180.

[0025] The communication unit 121 performs processing for the terminal device 10 to communicate with other devices. The communication unit 121 performs transmission processing on a signal generated by the control unit 180 and transmits the signal to the outside (for example, the server 20). The communication unit 121 performs reception processing on a signal received from the outside and outputs the signal to the control unit 180.

[0026] The input device 13 is a device for a user who owns the terminal device 10 to input instructions. The input device 13 is realized by, for example, a mouse 131, a keyboard 132, and a touch-sensitive device 133 in which instructions are input by touching the operation surface. The input device 13 converts instructions input by the user into electrical signals and outputs the electrical signals to the control unit 180. Note that the input device 13 is not limited to physical operation devices such as the mouse 131 and the keyboard 132. The input device 13 may include, for example, a receiving port that receives electrical signals input from an external input device.

[0027] The output device 14 is a device for presenting information to a user who owns the terminal device 10. The output device 14 is realized, for example, by a display 141 or the like. The display 141 displays data such as images, videos, and text in accordance with the control of the control unit 180. The display 141 is realized, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display or the like.

[0028] The storage unit 170 is realized by, for example, the memory 15, the storage 16, etc., and stores data and programs used by the terminal device 10.

[0029] The control unit 180 is realized by the processor 19 reading a program stored in the storage unit 170 and executing instructions included in the program. The control unit 180 controls the operation of the terminal device 10. Specifically, for example, the control unit 180 fulfills the functions of an operation reception unit 181, a transmission / reception unit 182, and a display control unit 183.

[0030] The operation receiving unit 181 performs processing for receiving user operations input from the input device 13. The transmitting / receiving unit 182 performs processing for the terminal device 10 to transmit and receive data to and from external devices such as the server 20 in accordance with a communication protocol. The display control unit 183 controls the output device 14 to present various information to the user.

[0031] <1.2 Server Functional Configuration> 3 is a block diagram showing the functional configuration of the server 20 included in the system 1 of this embodiment. As shown in FIG. 3, the server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0032] The communication unit 201 performs processing for the server 20 to communicate with external devices.

[0033] The storage unit 202 stores data and programs used by the server 20. For example, the storage unit 202 stores a test case database (DB) 281 and a task database (DB) 282.

[0034] The test case DB 281 is a database for storing information about test cases.

[0035] The task DB 282 is a database for managing tasks that have occurred. In the task DB 282, tasks are managed, for example, on a ticket-by-ticket basis.

[0036] The control unit 203 is realized by the processor 29 reading a program stored in the storage unit 202 and executing instructions included in the program. The control unit 203 controls the operation of the server 20. Specifically, for example, the control unit 203 fulfills the functions of an operation reception unit 2031, a transmission / reception unit 2032, a storage control unit 2033, a presentation unit 2034, an analysis unit 2035, a setting unit 2036, and a prediction unit 2037.

[0037] The operation receiving unit 2031 receives operations from the user. For example, the operation receiving unit 2031 receives the specification of a test case by a tester as a user, the input of test implementation results, etc. The operation receiving unit 2031 also receives, for example, input on a ticket by a developer as a user.

[0038] The transmitting / receiving unit 2032 controls the process in which the server 20 transmits and receives data to and from external devices such as the terminal device 10 in accordance with a communication protocol.

[0039] The storage control unit 2033 controls the storage of information in the storage unit 202. Specifically, for example, when a tester inputs information about a test case that has already been tested, the storage control unit 2033 stores the input information in the test case DB 281. When a developer inputs a response to the results of the implemented test, the storage control unit 2033 stores information about the input response in the test case DB 281. The storage control unit 2033 stores information obtained by analyzing the results of the implemented test in the test case DB 281. The storage control unit 2033 stores prediction information based on the analysis results of the implemented test in the test case DB 281.

[0040] Furthermore, when information about a task is input by, for example, a tester or a developer, the storage control unit 2033 stores the input information in the task DB 282.

[0041] The presentation unit 2034 presents the information stored in the test case DB 281 in a manner desired by the user. Specifically, the presentation unit 2034 displays a project selection screen on the display 141 of the terminal device 10. The presentation unit 2034 also displays a test suite selection screen on the display 141. In this embodiment, a test suite includes, for example, multiple test case data for a predetermined function. The test case data is, for example, data in which test cases defined in a test design document are converted into a format that enables test implementation management. Each of the multiple test case data has a common data structure. A test suite includes, for example, a set of test case data that makes it easy for a tester to implement tests. A set of test case data that makes it easy for a tester to implement tests is, for example, test case data for the same function or test case data for the same module.

[0042] The presentation unit 2034 also causes the display 141 to display a screen for inputting the results of the test for the test case. The presentation unit 2034 also causes the display 141 to display a screen for displaying analysis information based on the results of the test. The presentation unit 2034 also causes the display 141 to display a screen for displaying prediction information based on the results of the test.

[0043] The analysis unit 2035 analyzes information related to the results of the tests. Specifically, the analysis unit 2035 analyzes the results of the tests and calculates the number of man-hours required, productivity, number of defects, defect rate, etc. The analysis unit 2035 also analyzes the results of the tests and extracts test cases that are likely to cause defects. Test cases that are likely to cause defects can be rephrased as test cases that should be prioritized for testing. The analysis unit 2035 extracts test cases that should be prioritized for testing based on, for example, a numerical value related to the number of defects that will occur (such as the number of defects or the defect rate), productivity, etc.

[0044] The analysis unit 2035 also analyzes responses from developers to the test results and extracts test cases that are likely to cause defects, i.e., test cases that should be prioritized for testing. The analysis unit 2035 extracts test cases that should be prioritized for testing, for example, based on the numerical value related to the man-hours required to repair the defect that occurred and the cause of the defect that occurred.

[0045] The setting unit 2036 sets the order in which tests should be performed on test suites or test case data. For example, before starting the testing process of a project, the setting unit 2036 sets the order in which tests should be performed on test suites or test case data. Specifically, the setting unit 2036 assigns a priority order to test suites corresponding to important functions in the software being developed and to the test case data included in these test suites. In addition, the setting unit 2036 assigns a priority order to test suites or test case data based on the results of identical or similar tests performed in the past.

[0046] The setting unit 2036 also sets the order in which test suites or test case data should be executed, for example, by using the results of tests already executed in the testing process. Specifically, the setting unit 2036 assigns a priority order to the data on the test cases extracted by the analysis unit 2035 and the test suites that include this test case data.

[0047] The prediction unit 2037 predicts the trend of tests when tests are performed in the order selected by the user. Specifically, for example, the prediction unit 2037 predicts the number of defects, including defects that occur in test cases that have not yet been performed, or the defect rate, based on test cases that have already been performed. Furthermore, the prediction unit 2037 predicts the number of defect repair man-hours, including the number of defect repair man-hours for defects that occur in test cases that have not yet been performed, and the number of post-repair confirmation man-hours, based on the number of repair man-hours for defects that occurred in test cases that have already been performed and the number of post-repair confirmation man-hours, for example.

[0048] <2 Data Structure> FIG. 4 is a diagram showing the data structure of the test case DB 281 stored in the server 20. The test case DB 281 shown in FIG. 4 includes the items "perspective," "factor," and "level," which are used to define test cases in a test design document. "Perspective" represents the standpoint (way of thinking) on ​​which the test is based. "Factor" represents a factor that can affect changes in the state of a computer. "Level" represents a specific element of a "factor." FIG. 4 shows a case where tests have already been conducted for factor A-levels AA to AC of perspective A and factor B-levels BA to BC of perspective B for function A. Also, FIG. 4 is an example and does not exclude data that is not listed.

[0049] 4 is constructed for test cases of a predetermined project, for example. If there are multiple projects, the test case DB 281 may be constructed for each project, for example.

[0050] As shown in Figure 4, each record in the test case DB281 includes an item such as "screen," an item such as "function," an item such as "person in charge," an item such as "module," an item such as "perspective," an item such as "factor," an item such as "level," an item such as "total number of cases," an item such as "number of cases already performed," an item such as "man-hours performed," an item such as "productivity," an item such as "cause of defect," an item such as "actual results," an item such as "prediction," an item such as "screen," an item such as "function," an item such as "person in charge ...

[0051] The item "screen" stores information about the screen on which the function is executed. The information about the screen includes, for example, the name of the screen or a link for accessing the screen data.

[0052] The "function" item stores information about the functions of the software. The information about the function includes the name of the function, such as "login function." The name of the function may be entered arbitrarily by the person in charge, or may be selected from pre-registered names.

[0053] The item "Person in Charge" stores the name of the developer of the function. Note that this is not limited to a name, and any other information that can identify the developer may be used. It may also be an identification number, address, or the like.

[0054] The item "module" stores the module that the tester operates when conducting the test.

[0055] The item "viewpoint" stores the viewpoint (idea) on which the test is based. When the item "function" is a "login function," the item "viewpoint" may store, for example, "successful authentication," "authentication check," "password retention," etc.

[0056] The "factor" item stores factors that may affect changes in the computer's status. When the "perspective" item is "successful authentication," the "factor" item may store, for example, "whether there is an ID / PW mismatch."

[0057] The item "Level" stores a group of specific operations in "Factor." If the item "Factor" is "ID / PW inconsistency," the item "Level" includes, for example, "ID inconsistent," "PW inconsistent," "Both ID and PW inconsistent," and "ID and PW both consistent."

[0058] The item "total number of cases" stores the total number of test cases occurring at each level.

[0059] The item "Number of Implemented Cases" stores the number of test cases that have been implemented. In the test case DB 281 shown in Fig. 4, all test cases have been implemented for factor A-levels AA to AC of viewpoint A in function A and factor B-levels BA to BC of viewpoint B.

[0060] The item "Man-hours Implemented" stores the man-hours required to implement the corresponding test case. The man-hours may be obtained based on a report from the tester, or may be obtained automatically based on the tester's operation log.

[0061] The item "productivity" stores the time taken for one test case. In the example shown in Figure 4, the number of minutes taken for one test case is stored.

[0062] The item "Failure Cause" stores the number of occurrences of each cause of defects that occurred during testing. The item "Failure Cause" includes multiple sub-items. The causes of defects that occurred are assigned to the sub-items. The causes assigned to the sub-items are assigned priorities according to the process in which the cause is embedded.

[0063] Software development involves work processes that proceed in the following order, for example: requirements definition, basic design, detailed design, coding, and testing. There is a possibility that defects may be embedded in each of these development processes. If a defect is embedded in an upstream process, the longer it is delayed in detecting it in the testing process, the greater the cost of fixing the defect. Furthermore, the longer it is delayed in detecting it in the testing process, the greater the risk of degradation occurring. Therefore, by setting priorities for the causes of defects according to the process in which the causes are embedded, it is possible to understand the importance of resolving the defect that has occurred.

[0064] For example, defect factors assigned to sub-items are classified into factors originating from upstream processes, factors originating from midstream processes, and factors originating from downstream processes. Factors originating from upstream processes are, for example, factors related to defects embedded in requirements definition, and have a higher priority than factors originating from midstream processes and factors originating from downstream processes. Factors originating from midstream processes are, for example, factors related to defects embedded in basic design and detailed design, and have a higher priority than factors originating from downstream processes. Factors originating from downstream processes are, for example, factors related to defects embedded in coding and testing.

[0065] Causes of defects include, for example, "omission of requirements," "omission of consideration of the scope of impact," "misunderstanding between subsystems," "omission of specification consideration," "model dependency," "programming error," "misconfiguration error," "omission of unit testing," and "existing bug." Of these, for example, "omission of requirements" and "omission of consideration of the scope of impact" correspond to factors originating in upstream processes, "misunderstanding between subsystems," "omission of specification consideration," and "model dependency" correspond to factors originating in midstream processes, and "programming error," "misconfiguration error," "omission of unit testing," and "existing bug" correspond to factors originating in downstream processes. Note that, although the processes are classified into three categories in the above, the number of categories is not limited to three and may be two, four, or more.

[0066] In the example shown in Figure 4, the sub-items include "Omission of requirements," "Impact consideration," "Inter-system awareness," "Specification consideration," "Work error," and "Environmental differences." "Omission of requirements" and "Impact consideration" belong to factors originating from upstream processes, "Inter-system awareness" and "Specification consideration" belong to factors originating from midstream processes, and "Work error" and "Environmental differences" belong to factors originating from downstream processes. Note that the factors assigned to the sub-items are not limited to these. Other factors may be added, or some factors may be absent. Also, some factors may be replaced with other factors.

[0067] The "Results" item stores the results of the tests that have been conducted. The "Results" item includes multiple sub-items. In the example shown in FIG. 4, the "Results" item includes the sub-item "Number of Defects," the sub-item "Defect Rate," the sub-item "Man-hours for Repair," and the sub-item "Man-hours for Post-Repair Verification."

[0068] The sub-item "Number of Defects" stores the number of defects that occurred among the corresponding test cases. The sub-item "Defect Rate" stores the probability that a defect occurred. The information represented in the sub-item "Number of Defects" and the sub-item "Defect Rate" is based on, for example, the test results entered by the tester.

[0069] The sub-item "Man-hours to repair" stores the man-hours required to repair the defect. The information represented by the sub-item "Man-hours to repair" is based, for example, on information input by a developer after the program has been repaired. The sub-item "Man-hours to verify after repair" stores the man-hours required to test the program after the defect has been repaired. The information represented by the sub-item "Man-hours to verify after repair" is based, for example, on the results of testing the program after the repair, input by a tester.

[0070] The item "Prediction" represents the predicted test results when tests are performed according to the proposed priority order. The item "Prediction" includes multiple sub-items. In the example shown in Figure 4, the item "Prediction" includes the sub-item "Number of Defects," the sub-item "Defect Rate," the sub-item "Repair Man-hours," and the sub-item "Post-Repair Verification Man-hours." The information represented by the sub-item "Number of Defects," the sub-item "Defect Rate," the sub-item "Repair Man-hours," and the sub-item "Post-Repair Verification Man-hours" is calculated, for example, based on the results of tests that have already been performed.

[0071] The test case DB 281 shown in FIG. 4 also includes the items "functional importance" and "priority." The item "function importance" stores the importance of the function of the software being developed. In the example shown in Figure 4, the importance is expressed as "high," "medium," or "low," but is not limited to this. The importance may also be expressed as a number.

[0072] The "Priority" item represents the priority of a test case. The "Priority" item includes multiple sub-items. In the example shown in FIG. 4, the "Priority" item includes the sub-items "Plan," "Modify 1," and "Modify 2." The "Plan" sub-item stores a priority set before the start of the testing process. The "Modify 1" sub-item stores a priority set at a predetermined timing during the testing process based on the results of tests already performed. In the example shown in FIG. 4, the "Modify 1" sub-item stores a priority set after performing tests on a test case set to priority 1 in the "Plan" sub-item, such as testing factor A of perspective A in function A and factor B of perspective B. The "Modify 2" sub-item stores a priority set after performing tests on a test case set to priority 1 in the "Modify 1" sub-item, such as testing factor A of perspective D in function C.

[0073] <3 operations> The operation of the system 1 when a test is administered by a test administrator will now be described.

[0074] <3.1 Testing> FIG. 5 is a diagram illustrating the operation of the system 1 when a test is administered by a tester.

[0075] First, a tester operating terminal device 10-1 operates terminal device 10-1 to access server 20. The tester operates terminal device 10-1 to display information about the project. In response to the operation from the tester, terminal device 10-1 requests information about the project from server 20.

[0076] In response to a request from the terminal device 10-1, the server 20 transmits information about the project to the terminal device 10-1.

[0077] The terminal device 10-1 presents the information received from the server 20 to the tester. Specifically, for example, the terminal device 10-1 causes the display 141 to display a project list.

[0078] FIG. 6 is a schematic diagram showing an example of a project list displayed on the display 141. The project list displays a list of projects currently underway. In the example shown in FIG. 6, two projects are displayed. A record representing each project includes, for example, an item "project key," an item "project name," an item "current process," an item "start date," and an item "status."

[0079] The terminal device 10-1 accepts a project selection from the tester (step S101). Specifically, the tester selects, for example, a project related to software that requires testing from the projects displayed in the list. Once the project is selected by the tester, the terminal device 10-1 requests information about the test suite of the selected project from the server 20.

[0080] In response to a request from the terminal device 10-1, the server 20 acquires information about the test suite from the test case DB 281. The server 20 transmits the acquired information to the terminal device 10-1.

[0081] The terminal device 10-1 presents the information received from the server 20 to the tester. Specifically, for example, the terminal device 10-1 causes the display 141 to display a list of test suites.

[0082] FIG. 7 is a schematic diagram showing an example of a test suite list displayed on the display 141. The test suite list displays test suites included in the selected project. In the example shown in FIG. 7, five test suites are displayed. A record representing each test suite includes, for example, the items "No.", "Test Suite," "Responsible Person," "Priority," "Scope," "Feature," "Number of Cases," "Progress," and "Update Date." The "Priority" item has sub-items "Plan" and "Revision 1." Before testing is performed, the "Plan" sub-item stores the priority set in the planning stage. Meanwhile, the "Revision 1" sub-item is still blank.

[0083] The terminal device 10-1 accepts the selection of a test suite from the tester (step S102). Specifically, the tester refers to the sub-item "Plan" and selects a test suite with priority 1, that is, a test suite that should be tested with the highest priority. Once the test suite is selected by the tester, the terminal device 10-1 requests information about the test case data of the selected test suite from the server 20.

[0084] In response to a request from the terminal device 10-1, the server 20 transmits information about the test case data to the terminal device 10-1.

[0085] The terminal device 10-1 presents the information received from the server 20 to the tester. Specifically, for example, the terminal device 10-1 causes the display 141 to display a list of test case data.

[0086] FIG. 8 is a schematic diagram showing an example of a list of test case data displayed on the display 141. The list of test case data displays the test case data included in the selected test suite in a list format. In the example shown in FIG. 8, five test case data are displayed. The record representing each test case data includes, for example, an item "ID," an item "major item," an item "medium item," an item "minor item," an item "implementation result," an item "priority," and an item "remarks."

[0087] The item "Execution result" has field objects 1411 and 1412. The field object 1411 of a test case that has not been tested displays "UnExe," which indicates that the test has not yet been executed.

[0088] The "Priority" item has a sub-item "Plan" and a sub-item "Fix 1." Before testing is performed, the sub-item "Plan" stores the priority set in the planning stage. In the example shown in Figure 8, test cases in a test suite with priority 1 are displayed, so priority 1 is set for all test cases. Meanwhile, the sub-item "Fix 1" is still blank.

[0089] The tester refers to the sub-item "Plan" and selects one of the test cases with priority 1, that is, the test case that should be tested with the highest priority. Once the tester selects a test case, terminal device 10-1 tests the selected test case.

[0090] The terminal device 10-1 receives input of the test results from the tester (step S103). Specifically, for example, if the tester tests the test case with ID:1 and no malfunction occurs, the tester presses the field object 1411, and the display of the field object 1411 changes to a display indicating that no malfunction occurred. As a result, the display of the field object 1411 becomes, for example, "OK."

[0091] The terminal device 10-1 transmits the test result indicating that no defects have occurred and the man-hours required for the test to the server 20. The server 20 stores the test result and the man-hours required in the test case DB 281 using the storage control unit 2033. The server 20 also analyzes the test result using the analysis unit 2035, and calculates, for example, the man-hours required, productivity, the number of defects, and the defect rate.

[0092] When the tester selects "OK" for the display of the field object 1411, he or she begins testing the next test case, for example, the test case with ID:2.

[0093] On the other hand, if a defect occurs when testing the test case with ID: 1, the tester presses the field object 1411 to change the display of the field object 1411 to indicate that a defect has occurred. As a result, the display of the field object 1411 becomes, for example, "NG."

[0094] The terminal device 10-1 transmits the test result indicating that a defect has occurred and the man-hours required for the test to the server 20. The server 20 stores the test result and the man-hours required in the test case DB 281 using the storage control unit 2033. The server 20 also analyzes the test result using the analysis unit 2035, and calculates, for example, the man-hours required, productivity, the number of defects, and the defect rate.

[0095] The tester switches the display of field object 1411 to "NG" and creates a ticket to be issued to the developer. A ticket is a unit for managing work in a project. In this embodiment, the ticket allows, for example, the tester and the developer to share information about the occurrence of a defect and manage program modification as a task for the developer. Specifically, for example, the tester presses field object 1411 to switch the display to "NG" and then presses field object 1412.

[0096] When the field object 1412 is pressed, the terminal device 10-1 displays the ticket input form on the display 141.

[0097] Fig. 9 is a schematic diagram showing an example of a ticket input form displayed on display 141. According to the example shown in Fig. 9, an input window 1413 is displayed on display 141. Input window 1413 is provided with boxes for inputting, for example, "Process," "Type," "Title," "Priority," "Severity," "Assignee," "Reporter," and "Due date."

[0098] The ongoing process is entered in "Process." The entry is made in a pull-down format, where options are displayed by clicking the downward triangle mark on the right edge of the box. In the example shown in Figure 9, the ticket is being issued in the test process, so "Test" is entered.

[0099] The type of ticket is entered in "Type." The entry is made, for example, in a pull-down format. In the example shown in Figure 9, the ticket is issued to have the developer fix the defect, so "Bug" is entered.

[0100] The title of the ticket is entered in "Title." For example, any content can be entered by the tester. In the example shown in Figure 9, the title "AAA was BBBed but a problem occurred where it was not CCCed" is entered.

[0101] "Priority" is where the priority of the task created by issuing a ticket is entered. "Severity" is where the importance of the task created by issuing a ticket is entered. "Assignee" is where the person in charge of the task created by issuing a ticket is entered. Generally, this is the software developer. "Reporter" is where the creator of the ticket is entered. These entries are made, for example, in a pull-down format.

[0102] The due date of the task created by issuing a ticket is entered in "Due date." This can be entered directly by the tester, or by specifying the date via a calendar.

[0103] When the tester has finished entering the content to be included in the ticket, he or she presses field object 1414. When field object 1414 is pressed, terminal device 10-1 transmits the content entered by the tester to server 20 and requests server 20 to register the transmitted content as a ticket. After registering the ticket in server 20, the tester begins testing the next test case, for example, test case ID:2.

[0104] When the server 20 receives a request from the terminal device 10-1, the storage control unit 2033 stores the received content in the task DB 282 and registers it as a new ticket. The server 20 also issues the registered ticket to the developer listed in "Assignee" (step S201).

[0105] When a ticket is issued by the server 20, the terminal device 10-2 operated by the developer notifies the developer that the ticket has been issued. Upon receiving the notification, the developer displays the contents of the issued ticket on the display 141 of the terminal device 10-2.

[0106] FIG. 10 is a schematic diagram showing an example of the contents of a ticket displayed on the display 141. After displaying the contents of the ticket to the developer, the terminal device 10-2 accepts input from the developer (step S104). Specifically, for example, if the developer checks the displayed contents and realizes that a defect has occurred in a test for a specific test case, the developer investigates the cause of the defect. After identifying the cause, the developer enters the defect cause in the area on the ticket where the defect cause is to be entered. In the example shown in FIG. 10, "Relative Issue" corresponds to the area where the defect cause is to be entered. When the defect cause is entered in "Relative Issue," the terminal device 10-2 considers the contents of the ticket to have been updated and transmits the entered information to the server 20. After the developer enters the defect cause in the ticket, he or she begins repairing the software in which the defect occurred.

[0107] When the server 20 receives the information from the terminal device 10-2, the storage control unit 2033 updates the task DB 282 based on the received information. The server 20 also notifies the tester listed in "Reporter" that the ticket has been updated (step S202).

[0108] The tester repeats the processes of steps S102 to S202 until the number of test cases becomes zero.

[0109] <3.2 Implementation of modification confirmation tests> In step S104 shown in FIG. 5, the developer inputs the cause of the defect into the ticket, and then starts repairing the software in which the defect occurred.

[0110] FIG. 11 is a diagram illustrating the operation of the system 1 when software is modified by a developer and the modified software is tested by a tester.

[0111] The terminal device 10-2 receives an input from the developer reporting that the software repair has been completed (step S105). Specifically, for example, the developer inputs the repair details of the defect and the man-hours required to repair the defect into the ticket. When the repair details of the defect and the man-hours required to repair the defect are input, the terminal device 10-2 regards the ticket contents as having been updated and transmits the entered information to the server 20.

[0112] When the server 20 receives the information from the terminal device 10-2, the storage control unit 2033 updates the task DB 282 based on the received information. The server 20 also notifies the tester listed in "Reporter" that the ticket has been updated (step S203).

[0113] When the terminal device 10-1 is notified of the ticket update by the server 20, it notifies the tester that the ticket has been updated. Upon receiving the notification, the tester displays the contents of the updated ticket on the display 141 of the terminal device 10-1. The tester refers to the displayed contents and re-tests the test case in which the defect occurred.

[0114] The terminal device 10-1 receives input of the test results from the tester (step S106). Specifically, for example, if the tester tests a test case for which the field object 1411 is "NG" and no problems occur, the tester presses the field object 1411 to change the display of the field object 1411 to "OK."

[0115] The terminal device 10-1 transmits the implementation result that the defect has been resolved and the man-hours required for post-repair confirmation to the server 20. The server 20 stores the test implementation result and the post-repair confirmation man-hours in the test case DB 281 by the storage control unit 2033.

[0116] Also, for example, if a problem occurs when testing a test case for which field object 1411 is "NG," the tester creates a ticket by pressing field object 1412. After completing input of the content to be included in the ticket, the tester presses field object 1414. When field object 1414 is pressed, terminal device 10-1 transmits the content input by the tester to server 20 and requests server 20 to register the transmitted content as a ticket.

[0117] When the server 20 receives a request from the terminal device 10-1, the storage control unit 2033 stores the received content in the task DB 282 and registers it as a new ticket. The server 20 also issues the registered ticket to the developer listed in "Assignee."

[0118] <3.3 Setting Priorities> FIG. 12 is a flowchart showing an example of the operation of the server 20 when setting priorities set in test suites.

[0119] First, the analysis unit 2035 determines whether or not a priority reset instruction has been received from the tester (step S21). For example, a field object for instructing priority resetting is displayed (not shown) in the test suite list display shown in FIG. 7 or the test case data list display shown in FIG. 8. If there is a specific reason, the tester requests the server 20 to reset the priority by pressing the field object. The specific reason may be, for example, that there is a low possibility of a defect occurring in the remaining test case data in the test suite currently being tested, or that the tester wishes to reset the priority.

[0120] If there is an instruction to reset the priority (Yes in step S21), the analysis unit 2035 proceeds to step S23. If there is no instruction to reset the priority (No in step S21), the analysis unit 2035 determines whether predetermined requirements have been met for the tests of the test case data (step S22). The predetermined requirements include, for example, whether enough tests have been performed to analyze the results of the tests, and specifically, the following: All test case data included in the selected test suite has been tested, and the cause of any defects that have occurred has been entered, or the defects that have occurred have been resolved. In the selected test suite, tests have been performed for all test case data related to the specified viewpoints, causes of any defects that have occurred have been entered, or the defects that have occurred have been resolved. In the selected test suite, testing of all test case data related to the specified factors has been performed, factors have been entered for the defects that have occurred, or the defects that have occurred have been resolved. In the selected test suite, testing of all test case data related to the specified level has been performed, the causes of any defects that have occurred have been entered, or the defects that have occurred have been resolved. In the selected test suite, tests have been performed for a preset number of test case data, causes have been entered for any defects that have occurred, or defects that have occurred have been resolved. · Tests have been conducted for a preset number of test case data, the causes of any defects that have occurred have been entered, or the defects that have occurred have been resolved. - The specified time has elapsed within the pre-set test period.

[0121] If the predetermined requirements for the test case data test are met (Yes in step S22), the analysis unit 2035 moves the process to step S23. If the predetermined requirements for the test case data test are not met (No in step S22), the analysis unit 2035 repeats the processes of steps S21 to S22.

[0122] In step S23, the analysis unit 2035 analyzes the results of the performed tests and extracts, from the remaining test cases, test cases that should be prioritized for testing. In the example shown in Fig. 4, when tests are performed on test cases included in a test suite that was set to priority 1 during planning, the analysis unit 2035 analyzes the results of the performed tests and extracts, from the remaining test cases, test cases that should be prioritized for testing. Note that in Fig. 4, the performance of tests on test cases included in a test suite that was set to priority 1 during planning may be rephrased as the performance of tests on a preset number of test case data.

[0123] FIG. 13 is a flowchart showing a specific example of the operation of the analysis unit 2035. First, the analysis unit 2035 reads out the person in charge, module, viewpoint, factor, level, defect cause, and performance for the test case data for which the test has been performed from the test case DB 281 (step S231).

[0124] The analysis unit 2035 selects test cases that should be prioritized for testing from among the test cases that have been tested based on the implementation results (step S232). Specifically, for example, the analysis unit 2035 selects test cases that should be prioritized for testing, taking into consideration a numerical value related to the number of defects that occur (number of defects, defect rate, etc.), a numerical value related to the man-hours required to repair the defects that have occurred, the project process from which the cause of the defect that has occurred originates, or a combination of at least two of these.

[0125] More specifically, for example, if there is a test case group in which the numerical value relating to the number of occurrences of defects or the numerical value relating to the man-hours required to repair the occurred defects is higher than a limit value requiring special measures, the analysis unit 2035 selects the test case group for which the numerical value is calculated. In this embodiment, a test case group refers to a group of test cases that are treated as test cases of similar nature, and is expressed, for example, by factor or level. If the numerical value relating to the number of occurrences of defects and the numerical value relating to the man-hours required to repair the occurred defects do not exceed the limit values, the analysis unit 2035 selects, for example, a test case group in which a defect occurred due to a factor originating in an upstream process of the project.

[0126] 4, the analysis unit 2035 selects a test case group related to factor A, which has been confirmed to have factors of "missing requirements" and "considering impact," originating from an upstream process, as there are no test case groups for which the numerical values ​​related to the number of defects that occur and the numerical values ​​related to the man-hours required to repair the defects that have occurred exceed the limit values. In addition, the analysis unit 2035 selects a test case group related to factor B, which has been confirmed to have factors of "work error," originating from a downstream process, as the test case group that should be tested with the next highest priority.

[0127] Note that the selection of the test case group may be performed in units of other elements that define the test case group. For example, the analysis unit 2035 may select the test case group based on a viewpoint or a level.

[0128] The analysis unit 2035 identifies the characteristics of the selected test cases (step S233). For example, the analysis unit 2035 identifies elements that define each of the selected test cases, such as screens, functions, personnel in charge, modules, viewpoints, factors, and levels.

[0129] 4, the analysis unit 2035 identifies "Screen A, Function A, Person in Charge AAAA, Module A, Viewpoint A, Factor A, Level AA to Level AC" as the characteristics of a group of test cases in which a defect due to a factor originating from an upstream process has occurred. Also, the analysis unit 2035 identifies "Screen A, Function A, Person in Charge AAAA, Module A, Viewpoint B, Factor B, Level BA to Level BC" as the characteristics of a group of test cases in which a defect due to a factor originating from a downstream process has occurred.

[0130] The analysis unit 2035 extracts test cases that match the identified characteristics or similar test cases from the untested test cases (step S234). A matching test case is, for example, a test case in which all elements that define the test case, such as "screen, function, person in charge, module, viewpoint, factor, level," match the identified characteristics of the executed test case. A similar test case is, for example, a test case in which at least one of the elements that define the test case, such as "screen, function, person in charge, module, viewpoint, factor, level," matches the identified characteristics of the executed test case. The analysis unit 2035 may vary the degree of similarity depending on the matching elements. For example, if the "factors, levels" match, the analysis unit 2035 determines that the contents of the test cases are close and evaluates the degree of similarity as high. On the other hand, if the viewpoints, modules, or personnel match, the analysis unit 2035 evaluates that although there is a relationship, the degree of similarity is lower than when the "factors, levels" match.

[0131] In the example shown in FIG. 4, the analysis unit 2035 extracts test cases with matching "factors and levels" from untested test cases as test case groups with high similarity to the identified features of "Screen A, Function A, AAAA, Module A, Viewpoint A, Factor A, Levels AA to AC." That is, the analysis unit 2035 extracts test cases with "Screen B, Function C, AAAA, Module A, Viewpoint D, Factor A, Levels AA to AC" from untested test cases as similar test case groups.

[0132] In addition, the analysis unit 2035 extracts a test case group of "Screen B, Function B, BBBB, Module C, Viewpoint A, Factor B, Level BA to Level BC" and a test case group of "Screen A, Function A, AAAA, Module B, Viewpoint C, Factor C, Level CA to Level CC" as test case groups that have low similarity to the characteristics of the identified "Screen A, Function A, AAAA, Module A, Viewpoint A, Factor A, Level AA to Level AC."

[0133] In addition, the analysis unit 2035 extracts test cases with matching "factors and levels" from untested test cases as test cases with high similarity to the identified features of "Screen A, Function A, AAAA, Module B, Viewpoint B, Factor B, Level BA to Level BC." That is, the analysis unit 2035 extracts test cases with "Screen B, Function B, BBBB, Module C, Viewpoint A, Factor B, Level BA to Level BC" from untested test cases as similar test cases.

[0134] In addition, the analysis unit 2035 extracts a test case group of "Screen A, Function A, AAAA, Module B, Viewpoint B, Factor B, Level BA to Level BC" that has a low similarity to the characteristics of the identified "Screen A, Function A, AAAA, Module A, Viewpoint A, Factor A, Level AA to Level AC" and a test case group of "Screen A, Function A, AAAA, Module B, Viewpoint C, Factor C, Level CA to Level CC."

[0135] The analysis unit 2035 may determine similarity based on a functional specification associated with the test case or test suite. A functional specification is data describing specifications for the functions possessed by the test target. A test design document that defines a test case is created with reference to the functional specification. Information about the functional specification referenced during creation is associated with the test case or test suite. If a functional specification is associated with the test case selected in step S232, the analysis unit 2035 may extract test cases associated with the same functional specification from untested test cases as similar test cases.

[0136] Returning to the explanation of FIG. 12, the setting unit 2036 refers to the test cases extracted in step S23 and sets priorities for the test cases that have not yet been tested (step S24).

[0137] Specifically, the setting unit 2036 sets a high priority to a test case that matches or is highly similar to a test case that should be given priority for testing. More specifically, the setting unit 2036 sets the priority by comprehensively considering the reasons why a test should be given priority, the degree of similarity to the selected test case, etc. Reasons why a test should be given priority include various factors, such as circumstances specific to the project, the amount of repair work required, the defect rate, and the unevenness of the skills of the people in charge.

[0138] 4, the setting unit 2036 sets a priority of 1 for the extracted test case group of "Screen A, Function A, AAAA, Module A, Viewpoint A, Factor A, Levels AA to AC." Furthermore, the setting unit 2036 may set a priority for each level in this test suite based on the defect causes, defect rate, repair man-hours, etc. of tests that have already been performed. For example, the setting unit 2036 may set a priority of 1 for the test case group of level AA, a priority of 2 for the test case group of level AC, and a priority of 3 for the test case group of level AB.

[0139] The setting unit 2036 also sets a priority of 2 for the extracted test case group of "Screen B, Function B, BBBB, Module C, Viewpoint A, Factor B, Levels BA to BC." The setting unit 2036 may also set a priority for each level in this test suite based on the defect causes, defect rate, repair man-hours, etc. of tests that have already been performed. For example, the setting unit 2036 may set a priority of 1 for the test case group of level BA, a priority of 2 for the test case group of level BC, and a priority of 3 for the test case group of level BB.

[0140] Furthermore, the setting unit 2036 sets a priority of 3 to "screen A, function A, AAAA, module B, viewpoint C, factor C, levels CA to CC" since the degree of similarity is low.

[0141] Depending on the track record of the selected test case, the cause of the defect, and the degree of similarity of the test case extracted from the untested test cases, the same priority may be set for the test cases.

[0142] The storage control unit 2033 stores the set priority in the test case DB 281 as the corrected priority. 14 to 17 are schematic diagrams showing examples of what the display 141 shows when priorities are corrected based on the results of tests that have already been carried out.

[0143] Fig. 14 is a schematic diagram showing an example of a list of test suites displayed on the display 141. The example shown in Fig. 14 shows the display screen of the display 141 when the priorities are corrected by the setting unit 2036 after testing of the test suite for function A-1 is completed. According to the example shown in Fig. 14, the priority of the test suite for function C, which was priority 4, is now 1, the priority of the test suite for function B, which was priority 3, is now 2, the priority of the test suite for function A-2, which was priority 2, is now 3, and the priority of the test suite for function D, which was priority 5, is now 4.

[0144] Fig. 15 is a schematic diagram showing an example of a list of test case data displayed on the display 141. The example shown in Fig. 15 shows a list display of test case data when a test suite for function C with priority 1 is selected in Fig. 14. According to the example shown in Fig. 15, the priority of the test case data that was priority 4 has been changed to priority 1.

[0145] Fig. 16 is a schematic diagram showing an example of a list of test suites displayed on the display 141. The example shown in Fig. 16 shows the display screen of the display 141 when the priorities are corrected by the setting unit 2036 after testing of the test suite for function C is completed. According to the example shown in Fig. 16, the priority of the test suite for function B, which was priority 2, is now 1, the priority of the test suite for function A-2, which was priority 3, is now 2, and the priority of the test suite for function D, which was priority 4, is now 3.

[0146] Fig. 17 is a schematic diagram showing an example of a list of test case data displayed on the display 141. The example shown in Fig. 17 shows a list display of test case data when a test suite for function B with priority 1 is selected in Fig. 16. According to the example shown in Fig. 17, the priority of the test case data that was priority 2 has been changed to priority 1.

[0147] 3.4 Test Trend Prediction FIG. 18 is a flowchart showing an example of the operation of the server 20 when predicting test trends.

[0148] When a tester wants to check the test trend, i.e., the trend in the number of remaining test cases, the predicted trend in the number of defects, etc., the tester presses a field object to instruct the display of the test trend. The field object is displayed on the display 141, for example.

[0149] When the field object is pressed, the prediction unit 2037 determines whether the number of test cases that have already been tested satisfies a predetermined requirement (step S25). The predetermined requirement includes, for example, that a sufficient number of tests have been conducted to enable the trend of the tests to be predicted, and specifically, the following requirements: - Testing of all test case data included in any test suite has been carried out, or any defects that have occurred have been resolved. - In any test suite, testing has been performed for all test case data related to a specified viewpoint, or any defects that have occurred have been resolved. - In any test suite, testing has been performed for all test case data related to the specified factors, or any defects that occurred have been resolved. - In any test suite, testing has been performed for all test case data related to the specified level, or any defects that have occurred have been resolved. - Tests have been performed for a predetermined number of test case data in any test suite, or any defects that have occurred have been resolved. - Testing has been carried out for a predetermined number of test case data, or any defects that have occurred have been resolved.

[0150] If the number of test cases that have already been tested meets the predetermined requirements (Yes in step S25), the analysis unit 2035 identifies the characteristics of the test cases that have been tested (step S26). For example, the analysis unit 2035 identifies elements that define the test cases that have been tested, such as screens, functions, personnel in charge, modules, perspectives, factors, and levels. If the number of test cases that have already been tested does not meet the predetermined requirements (No in step S25), the analysis unit 2035 repeats the processing of step S25 until the number of test cases meets the requirements.

[0151] The analysis unit 2035 extracts test cases that match or are similar to the identified characteristics from the untested test cases (step S27). For example, the analysis unit 2035 extracts test cases from the untested test cases whose screens, functions, responsible parties, modules, perspectives, factors, and levels match the screens, functions, responsible parties, modules, perspectives, factors, and levels identified for the executed test cases. The analysis unit 2035 also extracts test cases from the untested test cases whose screens, functions, responsible parties, modules, perspectives, factors, and levels match the screens, functions, responsible parties, modules, perspectives, factors, and levels identified for the executed test cases.

[0152] The prediction unit 2037 calculates various parameters representing test trends for the extracted test cases based on the performance of the executed test cases and the defect factors (step S28). Specifically, the prediction unit 2037 calculates the number of defects, defect rate, man-hours, repair man-hours, or post-repair confirmation man-hours for the extracted test case group based on the performance of the executed test cases.

[0153] For example, the prediction unit 2037 sets a defect rate for a test case group corresponding to a test case group having a predetermined factor and level among executed test cases and a test case group having the same factor and level. The prediction unit 2037 may also correct the defect rate based on factors other than the factor and level. The prediction unit 2037 multiplies the number of cases in the test case group having the same factor and level by the set defect rate to calculate the number of defects for this test case group. The prediction unit 2037 calculates the man-hours, repair man-hours, or post-repair confirmation man-hours for this test case group based on, for example, the ratio between the number of defects in the corresponding test case group and the number of defects in the test case group having the same factor and level.

[0154] If there is a test case group that has the same factors and levels as a test case group in which a defect occurred due to a factor originating in an upstream process among the untested test cases, the prediction unit 2037 may calculate the number of defects predicted to occur from this test case group so that the later the test is performed, the greater the number of defects.

[0155] The prediction unit 2037 calculates the transition of the number of defects, the transition of the remaining test cases, the transition of the number of recovered defects, or the transition of the number of recovered cases for the period from the start of the testing process based on the number of defects, the man-hours, the repair man-hours, or the man-hours for post-repair confirmation for the executed tests and the number of defects, the man-hours, the repair man-hours, or the man-hours for post-repair confirmation predicted for the unexecuted test cases. At this time, the prediction unit 2037 calculates the transition when tests are executed in the order according to the set priority.

[0156] The storage control unit 2033 stores the calculated parameters in the test case DB 281 as predicted parameters.

[0157] The presentation unit 2034 presents the test trend to the user based on the calculated parameters (step S29). Specifically, for example, the presentation unit 2034 causes the display 141 to display a graph showing the test trend.

[0158] Fig. 19 is a schematic diagram showing an example of a graph showing test trends, which is displayed on the display 141. In the diagram shown in Fig. 19, the horizontal axis represents the test period, and the vertical axis represents the remaining number or the number of occurrences. In Fig. 19, the solid line represents the number of defects, the dashed line represents the number of defect fixes, the one-dot chain line represents the remaining number of cases, and the two-dot chain line represents the remaining number of fix confirmations.

[0159] As described above, in this embodiment, the server 20 stores the results of tests performed in the testing process in the test case DB 281 using the storage control unit 2033. Based on the results, the analysis unit 2035 selects test cases to be prioritized from among the test cases that have been tested. During the testing process, the analysis unit 2035 extracts test cases similar to the selected test case from among the untested test cases. The setting unit 2036 then sets the priority of the extracted test cases to be higher than that of the other untested test cases. This allows the server 20 to prioritize tests to be performed from among the untested tests based on the results of tests performed in the testing process. Furthermore, because tests with a high probability of causing defects can be started first, the chain reaction of new bugs resulting from software modifications can be prevented. This reduces the risk of degradation.

[0160] In this embodiment, the analysis unit 2035 selects a test case that should be given the next highest priority from among the test cases that have already been tested. The analysis unit 2035 extracts test cases that are similar to the selected test case that should be given the next highest priority from among the untested test cases. The setting unit 2036 then sets the priority of the extracted test case to be lower than the previously set priority. This allows the server 20 to re-set the order in which untested tests should be performed based on the results of tests performed in the testing process.

[0161] Furthermore, in this embodiment, the analysis unit 2035 extracts test cases that match any of the elements that define a tested test case as similar test cases from untested test cases. Specifically, for example, the analysis unit 2035 determines test cases with factors and levels that match the factors and levels of a tested test case as similar test cases. Furthermore, for example, the analysis unit 2035 determines test cases whose functional specifications referenced when creating a tested test case match those of a tested test case as similar test cases. This enables the server 20 to extract test cases similar to tested test cases from untested test cases with high accuracy.

[0162] In this embodiment, the analysis unit 2035 selects test cases that should be tested with priority in response to instructions from the user, which allows the server 20 to set priorities at the timing desired by the user.

[0163] In this embodiment, the analysis unit 2035 selects a test case to be prioritized when a predetermined number of tests have been performed, which allows the server 20 to set priorities without any instructions from the user.

[0164] In this embodiment, the analysis unit 2035 selects the test cases to be tested with priority based on the numerical value related to the number of defects included in the implementation results. This makes it possible to assign personnel who are good at reporting problems to cases with a high defect rate. Furthermore, since defects are expected to occur in the first half of the test, it becomes possible to efficiently assign developers.

[0165] In this embodiment, the analysis unit 2035 selects test cases that should be prioritized for testing based on the numerical value of the man-hours required to repair the defects that occurred, which is included in the implementation results. This allows test cases that are expected to require a large number of man-hours to repair to be tested in the first half of the testing process, making it possible to efficiently assign developers. In addition, it is possible to assign dedicated personnel to cases that require a long time.

[0166] In this embodiment, the storage control unit 2033 stores the causes of defects that occurred during testing in the testing process in the test case DB 281. The analysis unit 2035 selects test cases to be prioritized based on the causes of the defects included in the execution results, specifically, the project process from which the causes originate. This enables tests to be prioritized to detect defects embedded in upstream processes among the requirements definition process, design process, coding process, and testing process in software development. This reduces the cost of repairing defects and the risk of degradation. The server 20 also sets priorities using the results of tests performed in the testing process within the project. This allows testers to understand the test order appropriate for the project in real time.

[0167] In this embodiment, the presentation unit 2034 presents the set priorities to the user, allowing the tester to conduct the test while recognizing the test cases with high priorities.

[0168] In this embodiment, the presentation unit 2034 assigns a priority to each set of test cases (test suite) or each individual test case and presents the priority to the user, allowing the tester to efficiently access test cases that must be tested with priority.

[0169] In this embodiment, the prediction unit 2037 predicts the test results of the extracted test cases from the test execution results. The prediction unit 2037 then predicts the test trends when untested test cases are tested in the order of the set priorities. This allows the tester to visually grasp how the tests will progress in the testing process if the set priorities are followed. This allows the tester to intuitively determine whether the set priorities are correct.

[0170] Therefore, according to the server 20 according to this embodiment, tests can be carried out efficiently and effectively in the test process.

[0171] <Modification> In the above embodiment, the priority of the functions to be tested is set at the time of planning. However, the priority of the functions to be tested does not have to be set at the time of planning.

[0172] In the above embodiment, the test suites are displayed in the order of priority at the time of planning. However, the order in which the test suites are displayed may be the order of priority after resetting.

[0173] In the above embodiment, the display control unit 183 displays the priority on the display 141. However, the display control unit 183 does not necessarily need to display the priority. The display control unit 183 may arrange the test suites or test cases in the order of the set priority and display the test suites or test cases. This reduces the burden on the tester because the optimal order is achieved by performing the tests from top to bottom.

[0174] Although several embodiments of the present disclosure have been described above, these embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and modifications are intended to be included in the scope of the inventions and their equivalents as defined in the claims, as well as in the scope and spirit of the inventions.

[0175] <Additional Notes> The matters described in the above embodiments will be supplemented below.

[0176] (Appendix 1) A program to be executed by a computer having a processor 29 and a memory 25, the program causing the processor to execute the steps of storing information about tests performed in a testing process (steps S201, S202), selecting a test case to be tested with priority from among test cases that have already been tested based on the information about the test (step S232), extracting test cases similar to the selected test case from among untested test cases in the testing process (step S234), and setting the priority of testing the extracted test case higher than that of other untested test cases (step S24).

[0177] (Appendix 2) A program described in Appendix 1, in which in the selecting step, a test case that should be tested next is selected from among test cases that have already been tested, in the extracting step, a test case similar to the selected test case that should be tested next is extracted from test cases that have not yet been tested, and in the setting step, the priority for testing the extracted test case is set lower than the set priority.

[0178] (Appendix 3) A program described in (Appendix 1) or (Appendix 2), in which, in the extraction step, test cases that match any of the elements that define test cases that have been tested are extracted as similar test cases from test cases that have not been tested.

[0179] (Appendix 4) In the extraction step, a test case having factors and levels that match the factors and levels of a test case that has already been tested is considered to be a similar test case (Appendix 3).

[0180] (Appendix 5) In the extraction step, a test case that has already been tested and a test case that matches the functional specification referenced when creating the test case are considered to be similar test cases (Appendix 3).

[0181] (Appendix 6) The program according to any one of (Supplementary Note 1) to (Supplementary Note 5), wherein in the selecting step, a test case to be tested with priority is selected in response to an instruction from a user (Step S21).

[0182] (Appendix 7) The program according to any one of (Supplementary Note 1) to (Supplementary Note 5), wherein in the selecting step, when a predetermined number of tests have been performed, a test case to be preferentially tested is selected (Step S22).

[0183] (Appendix 8) A program described in any one of (Appendix 1) to (Appendix 7), wherein in the storing step, the test implementation results are stored, and in the selecting step, test cases that should be tested with priority are selected based on a numerical value related to the number of defects that occur, which is included in the implementation results.

[0184] (Appendix 9) A program described in any one of (Appendix 1) to (Appendix 7), wherein in the storing step, the man-hours required to repair defects that occur during testing in the testing process are stored, and in the selecting step, test cases that should be tested with priority are selected based on the man-hours required to repair the defects that occur.

[0185] (Appendix 10) A program described in any one of (Appendix 1) to (Appendix 7), wherein in the storing step, the cause of a defect that occurred during testing in the test process is stored, and in the selecting step, the test case that should be tested with priority is selected based on the cause of the defect that occurred.

[0186] (Appendix 11) In the selection step, a test case that should be tested as a priority is selected based on the project process from which the factor originates (Appendix 10).

[0187] (Appendix 12) 10. The program according to claim 1, further comprising: a processor configured to execute a step of presenting the set priority to a user (presentation unit 2034).

[0188] (Appendix 13) The program according to claim 12, wherein in the presenting step, a priority is assigned to each set of test cases or each individual test case, and the priorities are presented to the user.

[0189] (Appendix 14) A program described in any one of (Appendix 1) to (Appendix 13) that causes a processor to execute a step (step S28) of predicting the test results of the extracted test cases from the test execution results, and a step (step S28) of predicting the test trends when tests of untested test cases are executed in an order according to the set priority.

[0190] (Appendix 15) A method executed by a computer having a processor 29 and a memory 25, the method comprising the steps of: storing information about tests performed in a testing process; selecting a test case to be tested with priority from among test cases that have already been tested based on the information about the test; extracting test cases similar to the selected test case from untested test cases in the testing process; and setting the priority of testing the extracted test case higher than that of other untested test cases.

[0191] (Appendix 16) An information processing device (20) comprising a processor (29) and a memory (25), wherein the processor executes the steps of: storing information about tests performed in a testing process (storage control unit (2033)); selecting a test case that should be tested with priority from among test cases that have already been tested based on the information about the test; extracting test cases similar to the selected test case from among untested test cases in the testing process (analysis unit (2035)); and setting the priority of testing the extracted test case higher than that of other untested test cases (setting unit (2036)).

[0192] (Appendix 17) A system comprising: a means for storing information about tests performed in a testing process (storage control unit 2033); a step for selecting a test case that should be tested with priority from among test cases that have already been tested based on the information about the test; a means for extracting test cases similar to the selected test case from among untested test cases in the testing process (analysis unit 2035); and a means for setting the priority of testing the extracted test case higher than that of other untested test cases (setting unit 2036). [Explanation of symbols]

[0193] 1. System 10...Terminal device 121…Communications Department 13...Input device 131...Mouse 132...Keyboard 133...Touch-sensitive devices 14...Output device 141...Display 15...Memory 16…Storage 170...Storage section 180...Control unit 181...Operation reception section 182...Transmitter / receiver 183...Display control unit 19...Processor 20...Server 201…Communications Department 202...Storage section 203...Control unit 2031...Operation reception area 2032...Transmitter / receiver 2033...Memory control unit 2034…Presentation part 2035…Analysis Department 2036…Settings 2037…Prediction Department 25…Memory 26…Storage 29...Processor 80…Network

Claims

1. A program to be executed by a computer having a processor and a memory, the program causing the processor to: storing information about tests performed in the testing process; selecting a test case to be tested with priority from among the test cases that have already been tested based on the information about the test; In the testing step, a step of extracting test cases similar to the selected test case from untested test cases; setting a priority for testing the extracted test case higher than that for other untested test cases; Execute In the selecting step, a program selects the test cases to be tested with priority based on a failure rate calculated from the test results included in the test information.

2. In the selecting step, a group of test cases having a failure rate higher than a predetermined value is selected, In the extracting step, a test case group similar to the selected test case group is extracted from untested test cases; 2. The program according to claim 1, wherein in the setting step, the priority of testing the extracted test cases is set higher than that of other untested test cases.

3. In the selecting step, a test case to be tested next is selected from the test cases that have already been tested; In the extracting step, a test case similar to the selected test case to be tested with next priority is extracted from the untested test cases; 2. The program according to claim 1, wherein in the setting step, a priority for executing a test on the extracted test case is set lower than the set priority.

4. 3. The program according to claim 1, wherein in the extracting step, test cases that match any of the elements that define the executed test cases are extracted as the similar test cases from the untested test cases.

5. predicting test results of the extracted test cases from execution results of the tests; predicting the test trends when the tests of the untested test cases are performed in the order according to the set priorities; 5. The program according to claim 1, which causes the processor to execute the following steps.

6. 1. A computer-implemented method comprising a processor and a memory, the method comprising: storing information about tests performed in the testing process; selecting a test case to be tested with priority from among the test cases that have already been tested based on the information about the test; In the testing step, a step of extracting test cases similar to the selected test case from untested test cases; setting a priority of testing the extracted test case higher than that of other untested test cases; Run In the selecting step, the test cases to be tested are selected based on a failure rate calculated from the test results included in the test information.

7. An information processing device including a processor and a memory, the processor: storing information about tests performed in the testing process; selecting a test case to be tested with priority from among the test cases that have already been tested based on the information about the test; In the testing step, a step of extracting test cases similar to the selected test case from untested test cases; setting a priority for testing the extracted test case higher than that for other untested test cases; Run In the selecting step, the information processing device selects the test cases to be preferentially tested based on a failure rate calculated from the test implementation results included in the information related to the test.

8. means for storing information about tests performed in the testing process; a means for selecting a test case to be tested with priority from among the test cases that have already been tested based on the information about the test; a means for extracting test cases similar to the selected test case from untested test cases in the testing step; a means for setting a priority of testing the extracted test case higher than that of other untested test cases; Equipped with The selecting means is a system that selects the test cases that should be tested with priority based on a failure rate calculated from the test execution results included in the test information.

Images