Program, method, information processing device and system
By analyzing and prioritizing software tests based on historical defect data and test results, the system efficiently addresses potential defects early, reducing the risk of degradation and repair costs in software development.
Patent Information
- Application Number
- JP2024168467
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-09-27
- Publication Date
- 2024-12-05
- Estimated Expiration
- 2040-12-10
AI Technical Summary
The order in which software tests are performed during the testing process can significantly impact the efficiency and risk of degradation in software development, with tests likely to cause failures being delayed, increasing the risk of higher repair costs and degradation if not addressed early.
A system and method that analyzes executed test cases to prioritize and preferentially test untested test cases based on their similarity and historical defect data, setting higher priorities for tests likely to cause defects and adjusting priorities based on test results.
This approach allows for more efficient testing by identifying and addressing potential defects early, reducing the risk of degradation and lowering the cost of defect repairs by prioritizing tests that are likely to fail.
Smart Images

Figure 2024169722000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a program, a method, an information processing device, and a system. [Background technology]
[0002] To ensure the quality of software, its functions are tested before it is released. Software function testing is generally performed based on a test specification, which describes the test method or the conditions for judging pass / fail for each test case.
[0003] Regarding software testing, it is important to efficiently create a test plan and to create a highly accurate test plan in order to proceed with a project. In Patent Document 1, the labor hours required for unexecuted test items are predicted based on the actual labor hours of the same or similar tests that were performed in the past. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] Patent Publication No. 2020-98413 Summary of the Invention [Problem to be solved by the invention]
[0005] In software development, the order in which tests are performed is an important factor in progressing the testing phase of a project as planned. For example, if there are tests that are likely to cause defects, the later those tests are performed, the higher the risk of degradation will be. For this reason, it is better to perform tests that are likely to cause defects at an early stage of the testing phase in order to efficiently proceed with the testing phase.
[0006] The present disclosure aims to perform tests efficiently. [Means for solving the problem]
[0007] In order to solve the above problems, the present disclosure provides a program for causing a computer having a processor and a memory to execute the following steps: storing information about tests performed in a testing process, selecting a test case to be preferentially tested from among the test cases that have been tested based on the information about the tests, extracting a test case similar to the selected test case from among untested test cases in the testing process, and setting a priority for testing the extracted test case higher than that of other untested test cases. Effect of the Invention
[0008] According to the present disclosure, tests can be performed efficiently. [Brief description of the drawings]
[0009] [Figure 1] 1 is a diagram showing the overall configuration of a system according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a block diagram showing a configuration of a terminal device included in the system of the present embodiment. [Diagram 3] FIG. 2 is a block diagram showing the functional configuration of a server included in the system of the present embodiment. [Figure 4] 13 is a diagram showing a data structure of a test case DB stored in a server. [Diagram 5] FIG. 2 is a diagram illustrating the operation of the system when a test is administered by a test administrator. [Figure 6] FIG. 11 is a schematic diagram illustrating an example of a project list displayed on a display. [Figure 7] FIG. 13 is a schematic diagram illustrating an example of a list of test suites displayed on a display. [Figure 8] FIG. 11 is a schematic diagram illustrating an example of a list of test case data displayed on a display. [Figure 9] FIG. 11 is a schematic diagram illustrating an example of a ticket input form displayed on a display. [Figure 10] FIG. 13 is a schematic diagram showing an example of the contents of a ticket displayed on a display. [Figure 11] FIG. 2 is a diagram illustrating the operation of the system when software is modified by a developer and the modified software is tested by a tester. [Figure 12] 13 is a flowchart illustrating an example of the operation of a server when setting priorities set for a test suite. [Figure 13] 13 is a flowchart showing a specific example of the operation of an analysis unit. [Figure 14] FIG. 13 is a schematic diagram illustrating an example of a list of test suites displayed on a display. [Figure 15] FIG. 11 is a schematic diagram illustrating an example of a list of test case data displayed on a display. [Figure 16] FIG. 13 is a schematic diagram illustrating an example of a list of test suites displayed on a display. [Figure 17] FIG. 11 is a schematic diagram illustrating an example of a list of test case data displayed on a display. [Figure 18] 13 is a flowchart illustrating an example of the operation of a server when predicting a test trend. [Figure 19] FIG. 13 is a schematic diagram illustrating an example of a graph showing test trends displayed on a display. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Hereinafter, an embodiment will be described with reference to the drawings. Note that the embodiment described below does not unduly limit the contents of the present disclosure described in the claims. In addition, not all of the configurations described in the embodiment are necessarily essential components of the present disclosure.
[0011] <Summary> The system 1 according to the present embodiment analyzes the test cases that have already been performed in the testing process of a given project, and extracts test cases that should be tested with priority from among the untested test cases. The system 1 prompts the tester to prioritize testing of the extracted test cases. Then, in response to a request from a tester, developer, project manager, etc., the system 1 presents the requester with a predicted trend of testing due to a change in the test execution order.
[0012] In this embodiment, the test cases are set based on a test design document. The test design document is data in which tests for inspecting functions of an inspection target are designed. The inspection target represents an object to be inspected, and is, for example, hardware or software developed in a specific project. The inspection target includes, for example, a prototype, an intermediate product, and a final product.
[0013] <1 Overall configuration> Fig. 1 is a diagram showing an overall configuration of a system 1 of the present embodiment. As shown in Fig. 1, the system 1 includes a terminal device 10 and a server 20. The terminal device 10 and the server 20 are connected to each other via a network 80 using a wired or wireless communication standard so as to be able to communicate with each other. The network 80 is realized by, for example, the Internet and / or a communication network provided by a communication carrier. In the illustrated example, a plurality of terminal devices 10 are included in the system 1.
[0014] Although FIG. 1 shows a case where the server 20 is one computer, the server 20 may be realized by combining a plurality of computers.
[0015] The terminal device 10 is a terminal used by a user who uses the services provided by the server 20. For example, the terminal device 10-1 is a terminal used by a tester who performs tests. The terminal device 10-2 is a terminal used by a developer who develops software. Although there are the terminal devices 10-1 and 10-2, the structures of the devices themselves are almost the same, so that they will be described below as the terminal device 10.
[0016] The terminal device 10 is realized, for example, by a desktop PC (Personal Computer), a laptop PC, a head mounted display, etc. The terminal device 10 may also be a portable computer such as a smartphone or a tablet terminal.
[0017] 1, the terminal device 10 includes a communication IF (Interface) 12, an input device 13, an output device 14, a memory 15, a storage 16, and a processor 19. The communication IF 12, the input device 13, the output device 14, the memory 15, the storage 16, and the processor 19 are connected to each other so as to be able to communicate with each other via a bus, for example.
[0018] The communication IF 12 is an interface for transmitting and receiving signals so that the terminal device 10 can communicate with external devices. The input device 13 is an input device for receiving input operations from a user (employee). The input device 13 includes, for example, a touch panel, a touch pad, a pointing device such as a mouse, a keyboard, etc. The output device 14 is an output device for presenting information to a user. The output device 14 includes, for example, a display, a speaker, etc.
[0019] The memory 15 is for temporarily storing programs and data processed by the programs, and is realized by a volatile memory such as a dynamic random access memory (DRAM). The storage 16 is a storage device for saving data and is realized by a non-volatile memory such as a flash memory or a hard disk drive (HDD). The processor 19 is hardware for executing an instruction set described in a program, and is configured by an arithmetic unit, a register, peripheral circuits, and the like.
[0020] The server 20 is a device that manages the implementation status and results of tests defined based on a test design document. The server 20 is, for example, a computer connected to a network 80.
[0021] 1, the server 20 includes a communication IF 22, an input / output IF 23, a memory 25, a storage 26, and a processor 29. The communication IF 22, the input / output IF 23, the memory 25, the storage 26, and the processor 29 are communicatively connected to each other via, for example, a bus.
[0022] The communication IF 22 is an interface for transmitting and receiving signals so that the server 20 can communicate with an external device. The input / output IF 23 functions as an interface with an input device for receiving input operations from a user and an output device for presenting information to the user. The memory 25 is for temporarily storing programs and data processed by the programs, etc., and is realized by a volatile memory such as a DRAM.
[0023] The storage 26 is a storage device for saving data, and is realized by a non-volatile memory such as a flash memory or a HDD. The storage 26 does not necessarily have to be realized by a single circuit. The storage 26 may be realized by, for example, a plurality of storage circuits. The processor 29 is hardware for executing an instruction set described in a program, and is configured by an arithmetic unit, a register, peripheral circuits, etc.
[0024] <1.1 Terminal device configuration> 2 is a block diagram showing the configuration of the terminal device 10 included in the system 1 of this embodiment. As shown in FIG. 2, the terminal device 10 includes a communication unit 121, an input device 13, an output device 14, a storage unit 170, and a control unit 180.
[0025] The communication unit 121 performs processing for the terminal device 10 to communicate with other devices. The communication unit 121 performs transmission processing on a signal generated by the control unit 180 and transmits the signal to the outside (for example, the server 20). The communication unit 121 performs reception processing on a signal received from the outside and outputs the signal to the control unit 180.
[0026] The input device 13 is a device for inputting instructions by a user who owns the terminal device 10. The input device 13 is realized by, for example, a mouse 131, a keyboard 132, and a touch-sensitive device 133 in which an instruction is input by touching an operation surface. The input device 13 converts an instruction input by a user into an electric signal, and outputs the electric signal to the control unit 180. Note that the input device 13 is not limited to physical operation devices such as the mouse 131 and the keyboard 132. The input device 13 may include, for example, a receiving port that receives an electric signal input from an external input device.
[0027] The output device 14 is a device for presenting information to a user who owns the terminal device 10. The output device 14 is realized, for example, by a display 141 or the like. The display 141 displays data such as images, videos, and text in response to control by the control unit 180. The display 141 is realized, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display or the like.
[0028] The storage unit 170 is realized by, for example, the memory 15, the storage 16, etc., and stores data and programs used by the terminal device 10.
[0029] The control unit 180 is realized by the processor 19 reading a program stored in the storage unit 170 and executing instructions included in the program. The control unit 180 controls the operation of the terminal device 10. Specifically, the control unit 180 performs the functions of an operation reception unit 181, a transmission / reception unit 182, and a display control unit 183, for example.
[0030] The operation reception unit 181 performs processing for receiving user operations input from the input device 13. The transmission / reception unit 182 performs processing for the terminal device 10 to transmit and receive data to and from an external device such as the server 20 in accordance with a communication protocol. The display control unit 183 controls the output device 14 to present various information to the user.
[0031] <1.2 Functional configuration of the server> 3 is a block diagram showing a functional configuration of the server 20 included in the system 1 of the present embodiment. As shown in FIG. 3, the server 20 fulfills the functions of a communication unit 201, a storage unit 202, and a control unit 203.
[0032] The communication unit 201 performs processing for the server 20 to communicate with external devices.
[0033] The storage unit 202 stores data and programs used by the server 20. For example, the storage unit 202 stores a test case database (DB) 281 and a task database (DB) 282.
[0034] The test case DB 281 is a database for storing information about test cases.
[0035] The task DB 282 is a database for managing tasks that have occurred. In the task DB 282, tasks are managed, for example, on a ticket basis.
[0036] The control unit 203 is realized by the processor 29 reading a program stored in the storage unit 202 and executing instructions included in the program. The control unit 203 controls the operation of the server 20. Specifically, for example, the control unit 203 exerts functions as an operation reception unit 2031, a transmission / reception unit 2032, a storage control unit 2033, a presentation unit 2034, an analysis unit 2035, a setting unit 2036, and a prediction unit 2037.
[0037] The operation receiving unit 2031 receives operations from a user. For example, the operation receiving unit 2031 receives a test case specification by a tester as a user, an input of the results of the test, etc. The operation receiving unit 2031 also receives an input to a ticket by a developer as a user, for example.
[0038] The transmission / reception unit 2032 controls the process in which the server 20 transmits and receives data to and from external devices such as the terminal device 10 in accordance with a communication protocol.
[0039] The storage control unit 2033 controls the storage of information in the storage unit 202. Specifically, for example, when a tester inputs information about a test case that has already been tested, the storage control unit 2033 stores the input information in the test case DB 281. When a developer inputs a response to the result of the test that has been performed, the storage control unit 2033 stores information about the input response in the test case DB 281. The storage control unit 2033 stores information obtained by analyzing the results of the performed test in the test case DB 281. The storage control unit 2033 stores prediction information based on the analysis results of the performed test in the test case DB 281.
[0040] Furthermore, when task-related information is input by a tester or developer, the storage control unit 2033 stores the input information in the task DB 282.
[0041] The presentation unit 2034 presents the information stored in the test case DB 281 in a manner desired by the user. Specifically, the presentation unit 2034 displays a project selection screen on the display 141 of the terminal device 10. The presentation unit 2034 also displays a test suite selection screen on the display 141. In this embodiment, the test suite includes, for example, multiple test case data for a predetermined function. The test case data is, for example, data in which test cases defined in a test design document are converted into a format that allows test implementation management. Each of the multiple test case data has a common data structure. The test suite includes, for example, a set of test case data that makes it easy for a tester to implement a test. The set of test case data that makes it easy for a tester to implement a test is, for example, test case data for the same function, or test case data for the same module.
[0042] The presentation unit 2034 also causes the display 141 to display a screen for inputting the results of the test for the test case. The presentation unit 2034 also causes the display 141 to display a screen for displaying analysis information based on the results of the test. The presentation unit 2034 also causes the display 141 to display a screen for displaying prediction information based on the results of the test.
[0043] The analysis unit 2035 analyzes information related to the results of the tests. Specifically, the analysis unit 2035 analyzes the results of the tests and calculates the number of steps, productivity, number of defects, defect rate, and the like. The analysis unit 2035 also analyzes the results of the tests and extracts test cases that are likely to cause defects. Test cases that are likely to cause defects can be rephrased as test cases that should be given priority in testing. The analysis unit 2035 extracts test cases that should be given priority in testing based on, for example, a numerical value related to the number of defects that will occur (the number of defects or the defect rate, etc.), productivity, and the like.
[0044] The analysis unit 2035 also analyzes responses from developers to the results of the tests, and extracts test cases that are likely to cause defects, i.e., test cases that should be given priority in testing. The analysis unit 2035 extracts test cases that should be given priority, for example, based on a numerical value related to the labor required to repair a defect that has occurred and the cause of the defect that has occurred.
[0045] The setting unit 2036 sets the order in which tests should be performed on the test suites or test case data. For example, before starting the testing process of the project, the setting unit 2036 sets the order in which tests should be performed on the test suites or test case data. Specifically, the setting unit 2036 assigns a priority order to test suites corresponding to important functions in the software being developed and to test case data included in the test suites. In addition, the setting unit 2036 assigns a priority order to the test suites or test case data based on the results of the same or similar tests performed in the past.
[0046] The setting unit 2036 also sets the order in which test suites or test case data should be performed, for example, by using the results of tests already performed in the test process. Specifically, the setting unit 2036 assigns a priority order to data on test cases extracted by the analysis unit 2035 and test suites that include this test case data.
[0047] The prediction unit 2037 predicts the trend of tests when tests are performed in the order selected by the user. Specifically, for example, the prediction unit 2037 predicts the number of defects, including defects occurring in unexecuted test cases, or the defect rate, based on the executed test cases. In addition, the prediction unit 2037 predicts the defect repair man-hours, including the repair man-hours for defects occurring in unexecuted test cases, and the post-repair confirmation man-hours, based on the repair man-hours for defects occurring in the executed test cases and the post-repair confirmation man-hours, for example.
[0048] <2 Data Structure> FIG. 4 is a diagram showing a data structure of a test case DB281 stored in the server 20. The test case DB281 shown in FIG. 4 includes items such as "perspective," "factor," and "level," which are used to define test cases in a test design document. "Perspective" represents a standpoint (way of thinking) on which a test is based. "Factor" represents a factor that can affect changes in the state of a computer. "Level" represents a specific element of a "factor." FIG. 4 shows a case where tests have already been performed for factor A-levels AA to AC of perspective A and factor B-levels BA to BC of perspective B in function A. FIG. 4 is also an example, and does not exclude data that is not listed.
[0049] 4 is constructed for test cases of a specific project, for example. If there are multiple projects, the test case DB 281 may be constructed for each project, for example.
[0050] As shown in FIG. 4, each record in the test case DB281 includes an item such as a "screen", an item such as a "function", an item such as a "person in charge", an item such as a "module", an item such as a "perspective", an item such as a "factor", an item such as a "level", an item such as a "total number of cases", an item such as a "number of cases performed", an item such as a "number of labor hours performed", an item such as a "productivity", an item such as a "cause of defect", an item such as a "results ...
[0051] The item "screen" stores information about a screen on which a function is executed. The information about a screen includes, for example, the name of the screen or a link for accessing the screen data.
[0052] The item "function" stores information about functions possessed by the software. The information about the function includes the name of the function, for example, "login function." The name of the function may be arbitrarily input by the person in charge, or may be selected from those registered in advance.
[0053] The item "Person in Charge" stores the name of the developer of the function. Note that this is not limited to a name, and any other information that can identify the developer may be used. It may also be an identification number, an address, or the like.
[0054] The item "module" stores the module that the tester operates when conducting a test.
[0055] The item "viewpoint" stores a viewpoint (way of thinking) on which the test is based. When the item "function" is a "login function", the item "viewpoint" may store, for example, "successful authentication", "authentication check", "password retention", etc.
[0056] The item "factor" stores factors that may affect changes in the state of a computer. When the item "viewpoint" is "successful authentication," the item "factor" may store, for example, "whether there is an ID / PW mismatch."
[0057] The item "level" stores a group of specific operations in "factor." When the item "factor" is "ID / PW inconsistency," the item "level" includes, for example, "ID is inconsistent," "PW is inconsistent," "Both ID and PW are inconsistent," and "Both ID and PW are consistent."
[0058] The item "total number of cases" stores the total number of test cases occurring at each level.
[0059] The item “number of executed cases” stores the number of executed test cases. In the test case DB 281 shown in Fig. 4, all test cases have been executed for factor A-levels AA to AC of viewpoint A in function A and factor B-levels BA to BC of viewpoint B.
[0060] The item “Man-hours implemented” stores the man-hours required to implement the corresponding test case. The man-hours may be obtained based on a report from a tester, or may be obtained automatically based on the operation log of the tester.
[0061] The item "productivity" stores the time taken for one test case. In the example shown in Figure 4, the number of minutes taken for one test case is stored.
[0062] The item "Failure Cause" stores the number of occurrences of each cause of defects that occurred during testing. The item "Failure Cause" includes multiple sub-items. The causes of defects that occurred are assigned to the sub-items. The causes assigned to the sub-items are assigned priorities according to the process in which the cause is embedded.
[0063] In software development, the work processes are carried out in the following order, for example: requirements definition, basic design, detailed design, coding, and testing. There is a possibility that defects will be embedded in each of these development processes. If there is a defect embedded in an upstream process, the longer it is delayed in detecting the defect in the testing process, the greater the cost of fixing the defect. Furthermore, the longer it is delayed in detecting the defect in the testing process, the greater the risk of degradation occurring. Therefore, by setting priorities for the causes of the defect according to the process in which the cause is embedded, it is possible to understand the importance of resolving the defect that has occurred.
[0064] For example, defect factors assigned to subitems are classified into factors originating from an upstream process, factors originating from a midstream process, and factors originating from a downstream process. Factors originating from an upstream process are, for example, factors related to defects embedded in requirements definition, and have a higher priority than factors originating from a midstream process and factors originating from a downstream process. Factors originating from a midstream process are, for example, factors related to defects embedded in basic design and detailed design, and have a higher priority than factors originating from a downstream process. Factors originating from a downstream process are, for example, factors related to defects embedded in coding and testing.
[0065] Causes of defects include, for example, "omission of requirements," "failure to consider the scope of impact," "misunderstanding between subsystems," "failure to consider specifications," "model dependency," "programming error," "mistake in environment setting," "failure to conduct unit testing," and "existing bugs." Of these, for example, "omission of requirements" and "failure to consider the scope of impact" correspond to factors originating from upstream processes, "misunderstanding between subsystems," "failure to consider specifications," and "model dependency" correspond to factors originating from midstream processes, and "programming error," "mistake in environment setting," "failure to conduct unit testing," and "existing bugs" correspond to factors originating from downstream processes. Note that, although the processes are classified into three in the above, the number of classifications is not limited to three and may be two, four or more.
[0066] In the example shown in Figure 4, the sub-items include "Omission of requirements," "Impact consideration," "Inter-system awareness," "Specification consideration," "Work error," and "Environmental differences." "Omission of requirements" and "Impact consideration" belong to factors originating from upstream processes, "Inter-system awareness" and "Specification consideration" belong to factors originating from midstream processes, and "Work error" and "Environmental differences" belong to factors originating from downstream processes. Note that the factors assigned to the sub-items are not limited to these. Other factors may be added, or some of the factors may be absent. Also, some of the factors may be replaced with other factors.
[0067] The item "Results" stores the results of the tests that were performed. The item "Results" includes multiple sub-items. In the example shown in Fig. 4, the item "Results" includes the sub-item "Number of defects", the sub-item "Defect rate", the sub-item "Man-hours for repair", and the sub-item "Man-hours for post-repair verification".
[0068] The sub-item "Number of defects" stores the number of defects that occurred among the corresponding test cases. The sub-item "Defect rate" stores the probability that a defect occurred. The information represented in the sub-item "Number of defects" and the sub-item "Defect rate" is based on the test implementation results input by the tester, for example.
[0069] The sub-item "Man-hours to repair" stores the man-hours required to repair the defect. The information represented by the sub-item "Man-hours to repair" is based, for example, on information input by a developer after the program has been repaired. The sub-item "Man-hours to confirm after repair" stores the man-hours required to test the program after the defect has been repaired. The information represented by the sub-item "Man-hours to confirm after repair" is based, for example, on the results of testing the program after the repair, input by a tester.
[0070] The item "Prediction" represents the predicted test results when tests are performed according to the proposed priority order. The item "Prediction" includes multiple sub-items. In the example shown in FIG. 4, the item "Prediction" includes the sub-item "Number of Defects", the sub-item "Defect Rate", the sub-item "Repair Efforts", and the sub-item "Post-Repair Verification Efforts". The information represented by the sub-item "Number of Defects", the sub-item "Defect Rate", the sub-item "Repair Efforts", and the sub-item "Post-Repair Verification Efforts" is calculated, for example, based on the results of tests that have already been performed.
[0071] Moreover, the test case DB 281 shown in FIG. 4 includes the item "functional importance" and the item "priority". The item "importance of function" stores the importance of the function of the software being developed. In the example shown in Fig. 4, the importance is expressed as "high", "medium", or "low", but is not limited to this. The importance may be expressed as a number.
[0072] The item "Priority" represents the priority of a test case. The item "Priority" includes a plurality of subitems. In the example shown in FIG. 4, the item "Priority" includes the subitems "Plan", "Modify 1", and "Modify 2". The subitem "Plan" stores a priority set before the start of the test process. The subitem "Modify 1" stores a priority set based on the results of tests that have already been performed at a predetermined timing during the test process. In the example shown in FIG. 4, the subitem "Modify 1" stores a priority set after performing tests on a test case set to priority 1 in the subitem "Plan", for example, factor A of viewpoint A in function A and factor B of viewpoint B. The subitem "Modify 2" stores a priority set after performing tests on a test case set to priority 1 in the subitem "Modify 1", for example, factor A of viewpoint D in function C.
[0073] <3 operations> The operation of the system 1 when a test is administered by a test administrator will now be described.
[0074] <3.1 Testing> FIG. 5 is a diagram for explaining the operation of the system 1 when a test is administered by a test administrator.
[0075] First, a tester operating terminal device 10-1 operates terminal device 10-1 to access server 20. The tester operates terminal device 10-1 to display information related to the project. In response to the operation from the tester, terminal device 10-1 requests information related to the project from server 20.
[0076] In response to a request from the terminal device 10-1, the server 20 transmits information about the project to the terminal device 10-1.
[0077] The terminal device 10-1 presents to the tester the information received from the server 20. Specifically, for example, the terminal device 10-1 causes the display 141 to display a project list.
[0078] Fig. 6 is a schematic diagram showing an example of a project list displayed on the display 141. The project list displays a list of projects currently being carried out. In the example shown in Fig. 6, two projects are displayed. A record representing each project includes, for example, an item "project key", an item "project name", an item "current process", an item "start date", and an item "status".
[0079] The terminal device 10-1 accepts a project selection from a tester (step S101). Specifically, the tester selects, for example, a project related to software that requires testing from among the projects displayed in a list. When the project is selected by the tester, the terminal device 10-1 requests information about the test suite of the selected project from the server 20.
[0080] In response to a request from the terminal device 10-1, the server 20 acquires information about the test suite from the test case DB 281. The server 20 transmits the acquired information to the terminal device 10-1.
[0081] The terminal device 10-1 presents to the tester the information received from the server 20. Specifically, for example, the terminal device 10-1 causes the display 141 to display a list of test suites.
[0082] FIG. 7 is a schematic diagram showing an example of a test suite list displayed on the display 141. In the test suite list, test suites included in a selected project are displayed in a list form. In the example shown in FIG. 7, five test suites are displayed. A record representing each test suite includes, for example, an item "No.", an item "Test Suite," an item "Person in Charge," an item "Priority," an item "Scope," an item "Feature," an item "Number of Cases," an item "Progress," and an item "Update Date." The item "Priority" has a sub-item "Plan" and a sub-item "Modification 1." Before the test is performed, the sub-item "Plan" stores the priority set in the planning stage. Meanwhile, the sub-item "Modification 1" is still blank.
[0083] The terminal device 10-1 accepts the selection of a test suite from the tester (step S102). Specifically, the tester refers to the subitem "Plan" and selects a test suite with priority level 1, that is, a test suite that should be given the highest priority for testing. Once the test suite is selected by the tester, the terminal device 10-1 requests information on the test case data of the selected test suite from the server 20.
[0084] In response to a request from the terminal device 10-1, the server 20 transmits information about the test case data to the terminal device 10-1.
[0085] The terminal device 10-1 presents to the tester the information received from the server 20. Specifically, for example, the terminal device 10-1 causes the display 141 to display a list of test case data.
[0086] Fig. 8 is a schematic diagram showing an example of a list of test case data displayed on display 141. In the list of test case data, the test case data included in the selected test suite are displayed in a list form. In the example shown in Fig. 8, five test case data are displayed. The records representing each test case data include, for example, an item "ID", an item "major item", an item "medium item", an item "minor item", an item "implementation result", an item "priority", and an item "remarks".
[0087] The item "Execution result" is provided with field objects 1411 and 1412. In the field object 1411 of a test case for which a test has not been executed, "UnExe" is displayed, which indicates that the test has not yet been executed.
[0088] The item "Priority" has sub-items "Plan" and "Fix 1". Before testing is performed, the sub-item "Plan" stores the priority set in the planning stage. In the example shown in FIG. 8, test cases in a test suite with priority 1 are displayed, so priority 1 is set for all test cases. Meanwhile, the sub-item "Fix 1" is still blank.
[0089] The tester refers to the sub-item "Plan" and selects one of the test cases with priority 1, that is, the test case that should be given the highest priority. When the tester selects the test case, the terminal device 10-1 tests the selected test case.
[0090] The terminal device 10-1 accepts input of the results of the test from the tester (step S103). Specifically, for example, when the tester tests the test case with ID:1 and finds no malfunction, the tester presses the field object 1411, and the display of the field object 1411 is switched to a display indicating that no malfunction has occurred. As a result, the display of the field object 1411 becomes, for example, "OK."
[0091] The terminal device 10-1 transmits the implementation result indicating that no defect has occurred and the man-hours required for the test to the server 20. The server 20 stores the implementation result of the test and the man-hours required in the test case DB 281 by the storage control unit 2033. The server 20 also analyzes the implementation result of the test by the analysis unit 2035, and calculates, for example, the man-hours required, productivity, the number of defects, and the defect rate.
[0092] When the tester selects "OK" for the display of the field object 1411, he or she begins testing the next test case, for example, the test case with ID:2.
[0093] On the other hand, for example, if a defect occurs when testing a test case with ID:1, the tester presses the field object 1411 and switches the display of the field object 1411 to a display indicating that a defect has occurred. As a result, the display of the field object 1411 becomes, for example, "NG."
[0094] The terminal device 10-1 transmits the implementation result indicating that a defect has occurred and the man-hours required for the test to the server 20. The server 20 stores the implementation result of the test and the man-hours required in the test case DB 281 by the storage control unit 2033. The server 20 also analyzes the implementation result of the test by the analysis unit 2035, and calculates, for example, the man-hours required, productivity, the number of defects, and the defect rate.
[0095] The tester switches the display of field object 1411 to "NG" and creates a ticket to be issued to the developer. A ticket is a unit for managing work in a project. In this embodiment, the ticket allows, for example, the tester and the developer to share the occurrence of a defect and manage the modification of the program as the developer's task. Specifically, for example, the tester presses field object 1411 to switch the display to "NG" and then presses field object 1412.
[0096] When the field object 1412 is pressed, the terminal device 10-1 displays a ticket input form on the display 141.
[0097] Fig. 9 is a schematic diagram showing an example of a ticket input form displayed on display 141. According to the example shown in Fig. 9, an input window 1413 is displayed on display 141. The input window 1413 is provided with boxes for inputting, for example, "Process", "Type", "Title", "Priority", "Severity", "Assignee", "Reporter", and "Due date".
[0098] The ongoing process is entered in "Process." The input is done in a pull-down format that displays options when you click the downward triangle mark on the right edge of the box, for example. In the example shown in Figure 9, the ticket is being issued in the test process, so "Test" is entered.
[0099] The type of ticket is entered in "Type". The entry is made, for example, in a pull-down format. In the example shown in Fig. 9, the ticket is issued to have the developer fix the defect, so "Bug" is entered.
[0100] The title of the ticket is entered in "Title." For example, any content can be entered by the tester. In the example shown in Fig. 9, the title "AAA was BBBed, but a problem occurred where it was not CCCed" is entered.
[0101] "Priority" is the priority of the task created by issuing a ticket. "Severity" is the importance of the task created by issuing a ticket. "Assignee" is the person in charge of the task created by issuing a ticket. Generally, this is the software developer. "Reporter" is the person who created the ticket. These entries are made, for example, in a pull-down format.
[0102] In "Due date", the due date of the task created by issuing the ticket is input. The input may be directly input by the tester, or the date may be specified via a calendar.
[0103] When the tester has finished inputting the content to be included in the ticket, he or she presses field object 1414. When field object 1414 is pressed, terminal device 10-1 transmits the content input by the tester to server 20, and requests server 20 to register the transmitted content as a ticket. After registering the ticket in server 20, the tester begins testing the next test case, for example, test case ID:2.
[0104] When the server 20 receives a request from the terminal device 10-1, the storage control unit 2033 stores the received content in the task DB 282 and registers it as a new ticket. The server 20 also issues the registered ticket to the developer listed in "Assignee" (step S201).
[0105] When the ticket is issued by the server 20, the terminal device 10-2 operated by the developer notifies the developer that the ticket has been issued. Upon receiving the notification, the developer causes the contents of the issued ticket to be displayed on the display 141 of the terminal device 10-2.
[0106] FIG. 10 is a schematic diagram showing an example of the contents of a ticket displayed on the display 141. After displaying the contents of the ticket to the developer, the terminal device 10-2 accepts input from the developer (step S104). Specifically, for example, when the developer checks the displayed contents and understands that a defect has occurred in a test for a specific test case, the developer investigates the cause of the defect. After identifying the cause, the developer inputs the cause of the defect in the area for inputting the cause of the defect in the ticket. In the example shown in FIG. 10, "Relative Issue" corresponds to the area for inputting the cause of the defect. When the cause of the defect is input in "Relative Issue", the terminal device 10-2 transmits the input information to the server 20, regarding the contents of the ticket as having been updated. After inputting the cause of the defect in the ticket, the developer starts repairing the software in which the defect has occurred.
[0107] When the server 20 receives the information from the terminal device 10-2, the storage control unit 2033 updates the task DB 282 based on the received information. The server 20 also notifies the tester described in "Reporter" that the ticket has been updated (step S202).
[0108] The tester repeats the processes of steps S102 to S202 until the number of test cases becomes zero.
[0109] <3.2 Implementation of modification confirmation test> In step S104 shown in FIG. 5, the developer inputs the cause of the defect into the ticket, and then starts repairing the software in which the defect occurred.
[0110] FIG. 11 is a diagram for explaining the operation of the system 1 when software is modified by a developer and the modified software is tested by a tester.
[0111] The terminal device 10-2 receives an input from the developer reporting that the software repair has been completed (step S105). Specifically, for example, the developer inputs the repair content of the defect and the man-hours required for the repair into the ticket. When the repair content of the defect and the man-hours required for the repair are input, the terminal device 10-2 transmits the entered information to the server 20, regarding the ticket content as having been updated.
[0112] When the server 20 receives the information from the terminal device 10-2, the storage control unit 2033 updates the task DB 282 based on the received information. The server 20 also notifies the tester described in "Reporter" that the ticket has been updated (step S203).
[0113] When the terminal device 10-1 is notified of the ticket update by the server 20, it notifies the tester that the ticket has been updated. Upon receiving the notification, the tester causes the contents of the updated ticket to be displayed on the display 141 of the terminal device 10-1. The tester refers to the displayed contents and retests the test case in which the defect occurred.
[0114] The terminal device 10-1 receives an input of the test implementation result from the tester (step S106). Specifically, for example, when the tester implements a test of a test case for which the field object 1411 is "NG" and no malfunction occurs, the tester presses the field object 1411 to switch the display of the field object 1411 to "OK."
[0115] The terminal device 10-1 transmits the implementation result that the defect has been resolved and the man-hours required for post-repair confirmation to the server 20. The server 20 stores the test implementation result and the post-repair confirmation man-hours in the test case DB 281 by the storage control unit 2033.
[0116] Also, for example, when a problem occurs during testing of a test case for which field object 1411 is "NG," the tester creates a ticket by pressing field object 1412. When the tester has finished inputting the content to be included in the ticket, he or she presses field object 1414. When field object 1414 is pressed, terminal device 10-1 transmits the content input by the tester to server 20, and requests server 20 to register the transmitted content as a ticket.
[0117] When the server 20 receives a request from the terminal device 10-1, the storage control unit 2033 stores the received content in the task DB 282 and registers it as a new ticket. The server 20 also issues the registered ticket to the developer described in "Assignee."
[0118] <3.3 Setting Priorities> FIG. 12 is a flowchart showing an example of the operation of the server 20 when setting priorities set in test suites.
[0119] First, the analysis unit 2035 judges whether or not there is an instruction from the tester to reset the priority (step S21). For example, a field object for instructing the tester to reset the priority is displayed (not shown) in the list display of the test suite shown in FIG. 7 or the list display of the test case data shown in FIG. 8. If there is a specific reason, the tester requests the server 20 to reset the priority by pressing the field object. The specific reason may be, for example, that there is a low possibility of a defect occurring in the remaining test case data in the test suite during testing, or that the tester wishes to reset the priority.
[0120] If there is an instruction to reset the priority (Yes in step S21), the analysis unit 2035 moves the process to step S23. If there is no instruction to reset the priority (No in step S21), the analysis unit 2035 determines whether or not a predetermined requirement has been met for the test of the test case data (step S22). The predetermined requirement includes, for example, that a sufficient number of tests have been performed to analyze the results, and specifically, is as follows: - Tests for all test case data included in the selected test suite have been performed, causes for any defects that have occurred have been entered, or defects that have occurred have been resolved In the selected test suite, testing of all test case data related to the specified viewpoints has been performed, causes of the defect that occurred have been input, or the defect that occurred has been resolved. In the selected test suite, testing of all test case data related to the specified factors has been performed, factors have been input for the defect that occurred, or the defect that occurred has been resolved. In the selected test suite, testing of all test case data related to the specified level has been performed, causes of defects that have occurred have been input, or defects that have occurred have been resolved. In the selected test suite, tests have been performed for a preset number of test case data, causes have been entered for any defects that have occurred, or defects that have occurred have been resolved. - Tests have been performed for a preset number of test case data, causes of defects that have occurred have been entered, or defects that have occurred have been resolved - A certain amount of time has elapsed during the pre-set test period.
[0121] If the predetermined requirements for the test case data test are met (Yes in step S22), the analysis unit 2035 moves the process to step S23. If the predetermined requirements for the test case data test are not met (No in step S22), the analysis unit 2035 repeats the processes in steps S21 to S22.
[0122] In step S23, the analysis unit 2035 analyzes the results of the performed tests and extracts test cases that should be given priority from the remaining test cases. In the example shown in Fig. 4, when tests are performed on test cases included in a test suite that was set to priority 1 during planning, the analysis unit 2035 analyzes the results of the performed tests and extracts test cases that should be given priority from the remaining test cases. Note that in Fig. 4, the performance of tests on test cases included in a test suite that was set to priority 1 during planning may be rephrased as the performance of tests on a preset number of test case data.
[0123] FIG. 13 is a flowchart showing a specific example of the operation of the analysis unit 2035. First, the analysis unit 2035 reads out the person in charge, module, viewpoint, factor, level, defect cause, and performance for the test case data for which a test has already been performed from the test case DB 281 (step S231).
[0124] The analysis unit 2035 selects test cases to be preferentially tested from the already tested test cases based on the implementation result (step S232). Specifically, for example, the analysis unit 2035 selects test cases to be preferentially tested in consideration of a numerical value related to the number of occurrences of defects (number of defects, defect rate, etc.), a numerical value related to the labor required to repair the occurred defects, a project process from which the cause of the occurred defect originates, or a combination of at least two of these.
[0125] More specifically, for example, when there is a test case group in which a numerical value relating to the number of occurrences of defects or a numerical value relating to the labor required to repair the occurred defects is higher than a limit value requiring special measures, the analysis unit 2035 selects the test case group in which the numerical value is calculated. In this embodiment, a test case group refers to a group of test cases that are treated as test cases of similar nature, and is expressed by grouping, for example, by factor or level. When the numerical value relating to the number of occurrences of defects and the numerical value relating to the labor required to repair the occurred defects do not exceed the limit value, the analysis unit 2035 selects, for example, a test case group in which a defect occurred due to a factor originating from an upstream process of the project.
[0126] 4, the analysis unit 2035 selects a test case group related to factor A in which factors of "omission of requirements" and "considering impact" originating from an upstream process have been confirmed, assuming that there is no test case group in which the numerical value related to the number of occurrences of defects and the numerical value related to the labor required to fix the occurred defects exceed the limit value. In addition, the analysis unit 2035 selects a test case group related to factor B in which factors of "operational error" originating from a downstream process have been confirmed, as the test case group to be tested with the next highest priority.
[0127] Note that the selection of the test case group may be performed in units of other elements that define the test case group. For example, the analysis unit 2035 may select the test case group based on a viewpoint or a level.
[0128] The analysis unit 2035 identifies the characteristics of the selected test cases (step S233). For example, the analysis unit 2035 identifies elements that define each of the selected test cases, such as a screen, a function, a person in charge, a module, a viewpoint, a factor, a level, and the like.
[0129] 4, the analysis unit 2035 identifies "screen A, function A, person in charge AAAA, module A, viewpoint A, factor A, level AA to level AC" as the characteristics of a test case group in which a defect due to a factor originating from an upstream process has occurred. Also, the analysis unit 2035 identifies "screen A, function A, person in charge AAAA, module A, viewpoint B, factor B, level BA to level BC" as the characteristics of a test case group in which a defect due to a factor originating from a downstream process has occurred.
[0130] The analysis unit 2035 extracts test cases that match the identified characteristics or similar test cases from the untested test cases (step S234). A matching test case refers to a test case in which all elements that define the test case, such as "screen, function, person in charge, module, viewpoint, factor, level," match the identified characteristics of the executed test case. A similar test case refers to a test case in which at least one of the elements that define the test case, such as "screen, function, person in charge, module, viewpoint, factor, level," matches the identified characteristics of the executed test case. The analysis unit 2035 may vary the degree of similarity depending on the matching elements. For example, if the "factors, levels" match, the analysis unit 2035 determines that the contents of the test cases are close and evaluates that the degree of similarity is high. On the other hand, if the viewpoints, modules, or persons in charge match, the analysis unit 2035 evaluates that although there is a relationship, the degree of similarity is lower than when the "factors, levels" match.
[0131] In the example shown in FIG. 4, the analysis unit 2035 extracts test cases with matching "factors and levels" from untested test cases as test case groups with high similarity to the identified features of "screen A, function A, AAAA, module A, viewpoint A, factor A, levels AA to AC." That is, the analysis unit 2035 extracts test cases of "screen B, function C, AAAA, module A, viewpoint D, factor A, levels AA to AC" from untested test cases as similar test case groups.
[0132] In addition, the analysis unit 2035 extracts a test case group of "Screen B, Function B, BBBB, Module C, Viewpoint A, Factor B, Level BA to Level BC" and a test case group of "Screen A, Function A, AAAA, Module B, Viewpoint C, Factor C, Level CA to Level CC" as test case groups that have a low similarity to the characteristics of the identified "Screen A, Function A, AAAA, Module A, Viewpoint A, Factor A, Level AA to Level AC."
[0133] In addition, the analysis unit 2035 extracts test cases with matching "factors and levels" from untested test cases as test cases with high similarity to the identified features of "Screen A, Function A, AAAA, Module B, Viewpoint B, Factor B, Levels BA to BC." In other words, the analysis unit 2035 extracts test cases of "Screen B, Function B, BBBB, Module C, Viewpoint A, Factor B, Levels BA to BC" from untested test cases as similar test cases.
[0134] In addition, the analysis unit 2035 extracts a test case group of "Screen A, Function A, AAAA, Module B, Viewpoint B, Factor B, Level BA to Level BC" that has a low similarity to the characteristics of the identified "Screen A, Function A, AAAA, Module A, Viewpoint A, Factor A, Level AA to Level AC" and a test case group of "Screen A, Function A, AAAA, Module B, Viewpoint C, Factor C, Level CA to Level CC."
[0135] The analysis unit 2035 may make a judgment of similarity based on a functional specification associated with the test case or test suite. A functional specification is data describing specifications for functions possessed by an object to be inspected. A test design document that specifies a test case is created with reference to the functional specification. Information on the functional specification referred to when creating a test case or test suite is associated with the test case or test suite. When a functional specification is associated with the test case selected in step S232, the analysis unit 2035 may extract test cases associated with the same functional specification from untested test cases as similar test cases.
[0136] Returning to the description of FIG. 12, the setting unit 2036 refers to the test cases extracted in step S23 and sets priorities for the test cases that have not been tested (step S24).
[0137] Specifically, the setting unit 2036 sets a high priority to a test case that matches a test case that should be given priority for testing, or a test case that is highly similar to the test case that should be given priority for testing. More specifically, the setting unit 2036 sets the priority by comprehensively considering the reason why a test should be given priority for testing, the degree of similarity to the selected test case, etc. Reasons why a test should be given priority for testing include various factors such as circumstances specific to the project, the amount of repair work, the degree of defect rate, and bias in the skills of the personnel in charge.
[0138] 4, the setting unit 2036 sets priority 1 to the extracted test case group of "Screen A, Function A, AAAA, Module A, Viewpoint A, Factor A, Levels AA to AC." The setting unit 2036 may also set a priority for each level in this test suite based on the defect cause, defect rate, repair man-hours, etc. of the tests that have been performed. For example, the setting unit 2036 may set priority 1 to the test case group of level AA, priority 2 to the test case group of level AC, and priority 3 to the test case group of level AB.
[0139] Furthermore, the setting unit 2036 sets a priority of 2 to the extracted test case group of "Screen B, Function B, BBBB, Module C, Viewpoint A, Factor B, Levels BA to BC." Furthermore, the setting unit 2036 may set a priority for each level in this test suite based on the defect cause, defect rate, repair labor hours, etc. of the tests that have been performed. For example, the setting unit 2036 may set a priority of 1 to the test case group of level BA, a priority of 2 to the test case group of level BC, and a priority of 3 to the test case group of level BB.
[0140] Moreover, the setting unit 2036 sets a priority level of 3 to "screen A, function A, AAAA, module B, viewpoint C, factor C, levels CA to CC" since the degree of similarity is low.
[0141] Depending on the track record of the selected test case, the cause of the defect, and the degree of similarity of the test case extracted from the untested test cases, the test cases may be set to the same priority.
[0142] The storage control unit 2033 stores the set priority in the test case DB 281 as the corrected priority. 14 to 17 are schematic diagrams showing examples of what appears on the display 141 when priorities are corrected based on the results of tests that have already been performed.
[0143] Fig. 14 is a schematic diagram showing an example of a list of test suites displayed on the display 141. The example shown in Fig. 14 shows the display screen of the display 141 when the priorities are corrected by the setting unit 2036 after the test for the test suite of function A-1 is completed. According to the example shown in Fig. 14, the priority of the test suite of function C, which was priority 4, becomes priority 1, the priority of the test suite of function B, which was priority 3, becomes priority 2, the priority of the test suite of function A-2, which was priority 2, becomes priority 3, and the priority of the test suite of function D, which was priority 5, becomes priority 4.
[0144] Fig. 15 is a schematic diagram showing an example of a list of test case data displayed on the display 141. The example shown in Fig. 15 shows a list display of test case data when a test suite of function C with priority level 1 is selected in Fig. 14. According to the example shown in Fig. 15, the priority level of the test case data that was priority level 4 has been changed to priority level 1.
[0145] Fig. 16 is a schematic diagram showing an example of a list of test suites displayed on the display 141. The example shown in Fig. 16 shows the display screen of the display 141 when the priorities are corrected by the setting unit 2036 after the test for the test suite of function C is completed. According to the example shown in Fig. 16, the priority of the test suite of function B, which was priority 2, becomes priority 1, the priority of the test suite of function A-2, which was priority 3, becomes priority 2, and the priority of the test suite of function D, which was priority 4, becomes priority 3.
[0146] Fig. 17 is a schematic diagram showing an example of a list of test case data displayed on the display 141. The example shown in Fig. 17 shows a list display of test case data when a test suite for function B with priority level 1 is selected in Fig. 16. According to the example shown in Fig. 17, the priority level of the test case data that was priority level 2 has been changed to priority level 1.
[0147] <3.4 Test Trend Prediction> FIG. 18 is a flowchart showing an example of the operation of the server 20 when predicting a test trend.
[0148] When a tester wants to check the test trend, i.e., the trend of the remaining number of test cases, the predicted information of the trend of the number of defects, etc., the tester presses a field object to instruct the display of the test trend. The field object is displayed on the display 141, for example.
[0149] When the field object is pressed, the prediction unit 2037 judges whether the number of test cases that have already been tested satisfies a predetermined requirement (step S25). The predetermined requirement includes, for example, that a number of tests have been performed that allows the trend of the tests to be predicted, and specifically, the following requirements: - Testing of all test case data included in any test suite has been performed, or any defects that have occurred have been resolved. - In any test suite, testing has been performed for all test case data related to a specified viewpoint, or any defects that have occurred have been resolved. - In any test suite, testing has been performed on all test case data related to a specified factor, or any defects that have occurred have been resolved. - In any test suite, testing has been performed for all test case data related to a specified level, or any defects that have occurred have been resolved. In any test suite, testing has been performed for a preset number of test case data, or any defects that have occurred have been resolved. Testing has been performed on a preset number of test case data, or any defects that have occurred have been resolved.
[0150] If the number of already tested test cases satisfies the predetermined requirement (Yes in step S25), the analysis unit 2035 identifies the characteristics of the tested test cases (step S26). For example, the analysis unit 2035 identifies elements that define the tested test cases, such as screens, functions, personnel in charge, modules, viewpoints, factors, and levels. If the number of already tested test cases does not satisfy the predetermined requirement (No in step S25), the analysis unit 2035 repeats the process of step S25 until the number of test cases satisfies the requirement.
[0151] The analysis unit 2035 extracts test cases that match or are similar to the identified characteristics from the untested test cases (step S27). For example, the analysis unit 2035 extracts test cases from the untested test cases whose screens, functions, people in charge, modules, viewpoints, factors, and levels match the screens, functions, people in charge, modules, viewpoints, factors, and levels identified for the executed test cases. The analysis unit 2035 also extracts test cases from the untested test cases whose screens, functions, people in charge, modules, viewpoints, factors, and levels match the screens, functions, people in charge, modules, viewpoints, factors, and levels identified for the executed test cases.
[0152] The prediction unit 2037 calculates various parameters representing the trend of testing for the extracted test cases based on the results of the executed test cases and the cause of the defect (step S28). Specifically, the prediction unit 2037 calculates the number of defects, the defect rate, the labor hours, the repair labor hours, or the post-repair confirmation labor hours for the extracted test case group based on the results of the executed test cases.
[0153] For example, the prediction unit 2037 sets a defect rate for a test case group corresponding to a test case group having a predetermined factor and level among the executed test cases and a test case group having the same factor and level. The prediction unit 2037 may correct the defect rate based on an element other than the factor and level. The prediction unit 2037 multiplies the number of cases in the test case group having the same factor and level by the set defect rate to calculate the number of defects for this test case group. The prediction unit 2037 calculates the man-hours, repair man-hours, or post-repair confirmation man-hours for this test case group based on, for example, the ratio between the number of defects in the corresponding test case group and the number of defects in the test case group having the same factor and level.
[0154] If there is a test case group that has the same factors and levels as a test case group in which a defect occurred due to a factor originating from an upstream process among the untested test cases, the prediction unit 2037 may calculate the number of defects predicted to occur from this test case group so that the later the testing is performed, the greater the number of defects.
[0155] The prediction unit 2037 calculates the transition of the number of defects, the transition of the remaining test cases, the transition of the number of recovered defects, or the transition of the number of recovered cases for the period from the start of the test process based on the number of defects, the man-hours, the repair man-hours, or the post-repair confirmation man-hours for the performed tests and the number of defects, the man-hours, the repair man-hours, or the post-repair confirmation man-hours predicted for the unperformed test cases. At this time, the prediction unit 2037 calculates the transition when the tests are performed in the order according to the set priority.
[0156] The storage control unit 2033 stores the calculated parameters in the test case DB 281 as predicted parameters.
[0157] The presenting unit 2034 presents the test trend to the user based on the calculated parameters (step S29). Specifically, for example, the presenting unit 2034 causes the display 141 to display a graph showing the test trend.
[0158] Fig. 19 is a schematic diagram showing an example of a graph showing a test trend, displayed on the display 141. In the diagram shown in Fig. 19, the horizontal axis shows the test period, and the vertical axis shows the remaining number or the number of occurrences. In Fig. 19, the solid line shows the number of defects, the dashed line shows the number of defects to be fixed, the one-dot chain line shows the remaining number of cases, and the two-dot chain line shows the remaining number of fixes to be confirmed.
[0159] As described above, in this embodiment, the server 20 stores the results of the tests in the test process in the test case DB 281 by the storage control unit 2033. The analysis unit 2035 selects a test case to be preferentially tested from the test cases that have been tested based on the results. The analysis unit 2035 extracts a test case similar to the selected test case from the untested test cases in the test process. The setting unit 2036 sets the priority of the test for the extracted test case to be higher than that of the other untested test cases. This makes it possible for the server 20 to set a test to be preferentially tested from the untested tests based on the results of the tests performed in the test process. In addition, since it is possible to start with a test that has a high probability of causing a defect, it is possible to suppress the chain reaction of new bugs caused by software corrections. In other words, the risk of degradation is reduced.
[0160] In this embodiment, the analysis unit 2035 selects a test case that should be given the next highest priority from among the already-tested test cases. The analysis unit 2035 extracts test cases similar to the selected test case that should be given the next highest priority from among the untested test cases. The setting unit 2036 then sets the priority of the extracted test case to be lower than the priority previously set. This allows the server 20 to re-set the order of the untested tests based on the results of the tests performed in the testing process.
[0161] In addition, in this embodiment, the analysis unit 2035 extracts test cases that match any of the elements that define a tested test case from untested test cases as similar test cases. Specifically, for example, the analysis unit 2035 sets test cases whose factors and levels match those of a tested test case as similar test cases. Also, for example, the analysis unit 2035 sets test cases whose functional specifications referenced when creating a tested test case match those of a tested test case as similar test cases. This enables the server 20 to extract test cases similar to a tested test case from untested test cases with high accuracy.
[0162] In this embodiment, the analysis unit 2035 selects test cases to be tested with priority in response to an instruction from a user. This enables the server 20 to set the priority at the timing desired by the user.
[0163] In this embodiment, the analysis unit 2035 selects a test case to be preferentially tested when a predetermined number of tests have been performed, which enables the server 20 to set the priority without an instruction from the user.
[0164] In this embodiment, the analysis unit 2035 selects the test cases to be tested with priority based on the numerical value related to the number of defects included in the implementation result. This makes it possible to assign personnel who are good at reporting failures to cases with a high defect rate. In addition, since defects are expected to occur in the first half of the test, it becomes possible to efficiently assign developers.
[0165] In this embodiment, the analysis unit 2035 selects test cases that should be given priority for testing based on the numerical value related to the man-hours required to repair the defects that occurred, which is included in the implementation result. This makes it possible to test test cases that are expected to require a large number of man-hours to repair in the first half of the testing process, making it possible to efficiently assign developers. In addition, it is possible to assign a dedicated person to cases that take a long time.
[0166] In this embodiment, the storage control unit 2033 stores the cause of a defect that occurred in a test in the test process in the test case DB 281. The analysis unit 2035 selects a test case that should be preferentially tested based on the cause of the defect that occurred, which is included in the implementation result, specifically, the process of the project from which the cause originates. This makes it possible to preferentially perform tests to detect defects embedded in the upstream process among the requirements definition process, design process, coding process, and test process in software development. This reduces the cost of repairing defects and reduces the risk of degradation. The server 20 also sets the priority using the results of tests performed in the test process in the project. This makes it possible for a tester to grasp the test sequence suitable for the project in real time.
[0167] In this embodiment, the presentation unit 2034 presents the set priority to the user, which enables the tester to conduct the test while recognizing the test cases with high priority.
[0168] In this embodiment, the presentation unit 2034 assigns a priority to each set of test cases (test suite) or each individual test case, and presents the priority to the user. This allows the tester to efficiently access test cases that must be tested with priority.
[0169] In this embodiment, the prediction unit 2037 predicts the test results of the extracted test cases from the test implementation results. The prediction unit 2037 then predicts the trend of testing when the tests of untested test cases are implemented in the order according to the set priority. This allows the tester to visually grasp how the tests will progress in the testing process when the set priority is followed. This allows the tester to intuitively determine whether the set priority is correct or not.
[0170] Therefore, according to the server 20 according to the present embodiment, tests can be performed efficiently and effectively in the testing process.
[0171] <Modification> In the above embodiment, the priority of the functions to be tested is set at the planning stage. However, the priority of the functions to be tested does not have to be set at the planning stage.
[0172] In the above embodiment, the test suites are displayed in the order of priority at the time of planning. However, the order in which the test suites are displayed may be the order of priority after resetting.
[0173] Also, in the above embodiment, an example has been described in which the display control unit 183 displays the priority on the display 141. However, the display control unit 183 does not necessarily need to display the priority. The display control unit 183 may arrange the test suites or test cases in the order of the set priority and display the test suites or test cases. This reduces the burden on the tester, since performing the tests from top to bottom results in the optimal order.
[0174] Although several embodiments of the present disclosure have been described above, these embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and modifications are within the scope of the invention and its equivalents as described in the claims, as well as the scope and spirit of the invention.
[0175] <Additional Notes> The matters described in the above embodiments will be supplemented below.
[0176] (Appendix 1) A program to be executed by a computer having a processor 29 and a memory 25, the program causing the processor to execute the steps of storing information regarding tests performed in a testing process (steps S201, S202), selecting a test case to be tested as a priority from among test cases that have already been tested based on the information regarding the tests (step S232), extracting test cases similar to the selected test case from untested test cases in the testing process (step S234), and setting the priority of testing the extracted test case higher than that of other untested test cases (step S24).
[0177] (Appendix 2) A program described in Appendix 1, in which in a selecting step, a test case that should be tested as the next priority is selected from among test cases that have already been tested, in a extracting step, a test case similar to the selected test case that should be tested as the next priority is extracted from untested test cases, and in a setting step, the priority of testing the extracted test case is set lower than the set priority (Appendix 1).
[0178] (Appendix 3) A program described in (Appendix 1) or (Appendix 2), in which in the extraction step, a test case that matches any of the elements that define a test case that has been tested is extracted as a similar test case from untested test cases.
[0179] (Appendix 4) In the extraction step, a test case having factors and levels that match the factors and levels of a test case that has already been tested is regarded as a similar test case (Appendix 3).
[0180] (Appendix 5) In the extraction step, a test case that has already been tested and a test case whose functional specification referenced when the test case was created match are regarded as similar test cases (Appendix 3).
[0181] (Appendix 6) The program according to any one of (Supplementary Note 1) to (Supplementary Note 5), wherein in the selecting step, a test case to be tested with priority is selected in response to an instruction from a user (Step S21).
[0182] (Appendix 7) The program according to any one of (Supplementary Note 1) to (Supplementary Note 5), wherein in the selecting step, when a predetermined number of tests have been performed, a test case to be preferentially performed is selected (step S22).
[0183] (Appendix 8) A program described in any one of (Appendix 1) to (Appendix 7), wherein in the storing step, the results of the test are stored, and in the selecting step, a test case that should be given priority for testing is selected based on a numerical value related to the number of defects that occur, which is included in the results of the test.
[0184] (Appendix 9) A program described in any one of (Appendix 1) to (Appendix 7), in which, in a storing step, the labor hours required to repair defects that occur during testing in a testing process are stored, and, in a selecting step, a test case that should be given priority for testing is selected based on a numerical value related to the labor hours required to repair the defects that have occurred.
[0185] (Appendix 10) A program described in any one of (Appendix 1) to (Appendix 7), in which, in a storing step, the cause of a defect that occurred during testing in a test process is stored, and, in a selecting step, a test case that should be tested as a priority is selected based on the cause of the defect that occurred.
[0186] (Appendix 11) A program according to claim 10, wherein in the selection step, a test case that should be tested as a priority is selected based on the project process from which the cause originates.
[0187] (Appendix 12) The program according to any one of (Supplementary Note 1) to (Supplementary Note 11), which causes a processor to execute a step of presenting the set priority to a user (presentation unit 2034).
[0188] (Appendix 13) The program according to claim 12, wherein in the presenting step, a priority is assigned to each set of test cases or each individual test case, and the priority is presented to the user (Supplementary Note 12).
[0189] (Appendix 14) A program described in any one of (Appendix 1) to (Appendix 13) that causes a processor to execute a step (step S28) of predicting the test results of the extracted test cases from the test execution results, and a step (step S28) of predicting the test trends when tests of untested test cases are executed in an order according to a set priority.
[0190] (Appendix 15) A method executed by a computer having a processor 29 and a memory 25, the method comprising the steps of: storing information regarding tests performed in a testing process; selecting a test case to be tested as a priority from among test cases that have already been tested based on the information regarding the test; extracting test cases similar to the selected test case from untested test cases in the testing process; and setting the priority of testing the extracted test case higher than that of other untested test cases.
[0191] (Appendix 16) An information processing device 20 having a processor 29 and a memory 25, wherein the processor executes the steps of: storing information regarding tests performed in a testing process (storage control unit 2033); selecting a test case that should be tested as a priority from among test cases that have already been tested based on the information regarding the test; extracting test cases similar to the selected test case from untested test cases in the testing process (analysis unit 2035); and setting the priority of testing the extracted test case higher than other untested test cases (setting unit 2036).
[0192] (Appendix 17) A system comprising: a means (memory control unit 2033) for storing information regarding tests performed in a testing process; a step of selecting a test case that should be tested as a priority from among test cases that have already been tested based on the information regarding the test; a means (analysis unit 2035) for extracting test cases similar to the selected test case from untested test cases in the testing process; and a means (setting unit 2036) for setting the priority of testing the extracted test case higher than that of other untested test cases. [Explanation of symbols]
[0193] 1. System 10...Terminal device 121…Communications Department 13...Input device 131…Mouse 132…Keyboard 133…Touch-sensitive devices 14...Output device 141…Display 15…Memory 16…Storage 170...Storage section 180...Control unit 181...Operation reception section 182...Transmitter / receiver 183...Display control unit 19…Processor 20…Server 201…Communications Department 202...Storage section 203...Control unit 2031…Operation reception section 2032…Transmitter / receiver 2033…Memory control unit 2034…Presentation part 2035…Analysis Department 2036…Settings 2037…Prediction Department 25…Memory 26…Storage 29…Processor 80…Network
Claims
1. A program for causing a computer having a processor and a memory to execute the program, the program causing the processor to: storing information about tests performed in the testing process; selecting a test case to be tested with priority from among the test cases that have already been tested based on the information related to the test; In the testing step, a step of extracting a test case similar to the selected test case from untested test cases; setting a priority of testing the extracted test case higher than other untested test cases; A program that executes the following.
2. In the selecting step, a test case to be tested next is selected from the test cases that have been tested; In the extracting step, a test case similar to the selected test case to be tested with the next priority is extracted from the untested test cases; 2. The program according to claim 1, wherein in said setting step, a priority for executing a test for said extracted test case is set lower than said set priority.
3. 3. The program according to claim 1, wherein in the extracting step, a test case that matches any of the elements that define the executed test case is extracted as the similar test case from the untested test cases.
4. 4. The program according to claim 3, wherein in said extracting step, a test case having factors and levels matching those of said test case that has already been tested is taken as said similar test case.
5. 4. The program according to claim 3, wherein in the extracting step, a test case that has been subjected to testing and a functional specification that was referred to when creating the test case are regarded as the similar test case.
6. 6. The program according to claim 1, wherein in the selecting step, the test case to be preferentially tested is selected in response to an instruction from a user.
7. 6. The program according to claim 1, wherein in the selecting step, when a predetermined number of tests have been performed, the test case to be preferentially tested is selected.
8. In the step of storing, a result of the test is stored; 8. The program according to claim 1, wherein in the selecting step, the test cases to be tested with priority are selected based on a numerical value related to the number of occurrences of defects included in the implementation results.
9. In the storing step, the number of steps required to repair the defect occurring in the test in the test process is stored; 8. The program according to claim 1, wherein in the selecting step, the test cases to be preferentially tested are selected based on a numerical value related to man-hours required to repair the defect that has occurred.
10. In the storing step, a cause of a defect occurring in a test in the test process is stored; 8. The program according to claim 1, wherein in the selecting step, the test case to be preferentially tested is selected based on a cause of the defect that has occurred.
11. 11. The program according to claim 10, wherein in said selecting step, said test cases to be tested with priority are selected based on a project process from which said cause originates.
12. 12. The program according to claim 1, further comprising a step of causing the processor to execute a step of presenting the set priority to a user.
13. 13. The program according to claim 12, wherein in the presenting step, the priority is assigned to each set of test cases or each single test case, and the priority is presented to the user.
14. predicting a test result of the extracted test case from an execution result of the test; predicting a trend of tests when the tests of the untested test cases are performed in the order according to the set priority; The program according to any one of claims 1 to 13, which causes the processor to execute the steps:
15. A method implemented by a computer having a processor and a memory, comprising: storing information about tests performed in the testing process; selecting a test case to be tested with priority from among the test cases that have already been tested based on the information related to the test; In the testing step, a step of extracting a test case similar to the selected test case from untested test cases; setting a priority of testing the extracted test case higher than other untested test cases; How to do it.
16. An information processing device including a processor and a memory, The processor, storing information about tests performed in the testing process; selecting a test case to be tested with priority from among the test cases that have already been tested based on the information related to the test; In the testing step, a step of extracting a test case similar to the selected test case from untested test cases; setting a priority of testing the extracted test case higher than other untested test cases; An information processing device that executes the above.
17. A means for storing information regarding tests performed in a test process; A means for selecting a test case to be tested with priority from among the test cases that have already been tested based on the information related to the test; a means for extracting test cases similar to the selected test case from untested test cases in the testing step; a means for setting a priority of testing the extracted test case higher than other untested test cases; A system comprising:
Citation Information
Patent Citations
Testing item extraction system, testing item extraction device, testing item extraction method used for the device, and program therefor
JP2005332098A
Test case selection device and program of software system
JP2007334630A
Test method for integrated system
JP2014241066A
Program verification method
JP2015169981A
Hardware test device and hardware test method
JP2018160191A
