Program, control device, and control method for controlling the order of execution of multiple test sequences

Abstract

Control the order in which multiple test sequences are executed. [Solution] A control device (101) controls the order of execution of a plurality of test sequences. A detection unit (111) detects that a certain test has failed during execution of a certain test sequence. If a defect candidate has not yet been set, a setting unit (112) sets a functional unit used in the failed test as a defect candidate, and if a defect candidate has already been set, a narrowing down unit (113) narrows down the defect candidates to the functional unit used in the failed test. If there is one functional unit that is a defect candidate, an output unit (114) outputs a message indicating that the one functional unit has a defect. If there are multiple functional units that are defect candidates, a selection unit (115) selects, as a test sequence to be executed next, a test sequence in which a functional unit used in a test defined by the test sequence narrows down the multiple functional units that are defect candidates.

Description

[Technical field]

[0001] The present invention relates to a program, a control device, and a control method for controlling the order in which a plurality of test sequences are executed. [Background technology]

[0002] A device that provides various services, such as a web service, is realized by executing a program associated with the service, and in providing the service, a large number of tests are periodically and intermittently executed to confirm that there are no defects. If all tests are successfully executed, it means that no defects were found, but if any test fails, it means that there is a defect in one of the functional units executed by that test.

[0003] Some tests have dependencies on the order in which they are executed. Therefore, one or more tests arranged in the order in which they are executed according to their dependencies is called a test sequence. In other words, a test sequence determines the order in which one or more tests are executed.

[0004] In a situation where S test sequences 1, 2, ..., S are prepared as one test mission, the test mission is executed periodically and intermittently, for example, by batch processing. In order to make it possible to arbitrarily change the order in which the test sequences 1, 2, ..., S are executed within the test mission, tests that have dependencies on each other in the execution order are grouped together into one test sequence, as described above.

[0005] Here, let us assume that the number of tests whose execution order is determined by each test sequence s∈{1, 2, ..., S} is N(s). The N(s) tests whose execution order is determined by the test sequence s are denoted as v(s,1), v(s,2), ..., v(s,N(s)).

[0006] A test v(s,t) executed t-th in a test sequence s checks for defects in one or more functional units by using the functional units in its execution. Here, one of the functional units may be selected from the following as appropriate: (1) One piece of middleware. The middleware consists of one or more modules / packages. (2) A module / package, which consists of one or more source files. (3) One source file, which consists of one or more functions. (4) A function, which references one or more resource files. (5) One resource file. (6) A combination of these. For example, a certain piece of middleware A may be a single functional unit as a whole, while another piece of middleware B may be a functional unit consisting of multiple source files.

[0007] Below, the set of functional units used in test v(s,t) is defined as m(s,t) = {m s,t,1 , m s,t,2 , ..., m s,t,M(s,t)} is used.

[0008] Patent Document 1 discloses a technology that extracts attributes of the deliverables that were the subject of the changed test during a period when the test result changed from success to failure or from failure to success, and narrows down related candidates when associating the defect with other deliverables that are related to the defect. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] JP 2015-69615 A Summary of the Invention [Problem to be solved by the invention]

[0010] When a test v(s,t) that is defined as the tth test in a test sequence s is executed and fails, the M(s,t) functional units m(s,t) included in the set m(s,t) are s,t,1 , m s,t,2 , ..., m s,t,M(s,t) It is thought that there is a problem with one of them.

[0011] Here, if M(s,t)=1, that is, the number of elements in the set m(s,t) is 1, then the defective functional unit is m s,t,1 It is confirmed that this is the case.

[0012] If M(s,t)>1, that is, the number of elements in the set m(s,t) is more than one, then #m(s,t) functional units m s,t,1 , m s,t,2 , ..., m s,t,M(s,t) There is a demand for narrowing down as quickly as possible which of the above is defective by controlling the order in which test sequences are executed.

[0013] The present invention has been made to solve the above-mentioned problems, and has an object to provide a program, a control device, and a control method for controlling the order in which a plurality of test sequences are executed. [Means for solving the problem]

[0014] In order to achieve the above object, a control device according to the present invention is realized by executing a program for controlling an order of executing a plurality of test sequences on a computer, each test sequence of the plurality of test sequences defines an order for performing one or more tests; one or more functional units are utilized in performing each test of the one or more tests; The control device includes: Detecting that a test has failed during the execution of a test sequence; If it is detected that the test has failed and a defect candidate has not yet been set, a functional unit used in the failed test is set as the defect candidate; If it is detected that the test has failed and the defect candidate has been set, narrowing down the defect candidate functional units to the functional units used in the failed test; If there is one functional unit that is a defect candidate, outputting a message indicating that the one functional unit has a defect; If there are a plurality of functional units that are defect candidates, a selection unit selects a test sequence to be executed next from among the plurality of test sequences that have not been executed, and selects the test sequence to be executed next such that a functional unit used in any test determined by the test sequence to be executed next narrows down the plurality of functional units that are defect candidates. Configure it as follows. Effect of the Invention

[0015] According to the present invention, it is possible to provide a program, a control device, and a control method for controlling the order in which a plurality of test sequences are executed. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is an explanatory diagram showing a schematic configuration of a control device according to an embodiment of the present invention. [Diagram 2] 4 is a flowchart showing a flow of a control process executed by a control device according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] The following describes an embodiment of the present invention. Note that the embodiment is for illustrative purposes only and does not limit the scope of the present invention. Therefore, a person skilled in the art can adopt an embodiment in which each or all of the elements are replaced with an equivalent, and such an embodiment is also included in the scope of the present invention.

[0018] (How the control device is realized) The control device according to the present embodiment is generally realized by causing a computer to execute a control program. In this case, the computer may execute a service program for implementing the service to be tested by itself, or may execute the service program on another computer.

[0019] For example, consider the case where the service is a web service. A server computer that executes a web service program to provide the web service may itself execute a test program and function as a control device.

[0020] Also, a test program may be executed on a client computer that is capable of receiving the web service from the server computer, and the client computer may function as a control device.

[0021] As the client computer, in addition to a general computer, a smartphone or a tablet computer can also be used.

[0022] Furthermore, the control device can be realized by executing the processing using a dedicated electronic circuit.

[0023] In addition, as an intermediate form between a computer and a dedicated electronic circuit, it is also possible to configure the control device of this embodiment by applying a technology such as FPGA (Field Programmable Gate Array), which compiles a test program into a design script for an electronic circuit and dynamically configures the electronic circuit based on the design script.

[0024] This test program can be recorded on a computer-readable non-transitory information recording medium such as a compact disk, a flexible disk, a hard disk, a magneto-optical disk, a digital video disk, a magnetic tape, a ROM (Read Only Memory), an EEPROM (Electrically Erasable Programmable ROM), a flash memory, a semiconductor memory, etc. This information recording medium can also be distributed and sold independently of a computer.

[0025] In a computer, a program recorded on a non-transitory information recording medium such as a flash memory, a hard disk, or an SSD (Solid State Drive) is read into a temporary storage device such as RAM (Random Access Memory), and then the CPU (Central Processing Unit) executes the instructions contained in the read program.

[0026] However, in an architecture that allows ROM and RAM to be mapped into a single memory space for execution, the CPU directly reads and executes the instructions contained in the program stored in ROM.

[0027] Furthermore, the test program can be distributed to various computers via a transitory transmission medium such as a computer communication network, independently of the computer on which the test program is executed, and executed in different environments.

[0028] (Test example) The relationship between test sequences, tests, and functional units will be explained again below. Tables 1 to 6 are tables showing examples of test sequences for web services.

[0029] [Table 1]

[0030] [Table 2]

[0031] [Table 3]

[0032] [Table 4]

[0033] [Table 5]

[0034] [Table 6]

[0035] In this example, six test sequences are shown, and tests are performed on eight functional units: "Apache", "PHP", "JavaScript in CDN", "DB", "Internal API", "3rd Party payment API", "3rd Party Login page", and "Search Engine". Therefore, S=6.

[0036] Where: In test sequence 1 “Purchase Item” according to Table 1, six tests corresponding to steps 1-6 are executed in sequence. Therefore, N(1)=6. In test sequence 2 "My Page" according to Table 2, three tests corresponding to steps 1-3 are executed in sequence. Therefore, N(2)=3. In test sequence 3 “Change Payment” according to Table 3, three tests corresponding to steps 1-3 are executed in sequence. Therefore, N(3)=3. In the test sequence 4 “Open Native App and Login” according to Table 4, two tests corresponding to steps 1-2 are executed in sequence. Therefore, N(4)=3. In test sequence 5 “Favorite” according to Table 5, three tests corresponding to steps 1-3 are executed in sequence. Therefore, N(5)=3. In test sequence 6 "Campaign" according to Table 6, one test corresponding to step 1 is executed in sequence. Therefore, N(6)=1.

[0037] The functional unit used in each test in each table is indicated by a "√" symbol (check mark). For example, The set of functional units “Apache”, “PHP”, “JavaScript in CDN”, “DB”, and “Internal API” used in test v(1,4) in step 4 “Choose Item” of test sequence 1 “Purchase Item” is m(1,4) = {1, 2, 3, 4, 5}. The set of functional units “Apache” and “PHP” used in test v(2,1) in step 1 “Go to Top MyPage” of test sequence 2 “My Page” is m(2,1) = {1, 2}. The set of functional units “3rd Party Login page” used in test v(3,2) in step 2 “Login” of test sequence 3 “Change Payment” is m(3,2) = {7}. The set of functional units “Internal API” and “3rd Party Login page” used in test v(4,2) in step 2 “Login” of test sequence 4 “Open Native App and Login” is m(3,2) = {5, 7}. The set of functional units “Internal API”, “3rd Party Login page”, “JavaScript in CDN”, and “Search Engine” used in test v(5,2) in step 2 “Login” of test sequence 5 “Favorite” is m(3,2) = {1, 2, 3, 8}. The set of functional units “Internal API” and “JavaScript in CDN” used in test v(6,1) in step 1 “Go Campaign” of test sequence 6 “Campaign Page” is m(3,2) = {1, 3}.

[0038] When the control device according to this embodiment is applied to the above example, the control device controls the order of execution of six test sequences. If all tests related to all test sequences are successful, there is no problem, but if a test in a test sequence fails, the control device narrows down which functional unit is defective as early as possible and outputs one defective functional unit.

[0039] For example, when test sequence 3 "Change Payment" is executed, the functional unit used by test v(3,1) related to step 1 "Go to Payment" is m(3,1) = {1, 2}, and if this test fails, it is considered that there is a defect in either functional unit 1 "Apache" or functional unit 2 "PHP". In other words, the defect candidate at this point is F = {1, 2}.

[0040] So, to narrow this down, Tests that use "Apache" but not "PHP", or Testing without using "Apache" but using "PHP" It will be sufficient to execute one of the following.

[0041] In the above example, the only test that uses "Apache" but not "PHP" is test v(6,1) in test sequence 6. Therefore, the control device selects test sequence 6, which defines test v(6,1), as the next test to be executed.

[0042] If the selected test sequence 6 is executed and test v(6,1) fails, the current fault candidates F are narrowed down by the set m(6,1) of functional units used in test v(6,1). That is, F∩m(6,1) = {1, 2}∩{1} = {1} As shown above, you can narrow down your search using the set operation “∩”.

[0043] On the other hand, if test v(6,1) does not fail, the defect candidate F cannot be narrowed down any further in the test sequences 1 to 6 in this example. Therefore, the defect candidate at that point in time is output along with a message that the narrowing down could not be done any further. After this, any unexecuted test sequences may not be executed, or any unexecuted test sequences may be executed in order to search for defects in functional units other than the output defect candidates.

[0044] The basic function of the control device according to the present embodiment for narrowing down defect candidates has been described above. In the following, the control device according to the present embodiment will be described in more detail.

[0045] The above explanation is provided to facilitate understanding of the present embodiment, and the number of test sequences S used in actual operation is usually much larger. Also, the number of tests N(s) executed in a certain test sequence s can be changed arbitrarily.

[0046] (Overview of the control device) 1 is an explanatory diagram showing a schematic configuration of a control device according to an embodiment of the present invention. The following description will be given with reference to this diagram.

[0047] The control device 101 according to this embodiment executes a program to control the order in which a plurality of test sequences are executed.

[0048] As noted above, each test sequence of the multiple test sequences defines an order for executing one or more tests.

[0049] Additionally, one or more functional units are utilized in the execution of each test of the one or more tests.

[0050] Here, the control device 101 includes a detection unit 111, a setting unit 112, a narrowing down unit 113, an output unit 114, and a selection unit 115. In addition, the control device 101 may further include an extraction unit 116 as an optional element.

[0051] The detection unit 111 detects that a certain test has failed during the execution of a certain test sequence.

[0052] Following the above-described example, it is assumed that while a plurality of test sequences are being executed in sequence, the detection unit 111 detects that test v(3,1) of test sequence 3 has failed.

[0053] Then, if it is detected that the test has failed and the defect candidate F has not yet been set, the setting unit 112 sets the functional unit used in the failed test as the defect candidate F.

[0054] In the above example, since no test has failed so far, the defect candidate F is an empty set (φ), which corresponds to the defect candidate F not being set yet. Therefore, the setting unit 112 sets the set m(3,2}={1,2} of functional units as the defect candidate F.

[0055] Now, if there are multiple functional units that are defect candidates, the selection unit 115 selects the next test sequence to be executed from the unexecuted test sequences among the multiple test sequences, such that a functional unit used in any of the tests defined by the next test sequence to be executed narrows down the multiple functional units that are defect candidates.

[0056] In the above example, since the defect candidates F={1, 2}, the number of functional units that are defect candidates is #F=2. The selection unit 115 selects A test v(s,t) that has not yet been executed, The functional unit m(s,t) used by the test v(s,t) narrows down the defect candidates F. Then, select a test sequence s such that s is the next one to be executed.

[0057] Where: F∩m(s,t)⊂F and F∩m(s,t)≠F and F∩m(s,t)≠φ That is, 0 < #〔F∩m(s,t)〕< #F If the above condition is satisfied, the functional unit m(s,t) narrows down the defect candidates F.

[0058] In the above illustrative example, only test v(6,1) can narrow down the fault candidates F={1,2}, so test sequence 6 is selected to be executed next.

[0059] Then, if it is detected that the test has failed and defect candidates have already been set, the narrowing-down unit 113 narrows down the functional units that are defect candidates to the functional units used in the failed test.

[0060] In the above described example, test sequence 6 selected as the next one to be executed is executed, and the fact that test v(6,1) has failed is detected by detection unit 111.

[0061] At this point, since the defect candidate F≠φ, the narrowing-down unit 113 calculates the defect candidate F as follows: F ← F∩m(6,1) In this example, before the execution of test v(6,1), F={1,2}, so the failure of test v(6,1) results in F ← {1,2} ∩ {1} = {1} Therefore, the defect candidate F={1}, and the number of functional units contained therein, #F, is 1.

[0062] Finally, if there is one functional unit that is a defect candidate, the output section 114 outputs a message indicating that the one functional unit has a defect.

[0063] In the above example, defect candidate F={1}, and there is one functional unit that is a defect candidate, so output section 114 outputs a message indicating that functional unit 1 has a defect.

[0064] As described above, the defect candidate F corresponds to a set of IDs representing functional units. Therefore, in this embodiment, a candidate set representing the defect candidate F is stored in a storage area 131 such as a RAM or SSD of a computer. For ease of understanding, the symbol "F" used for the defect candidate F is also used for the candidate set.

[0065] If the candidate set F is stored in a storage area 131 such as a RAM or SSD using a data representation such as an array, a bit array, an associative array, a vector, a queue, or a stack, it becomes possible to set or narrow down the elements of the candidate set F.

[0066] As described above, if it is detected that the test v(s,t) has failed and the candidate set F is an empty set φ, the setting unit 112 F ← m(s,t) By this, the functional unit (the element of m(s,t)) used in the failed test v(s,t) is added to the candidate set F.

[0067] Also, if it is detected that the test v(s,t) has failed and the candidate set F is not an empty set φ, the narrowing unit 113 F ← F∩m(s,t) From the candidate set F, remove functional units that are not utilized in any failed test v(s,t).

[0068] If the candidate set F has one element, the output section 114 outputs a message indicating that the functional unit that is the one element has a defect.

[0069] If all of the multiple test sequences are executed and there are multiple functional units that are defect candidates, output section 114 outputs a message indicating that at least one of the multiple functional units that are defect candidates has a defect.

[0070] In the above explanatory example, when execution of all test sequences has been completed, if the number of elements in candidate set F is #F≧2, each element of candidate set F is output and a report is made that one or more functional units have a defect, but that it has not been possible to narrow it down to "one functional unit that is definitely defective."

[0071] Also, instead of or in addition to "when all test sequences have been executed," a restriction based on the elapsed time from the start of execution of the test sequences may be imposed. For example, in a mode in which the operation status of a service is monitored by executing test sequences 1, 2, ..., S every day, if some kind of malfunction occurs, the time required to execute all test sequences tends to be longer than when there is no malfunction, due to reasons such as the need to wait for timeouts in various processes.

[0072] Therefore, if the total time required to execute the executed test sequences among the multiple test sequences exceeds a predetermined time and there are multiple functional units that are defect candidates, the output section 114 may output a message indicating that at least one of the multiple functional units that are defect candidates has a defect.

[0073] This "predetermined time" may be, for example, the average of the total time required to execute all test sequences when no defect is found in all test sequences, or it may be determined according to the frequency with which all test sequences are repeatedly executed. For example, if all test sequences are executed every day and the required time is approximately H hours, then the predetermined time may be k×H hours, which is a constant k times that time.

[0074] If all of the test sequences are executed without detecting any failure, the output section 114 outputs a message indicating that no malfunction was found in any of the one or more functional units.

[0075] In this manner, in this embodiment, attention is focused on functional units included in candidate set F to narrow down the functional units having a defect, but depending on the functional units used by the combination of multiple failed tests, it may be possible to narrow down the functional units to those not included in candidate set F. Extraction unit 116 extracts such combinations.

[0076] That is, the extraction unit 116 extracts from a plurality of combinations of a failure set having a test element detected as having failed by the detection unit 111, A combination in which one functional unit is used in all tests included in the combination Attempt to extract.

[0077] For example, in a mode in which the history of tests that have been detected as having failed is stored in the memory area 131 as a failure set R, when the number of tests included in the failure set R is #R, the combinations for the failure set can be determined by which of the #R tests are selected and which are not selected. In other words, each combination is associated with a subset of the "set having #R elements." Therefore, the combinations are a total of 2 #R There is a street.

[0078] In other words, the combinations are the elements of the power set Power(R) of the failure set R, and the number of elements of the power set Power(R) is 2#R , and all elements of the power set Power(R) are included in the failure set R.

[0079] So, 2 #R For each combination of the possible combinations, check the functional units used in all the tests included in that combination. That is, W∈Power(R) For a combination W that satisfies (i.e., a subset of the failure set R and an element of the power set Power(R) of the failure set R), X = ∩ (s,t)∈W m(s,t) Thus, a set X of functional units used in all tests included in the combination can be obtained.

[0080] And if the number of functional units used in all tests included in the combination is one, that is, #X = #〔∩ (s,t)∈W m(s,t) = 1 If so, the extraction unit 116 extracts the combination W.

[0081] On the other hand, if a combination is not found in which "the number of functional units used in all tests included in the combination is one", the attempt by the extraction section 116 will fail.

[0082] Then, when the extraction by the extraction unit 116 is successful, the output unit 114 outputs one functional unit related to the extracted combination W, i.e., 〔∩ (s,t)∈W m(s,t) = {u} It outputs a message indicating that one functional unit u contained in has a defect.

[0083] In the following, various methods for selecting a test sequence s that narrows down the candidate set F, a test v(s, t), and a set m(s, t) of functional units used by the test will be described.

[0084] The first method is to select the unexecuted test sequences: 0 < #〔F∩m(s,t)〕< #F In this method, the order of the test sequences 1, 2, ..., S that are periodically executed is not changed, but some of them are skipped to narrow down the defective functional units.

[0085] The second method is to select the unexecuted test sequences. 0 < #〔F∩m(s,t)〕< #F Then, a test sequence s for the selected s is selected. In this method, the execution order of the test sequences executed after a defect is discovered is changed.

[0086] The third method uses the expected time required for each test. In an environment where test sequences 1, 2, ..., S are executed periodically, the expected time e(s,t) required for each test v(s,t) can be estimated by collecting and analyzing the execution history.

[0087] Then, the expected time from when the execution of the test sequence s starts until the test v(s,t) for step t in the test sequence s ends is E(s,t) = Σ i=1 t e(s,i) It can be calculated as follows:

[0088] So, for unexecuted test sequences, 0 < #〔F∩m(s,t)〕< #F Among s that satisfy E(s,t) = Σ i=1 t e(s,i) The test sequence s for s such that is the shortest is selected as the next one to be executed. This method speeds up the narrowing down process by making the narrowed-down tests execute as early as possible.

[0089] The fourth method is to select the test sequence that narrows down the problem as much as possible when a defect is found. That is, for the test sequences that have not been executed, 0 < #〔F∩m(s,t)〕< #F Among all s that satisfy F, the test sequence s for s with the smallest F∩m(s,t) is selected as the next one to be executed. This method speeds up the narrowing down process by maximizing the degree of narrowing down.

[0090] The fifth method is to combine the above methods as appropriate. For example, (1) In the fourth method, if there are multiple cases where #[F∩m(s,t)] has the same minimum value, select the one with the shortest E(s,t) from among the multiple cases using the third method. (2) Randomly select a predetermined number of E(s,t) from the top of the shortest order, or (3) Select the one with the smallest evaluation value calculated based on E(s,t) and #〔F∩m(s,t)〕 (for example, the score calculated by multiplying each by a weight). In addition to any of the above three combinations, various other combinations can be used.

[0091] (Control Processing) 2 is a flowchart showing the flow of control processing executed by the control device according to the embodiment of the present invention. The following description will be given with reference to this figure. Note that each step described below can be appropriately changed or omitted depending on the application.

[0092] When this process starts, the control device 101 initializes a candidate set F to an empty set φ (step S201), and initializes a failure set R to an empty set φ (step S202).

[0093] Then, if there are any unexecuted test sequences among the S test sequences 1, 2, ..., S related to the service to be tested (step S203; Yes), the control device 101 selects one of the unexecuted test sequences s as the next one to be executed in a predetermined order or randomly (step S204).

[0094] The default order is: in a predetermined order, Most frequent failures in the past, The sequences used in the actual operation of the service are assigned to one of the test sequences, and the order of the number of times they were assigned is as follows: Combinations of these Various orders can be adopted.

[0095] Furthermore, the control device 101 executes the selected test sequence s by itself or causes a server computer that provides the service to execute the sequence (step S205).

[0096] If the test sequence s is successful (step S206; successful), the control device 101 advances the process to step S241.

[0097] On the other hand, if it is detected that the test sequence s has failed (step S206; failure), the control device 101 identifies the failed test v(s, t) (step S207).

[0098] If the candidate set F is an empty set φ (step S208; empty set), the control device 101 adds a set m(s, t) of functional units used by the test v(s, t) to the candidate set F. F ← m(s,t) (step S209), and the control proceeds to step S221.

[0099] If the candidate set F is not an empty set, i.e., if the candidate set F has already been set (step S208; set), the control device 101 checks whether the intersection F∩m(s,t) of the candidate set F and the set m(s,t) of functional units used by the test v(s,t) is an empty set φ (step S210).

[0100] If φ is an empty set (step S210;=φ), the control device 101 advances the process to step S211.

[0101] If the candidate set F is not an empty set φ (step S210; ≠ φ), the control device 101 sets the candidate set F as follows: F ← F∩m(s,t) (step S211), and the control proceeds to step S221.

[0102] Now, if the candidate set F has one element, that is, F={u} (step S221; Yes), the control device 101 outputs a message indicating that the functional unit u is defective (step S222), and ends this process.

[0103] On the other hand, if the candidate set F has a plurality of elements, that is, F≠{u} (step S221; No), the control device 101 advances the process to step S231.

[0104] Now, if the common part F∩m(s,t) is an empty set φ (step S209;=φ), it means that a defect has been found in a functional unit that is not included in the candidate set currently being considered.

[0105] Therefore, the control device 101 generates identification information (s, t) for identifying the test v(s, t) in which the defect was found, as follows: R←R∪{(s,t)} Then, the following process is repeated for each element W of the power set Power(R) of the failure set R (step S232). As described above, each element W is a subset of the failure set R (W ⊂ R) and corresponds to a combination of elements extracted from the failure set R.

[0106] First, a set X of functional units used by all tests v(s,t) identified by elements (s,t) of the control device 101 subset V is X←∩ (s,t)∈W m(s,t) (step S233).

[0107] Then, it is checked whether the number of elements in the set X is 1, that is, whether X={u}. If X={u} (step S234; Yes), the control device 101 outputs a message indicating that the functional unit u is defective (step S235), and ends this process.

[0108] If X≠{u} (step S234; No), the control device 101 continues the repetition (step S236). When the repetition ends, the control device 101 advances the process to step S241.

[0109] After the above processes are completed, the control device 101 checks whether the candidate set F has been set or is an empty set (step S241).

[0110] If the candidate set F has already been set (step S241; set), the control device 101 checks whether a new test sequence s remains that can be narrowed down (step S242). As described above, for the test v(s,t) and the set m(s,t) of functional units, m(s,t) is set so that m(s,t) narrows down the candidate set F, that is, 0<#〔F∩m(s,t)〕< #F Using the above five examples or a method similar to them, find the next test sequence s to be executed so that

[0111] If such a new test sequence s is found (step S243: YES), the new test sequence s is selected (step S244), and control proceeds to step S205.

[0112] On the other hand, if a new test sequence s is not found (step S243; None), the control device 101 outputs a message to the effect that there is a defect in one of the elements of the candidate set F, but it has not been possible to identify which one (step S244), and ends this process.

[0113] Also, if F is an empty set (step S241; empty set), the control device 101 advances the process to step S203.

[0114] In addition, if there are no unexecuted test sequences (step S203; None), and if the candidate set F has been set (step S251; Set), the control device 101 outputs the elements of the candidate set F (step S252), as in step S235, and terminates this process.

[0115] On the other hand, if the candidate set F is an empty set φ (step S251; empty set), the control device 101 outputs a message indicating that no defect has been found (step S253), and ends this process.

[0116] In addition, in the output by the output unit 114 according to the present embodiment, If no errors are found, a message to that effect is output. If the defective functional unit can be narrowed down to one, the functional unit is output. If there are multiple defective functional units and it is not possible to narrow them down, output those functional units. However, The history of elements added to the failure set R, i.e., the history of failed tests, History of successful test sequences etc. may also be output.

[0117] In addition, the output unit 114 Information identifying the test sequences and tests used to narrow the candidate set, i.e. F ← m(s,t) or, F ← F ∩ m(s,t) or, X←∩ (s,t)∈W m(s,t) It is also possible to output the history of (s,t) used in

[0118] By examining this information, it is possible to know which test in which test sequence has failed.

[0119] Further, in the above control, no restriction is imposed based on the elapsed time from the start of execution of the test sequence. However, it is possible to impose a restriction based on the elapsed time by adopting a mode in which, for example, immediately before executing step S242, it is checked whether the elapsed time has exceeded a predetermined upper limit, and if so, the processing proceeds to step S251.

[0120] In addition, in a mode in which the extraction unit 116 is omitted, the execution of steps S202 and S231-236 related to the failure set R can be omitted.

[0121] In this manner, according to this embodiment, by controlling the order in which a plurality of test sequences are executed, it becomes possible to narrow down in which functional unit a defect has occurred as early as possible.

[0122] This embodiment is particularly suitable when priority is given to detecting one functional unit that is almost certainly defective as early as possible, rather than detecting all functional units that are likely to be defective.

[0123] (summary) The program, control device, and control method according to the present embodiment described above will be described below.

[0124] (Appendix 1) A program for controlling an order of executing a plurality of test sequences, each test sequence of the plurality of test sequences defines an order for performing one or more tests; one or more functional units are utilized in performing each test of the one or more tests; The program causes a computer to a detection unit for detecting a failure of a test during execution of a test sequence; a setting unit that sets a functional unit used in the failed test as a defect candidate if the test failure is detected and a defect candidate has not yet been set; a narrowing-down unit that narrows down the functional units that are the defect candidates to the functional units used in the failed test if it is detected that the test has failed and the defect candidates have been set; an output unit that outputs, if there is one functional unit that is a defect candidate, a message indicating that the one functional unit has a defect; a selection unit that selects a test sequence to be executed next from among the test sequences that have not been executed yet, if there are a plurality of functional units that are defect candidates, such that a functional unit used in any of the tests determined by the test sequence to be executed next narrows down the plurality of functional units that are defect candidates; A program characterized by causing the program to function as a

[0125] (Appendix 2) A candidate set representing the defect candidates is stored in a memory area of ​​the computer; If it is detected that the test has failed and the candidate set is an empty set, the setting unit adds a functional unit used in the failed test to the candidate set; When it is detected that the test has failed and the candidate set is not an empty set, the narrowing unit removes from the candidate set functional units that are not used in the failed test; If the candidate set has one element, the output unit outputs a message indicating that the functional unit that is the one element has a defect. 2. The program according to claim 1,

[0126] (Appendix 3) When all of the plurality of test sequences are executed and there are a plurality of functional units that are defect candidates, the output unit outputs a message indicating that at least one of the plurality of functional units that are defect candidates has a defect. 3. The program according to claim 1 or 2.

[0127] (Appendix 4) If the total time required to execute the executed test sequence among the plurality of test sequences exceeds a predetermined time and there are a plurality of functional units that are defect candidates, the output unit outputs a message indicating that at least one of the plurality of functional units that are defect candidates has a defect. 4. The program according to any one of claims 1 to 3.

[0128] (Appendix 5) If all of the plurality of test sequences are executed without detecting any failure, the output unit outputs a message indicating that there is no defect in any of the one or more functional units. 5. The program according to claim 1,

[0129] (Appendix 6) Among the unexecuted test sequences, a test sequence that defines a test using a functional unit that narrows down the plurality of functional units that are defect candidates and that has the shortest expected time from the start of execution of the test sequence to the end of the test is selected as the test sequence to be executed next. 6. The program according to any one of claims 1 to 5.

[0130] (Appendix 7) Among the unexecuted test sequences, a test sequence that defines a test that uses a functional unit that has the smallest number of functional units after narrowing down the plurality of functional units that are defect candidates is selected as the test sequence to be executed next. 6. The program according to any one of claims 1 to 5.

[0131] (Appendix 8) From a plurality of combinations of a failure set having elements of tests detected as having failed by the detection unit, A combination in which one functional unit is used in all tests included in the combination An extractor that attempts to extract Further equipped with If the extraction is successful, the output unit outputs a message indicating that one functional unit related to the extracted combination has a defect. 8. The program according to any one of claims 1 to 7.

[0132] (Appendix 9) A control device for controlling an order of executing a plurality of test sequences, each test sequence of the plurality of test sequences defines an order for performing one or more tests; one or more functional units are utilized in performing each test of the one or more tests; The control device includes: a detection unit for detecting a failure of a test during execution of a test sequence; a setting unit that sets a functional unit used in the failed test as a defect candidate if the test failure is detected and a defect candidate has not yet been set; a narrowing-down unit that narrows down the functional units that are the defect candidates to the functional units used in the failed test if it is detected that the test has failed and the defect candidates have been set; an output unit that outputs, if there is one functional unit that is a defect candidate, a message indicating that the one functional unit has a defect; a selection unit that selects a test sequence to be executed next from among the test sequences that have not been executed yet, if there are a plurality of functional units that are defect candidates, such that a functional unit used in a test determined by the test sequence to be executed next narrows down the plurality of functional units that are defect candidates; A control device comprising:

[0133] (Appendix 10) A control method executed by a control device that controls an order of executing a plurality of test sequences, comprising: each test sequence of the plurality of test sequences defines an order for performing one or more tests; one or more functional units are utilized in performing each test of the one or more tests; a detection step in which the control device detects that a certain test has failed during execution of a certain test sequence; a setting step of setting a functional unit used in the failed test as a failure candidate by the control device when it is detected that the test has failed and a failure candidate has not yet been set; a narrowing-down process in which, if it is detected that the test has failed and the defect candidates have been set, the control device narrows down the functional units that are defect candidates to the functional units used in the failed test; an output step in which, if the number of functional units that is the defect candidate is one, the control device outputs a message indicating that the one functional unit is defective; a selection step in which, if there are a plurality of functional units that are defect candidates, the control device selects a test sequence to be executed next from among the plurality of test sequences that have not been executed, the selection step being such that a functional unit used in any one of the tests determined by the test sequence to be executed next narrows down the plurality of functional units that are defect candidates; A control method comprising:

[0134] The present invention allows various embodiments and modifications without departing from the broad spirit and scope of the present invention. The above-described embodiments are for the purpose of explaining the present invention and do not limit the scope of the present invention. That is, the scope of the present invention is indicated by the claims, not the embodiments. Various modifications made within the scope of the claims and the scope of the invention equivalent thereto are considered to be within the scope of the present invention. [Industrial Applicability]

[0135] According to the present invention, it is possible to provide a program, a control device, and a control method for controlling the order in which a plurality of test sequences are executed. [Explanation of symbols]

[0136] 101 Control device 111 Detection unit 112 Setting section 113 Narrowing section 114 Output section 115 Selection Section 116 Extraction part

Claims

1. A program for controlling an order of executing a plurality of test sequences, each test sequence of the plurality of test sequences defines an order for performing one or more tests; one or more functional units are utilized in performing each test of the one or more tests; The program causes a computer to a detection unit for detecting a failure of a test during execution of a test sequence; a setting unit that sets a functional unit used in the failed test as a defect candidate if the test failure is detected and a defect candidate has not yet been set; a narrowing-down unit that narrows down the functional units that are the defect candidates to the functional units used in the failed test if it is detected that the test has failed and the defect candidates have been set; an output unit that outputs, if there is one functional unit that is a defect candidate, a message indicating that the one functional unit has a defect; a selection unit that selects a test sequence to be executed next from among the test sequences that have not been executed yet, if there are a plurality of functional units that are defect candidates, such that a functional unit used in any of the tests determined by the test sequence to be executed next narrows down the plurality of functional units that are defect candidates; A program characterized by causing the program to function as a

2. A candidate set representing the defect candidates is stored in a memory area of ​​the computer; If it is detected that the test has failed and the candidate set is an empty set, the setting unit adds a functional unit used in the failed test to the candidate set; When it is detected that the test has failed and the candidate set is not an empty set, the narrowing unit removes from the candidate set functional units that are not used in the failed test; If the candidate set has one element, the output unit outputs a message indicating that the functional unit that is the one element has a defect.

2. The program according to claim 1 .

3. When all of the plurality of test sequences are executed and there are a plurality of functional units that are defect candidates, the output unit outputs a message indicating that at least one of the plurality of functional units that are defect candidates has a defect.

2. The program according to claim 1 .

4. If the total time required to execute the executed test sequence among the plurality of test sequences exceeds a predetermined time and there are a plurality of functional units that are defect candidates, the output unit outputs a message indicating that at least one of the plurality of functional units that are defect candidates has a defect.

2. The program according to claim 1 .

5. If all of the plurality of test sequences are executed without detecting any failure, the output unit outputs a message indicating that there is no defect in any of the one or more functional units.

2. The program according to claim 1 .

6. Among the unexecuted test sequences, a test sequence that defines a test using a functional unit that narrows down the plurality of functional units that are defect candidates and that has the shortest expected time from the start of execution of the test sequence to the end of the test is selected as the test sequence to be executed next.

2. The program according to claim 1 .

7. Among the unexecuted test sequences, a test sequence that defines a test that uses a functional unit that has the smallest number of functional units after narrowing down the plurality of functional units that are defect candidates is selected as the test sequence to be executed next.

2. The program according to claim 1 .

8. From a plurality of combinations of a failure set having elements of tests detected as having failed by the detection unit, A combination in which one functional unit is used in all tests included in the combination An extractor that attempts to extract Further equipped with If the extraction is successful, the output unit outputs a message indicating that one functional unit related to the extracted combination has a defect.

2. The program according to claim 1 .

9. A control device for controlling an order of executing a plurality of test sequences, each test sequence of the plurality of test sequences defines an order for performing one or more tests; one or more functional units are utilized in performing each test of the one or more tests; The control device includes: a detection unit for detecting a failure of a test during execution of a test sequence; a setting unit that sets a functional unit used in the failed test as a defect candidate if the test failure is detected and a defect candidate has not yet been set; a narrowing-down unit that narrows down the functional units that are the defect candidates to the functional units used in the failed test if it is detected that the test has failed and the defect candidates have been set; an output unit that outputs, if there is one functional unit that is a defect candidate, a message indicating that the one functional unit has a defect; a selection unit that selects a test sequence to be executed next from among the test sequences that have not been executed yet, if there are a plurality of functional units that are defect candidates, such that a functional unit used in a test determined by the test sequence to be executed next narrows down the plurality of functional units that are defect candidates; A control device comprising:

10. A control method executed by a control device that controls an order of executing a plurality of test sequences, comprising: each test sequence of the plurality of test sequences defines an order for performing one or more tests; one or more functional units are utilized in performing each test of the one or more tests; a detection step in which the control device detects that a certain test has failed during execution of a certain test sequence; a setting step of setting a functional unit used in the failed test as a failure candidate by the control device when it is detected that the test has failed and a failure candidate has not yet been set; a narrowing-down process in which, if it is detected that the test has failed and the defect candidates have been set, the control device narrows down the functional units that are defect candidates to the functional units used in the failed test; an output step in which, if the number of functional units that is the defect candidate is one, the control device outputs a message indicating that the one functional unit is defective; a selection step in which, if there are a plurality of functional units that are defect candidates, the control device selects a test sequence to be executed next from among the plurality of test sequences that have not been executed, the selection step being such that a functional unit used in any one of the tests determined by the test sequence to be executed next narrows down the plurality of functional units that are defect candidates; A control method comprising:

Images