Analysis device and analysis method

Abstract

To make it possible to identify the cause of a test failure among software to be tested, test data, test script, and test environment.SOLUTION: An analysis device identifies the cause, when a test of software to be tested fails in a test environment which is the environment in which the test is executed. In the test, the execution of the software to be tested and the confirmation of execution results are performed by a test script without human intervention. Test data is data read into the software to be tested and the test script during execution of the test. The analysis device includes a score calculation unit configured to calculate a failure cause score based on the presence or absence of a change for each of the software to be tested, the test data, the test script, and the test environment, and an estimation unit configured to estimate, based on the failure cause score, which of the software to be tested, the test data, the test script, and the test environment is the cause of the test failure.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an analysis device and an analysis method. [Background technology]

[0002] As the importance of ensuring the quality of software increases, tools for finding defects have also been developed. Patent Document 1 discloses an apparatus for generating related candidates for deliverables, which includes a period extraction unit that inputs predetermined management tool data including test results for each deliverable from a test management tool, a configuration management tool, and a defect management tool, for a defect that is a deliverable in a defect management tool, a revision that is a deliverable in a configuration management tool, and a test item that is a deliverable in a test management tool, and extracts a period from when the test result related to the defect changes from failure to success or from success to failure, and a narrowing-down processing unit that uses the relationship of attribute changes between the deliverables in the period extracted by the period extraction unit to narrow down related candidates in the association between the defect and other deliverables related to the defect. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-69615 A Summary of the Invention [Problem to be solved by the invention]

[0004] The invention described in Patent Document 1 does not allow for identifying the cause of a test failure. [Means for solving the problem]

[0005] An analysis device according to a first aspect of the present invention is an analysis device that identifies the cause of a failure of a test of software to be tested in a test environment in which the test is executed, wherein the execution of the software to be tested and the confirmation of the execution results are executed by a test script without human intervention, and test data is data that is read into the software to be tested and the test script when the test is executed, and the analysis device is equipped with a score calculation unit that calculates a failure cause score for each of the software to be tested, the test data, the test script, and the test environment based on whether or not there has been a change, and an estimation unit that estimates whether the cause of the test failure is the software to be tested, the test data, the test script, or the test environment based on the failure cause score. An analysis method according to a second aspect of the present invention is an analysis method in which a computer identifies a cause of a failure of a test of software under test in a test environment in which a test is executed, wherein the execution of the software under test and confirmation of the execution results are executed by a test script without human intervention in the test, and test data is data that is read into the software under test and the test script when the test is executed, the analysis method including: calculating a failure cause score for each of the software under test, the test data, the test script, and the test environment based on the presence or absence of changes; and estimating, based on the failure cause score, whether the cause of the test failure is the software under test, the test data, the test script, or the test environment. Effect of the Invention

[0006] According to the present invention, it is possible to identify whether the cause of a test failure is the software under test, the test data, the test script, or the test environment. [Brief description of the drawings]

[0007] [Figure 1] Functional configuration diagram of an analysis device according to a first embodiment [Diagram 2] Hardware configuration of the analysis device [Diagram 3] A diagram explaining the outline of the tests that the analysis device analyzes. [Figure 4] A diagram showing an example of configuration information [Diagram 5] A diagram showing an example of test case information [Figure 6] FIG. 13 is a diagram showing an example of score calculation rule information. [Figure 7] A diagram showing an example of change history information [Figure 8] A diagram showing an example of test execution information [Figure 9] FIG. 13 is a diagram showing an example of score aggregation information. [Figure 10] FIG. 13 is a diagram showing an example of name correspondence information. [Figure 11] A diagram showing an example of output to an output device. [Figure 12] A flowchart showing the process of the score calculation unit in the first embodiment. [Figure 13] Flowchart for explaining the details of the score calculation process [Figure 14] 1 is a functional block diagram of an analysis device according to a second embodiment of the present invention; [Figure 15] A diagram showing an example of extended test case information [Figure 16] FIG. 13 is a diagram showing an example of score calculation extension rule information. [Figure 17] FIG. 13 is a diagram showing an example of extended score aggregation information. [Figure 18] FIG. 13 is a diagram showing an example of element definition information. [Figure 19] FIG. 13 is a diagram showing an example of element-related information. [Figure 20] A diagram showing an example of the initial value of a conditional expression [Figure 21] A diagram showing an example of detailed configuration information [Figure 22] A diagram showing an example of detailed related information [Figure 23] FIG. 13 is a diagram showing an example of output to an output device by a display generation unit. [Figure 24] Flowchart showing the process of the rule generation unit [Diagram 25]Flowchart showing the process of the rule generation unit [Figure 26] Flowchart showing details of the conditional expression addition process [Figure 27] Flowchart showing details of the conditional expression addition process [Figure 28] 11 is a flowchart showing details of a difference value calculation process. [Figure 29] A flowchart showing the process of the score calculation unit in the second embodiment. [Diagram 30] A flowchart showing the process of the score calculation unit in the second embodiment. [Diagram 31] A flowchart showing the process of the test addition execution unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] --First embodiment-- A first embodiment of an analysis device according to the present invention will be described below with reference to FIGS.

[0009] 1 is a functional configuration diagram of an analysis device 1 that analyzes software test results. The analysis device 1 includes, as its functions, a score calculation unit 151, an estimation unit 152, and a display generation unit 153. The analysis device 1 also includes a storage unit 84. The storage unit 84 stores test case information 101, score calculation rule information 102, change history information 103, test execution information 104, score tally information 105, and name correspondence information 106.

[0010] FIG. 2 is a hardware configuration diagram of the analysis device 1. The analysis device 1 includes a CPU 81, which is a central processing unit, a ROM 82, which is a read-only storage device, a RAM 83, which is a readable / writable storage device, a storage unit 84, and an input / output interface 85. The analysis device 1 is connected to an input device 2 and an output device 3 via the input / output interface 85. The input device 2 is a keyboard or a mouse. The output device 3 is a device for displaying images, such as a liquid crystal display. However, a touch panel in which the input device 2 and the output device 3 are integrated may also be used. The input device 2 transmits input from a person using the analysis device (hereinafter, referred to as a "user") to the analysis device 1. The output device 3 presents information output by the analysis device 1 to the user.

[0011] The CPU 81 deploys a program stored in the ROM 82 in the RAM 83 and executes it to realize the score calculation unit 151, the estimation unit 152, and the display generation unit 153. However, instead of the combination of the CPU 81, the ROM 82, and the RAM 83, the score calculation unit 151, the estimation unit 152, and the display generation unit 153 may be realized by a field programmable gate array (FPGA) that is a rewritable logic circuit or an application specific integrated circuit (ASIC) that is an integrated circuit for a specific application. Also, instead of the combination of the CPU 81, the ROM 82, and the RAM 83, the score calculation unit 151, the estimation unit 152, and the display generation unit 153 may be realized by a combination of different configurations, for example, a combination of the CPU 81, the ROM 82, the RAM 83, and the FPGA, instead of the combination of the CPU 81, the ROM 82, and the RAM 83.

[0012] The storage unit 84 is a non-volatile storage device, such as a hard disk drive or a flash memory. Most of the information stored in the storage unit 84 is created in advance, and only the score tally information 105 is created by the score calculation unit 151 and the estimation unit 152. The input / output interface 85 may be a general-purpose communication interface or a dedicated interface that matches the specifications of the input device 2 and the output device 3.

[0013] FIG. 3 is a diagram for explaining an overview of the test analyzed by the analysis device 1. In this embodiment, the test to be analyzed is for software. However, not only the software itself but also conditions associated with the software affect the test, so these are also included in the test items. Specifically, in this embodiment, the test components are the test target software Src, test data Dat, test script Ts, and test environment Env. Although various conditions differ for each test, they have in common that the four components of the test target software Src, test data Dat, test script Ts, and test environment Env are combined.

[0014] In the following, the test target software Src, test data Dat, test script Ts, and test environment Env may be referred to as the "types" of components. These components are conceptual, and concrete ones are used in practice. For example, the test target software Src contains multiple source codes and resources such as icons, which are updated according to the development status and the version numbers are changed.

[0015] The software under test Src is essentially the software to be tested. The software under test Src may consist of one or more source codes. In the example shown in FIG. 2, the software under test Src includes two source codes, but the number of source codes may increase or decrease as development progresses. In addition, the version number of each source code is changed every time the content is changed. In other words, the version of the source code may be different for each test, and the number of source codes may be different. In addition to the source codes, the software under test Src may also include, for example, statically linked libraries.

[0016] The test data Dat is data that is read by the test target software Src and the test script Ts when the test is executed, and is, for example, a combination of variable names and their values. The test data Dat is composed of one or more pieces of data. These pieces of data are prepared according to the test target software Src, and are, for example, text data, image data, audio data, or a combination of these. In the example shown in Figure 2, the test data Dat is composed of one sample data. The version number of each piece of data is changed every time the content is changed.

[0017] A test script Ts is a script file for executing the test target software Src in a test and checking the execution results without human intervention. The test script Ts is, for example, a shell script or a batch file. In the example shown in FIG. 2, the test script Ts is composed of one script. The version number of each script is changed every time the contents are changed. Note that, for the sake of convenience, it is called a test "script" in this embodiment, but it does not have to be a script and may be a program itself.

[0018] The test environment Env is the hardware used for testing and the software used for testing, excluding the test target software Src. In the example shown in Figure 2, the test environment Env is composed of two components: the OS and the hardware configuration. Each time the test environment is changed or updated, the versions of these components are updated. In the explanation of this figure, it was explained that the version number is changed each time the configuration is changed, but instead, the name of the configuration may be changed. This will be described in more detail later.

[0019] FIG. 4 is a diagram showing an example of the configuration information 100. The configuration information 100 is information showing the correspondence between the specific configurations of the test target software Src, the test data Dat, the test script Ts, and the test environment Env and the configuration ID. However, the configuration information 100 does not include the version number of each configuration. In other words, the configuration information 100 is information that can specify the conditions of each test by combining with the version information of each configuration. In this embodiment, the configuration ID is a combination of "K" and a number. For example, when the configuration ID of the test target software Src is "K1", it is shown that the test target software Src is composed of two parts, an "initial operation source code" and a "main processing source code".

[0020] In Fig. 4, for convenience of drawing, the test target software Src, test data Dat, test script Ts, and test environment Env are shown together, but each configuration ID is independent. For example, in a certain test, the test target software Src may have a configuration ID of "K2", the test data Dat may have a configuration ID of "K4", the test script Ts may have a configuration ID of "K1", and the test environment Env may have a configuration ID of "K2".

[0021] FIG. 5 is a diagram showing an example of the test case information 101. The test case information 101 is information indicating the configuration of each test case. The test case information 101 is composed of a plurality of records, and each record includes a field of a test case ID 1011 and a field of a configuration ID 1012. The test case ID 1011 stores an identifier of the test case. In this embodiment, a "test case" is a combination of a configuration ID specified for each of the test target software Src, the test data Dat, the test script Ts, and the test environment Env that constitute the test. For example, the test case ID "TC1" is a combination of the configuration "K1" of the test target software Src, the configuration "K1" of the test data Dat, the configuration "K1" of the test script Ts, and the configuration "K1" of the test environment Env.

[0022] FIG. 6 is a diagram showing an example of the score calculation rule information 102. The score calculation rule information 102 is information on rules for calculating a failure cause score. The failure cause score is a score for identifying whether the cause of a test failure is the test script Ts, the test data Dat, the software to be tested Src, or the test environment Env. The failure cause score is calculated for each test for each of the test script Ts, the test data Dat, the software to be tested Src, and the test environment Env. The initial value is zero, and the largest score among the four is determined to be the cause of the test failure.

[0023] The score calculation rule information 102 is composed of multiple records, and each record includes fields for a rule ID 1021, a conditional expression 1022, and an additional value 1023. The rule ID 1021 is an identifier of the rule that calculates the score, and is represented by a combination of "R" and a number. The conditional expression 1022 is a conditional expression that adds a score, and if the conditional expression is true, the value described in the additional value 1023 is added to each failure cause score. The additional value 1023 indicates the value added to the failure cause score if the conditional expression is true.

[0024] The conditional expression includes a uniquely appropriate function. The function "IsChangedV" is a function that outputs whether or not a specific type of test configuration has changed from the first version to the second version. The function "IsChangedV" outputs "true" if there has been a change, and outputs "false" if there has not been a change. The first argument of the function "IsChangedV" represents the type of test configuration, the second argument represents a number indicating the first version, and the third argument represents a number indicating the second version. The numbers of the second and third arguments are integers, and "1" or greater represents the specific version number itself, "0" represents the current version, "-1" represents the previous version, "-2" represents the version two versions ago, and so on.

[0025] For example, "IsChangedV(Ts_x, -1,0)" outputs whether there has been a change in the test script Ts since the last time it was executed. However, the third argument is set to "0" as the default argument. In other words, "IsChangedV(Ts_x, -1)" and "IsChangedV(Ts_x, -1,0)" mean the same input.

[0026] FIG. 7 is a diagram showing an example of the change history information 103. The change history information 103 is information indicating the change history of the configuration of the test. The change history information 103 is composed of a plurality of records, and each record has fields of type 1031, configuration ID 1032, version 1033, and change date and time 1034. The type 1031 is information indicating any one of the test target software Src, the test data Dat, the test script Ts, and the test environment Env. The configuration ID 1032 is information specifying the configuration, and is any one of the configuration IDs 1001 shown in the configuration information 100. The version 1033 is information specifying the version of the configuration.

[0027] In this embodiment, in order to simplify the explanation, the version numbers are the same for the same configuration, but the version numbers may be different even within the same configuration. In that case, information specifying each version number is version 1033. Change date and time 1034 is the date and time when the version was changed. According to the example of FIG. 6 based on the example of FIG. 3, it is shown that the "automatic execution script" with the configuration ID "K1" of the test script Ts was created as version "1" at "February 3, 2021, 4:05:32" Japan time.

[0028] 8 is a diagram showing an example of the test execution information 104. The test execution information 104 is information for managing the configuration and execution of each test. The above-mentioned test case did not specify the version of the configuration, but the test execution information 104 also specifies the version. The test execution information 104 is made up of multiple records, and each record has fields for a test execution ID 1041, a test case ID 1042, a configuration ID and version 1043, a priority 1044, and a test status 1045.

[0029] Test execution ID 1041 is an identifier for identifying a test, and is represented by a combination of "E" and a number. Test case ID 1042 is the same as test case ID 1011 in test case information 101. Configuration ID and version 1043 are the configuration ID and version number of each configuration for test script Ts, test data Dat, test target software Src, and test environment Env, respectively. Specifically, it is a combination of configuration ID 1011 in test case information 101 and version 1053 in score aggregation information 105. Note that the combination of test case ID and configuration ID is shown in test case information 101, so in this respect the information in test case information 101 and test execution information 104 overlap.

[0030] The priority 1044 is the priority for executing the test. A test identified by a test execution ID of a record in which the priority 1044 is set to "high" is executed before a test identified by a test execution ID of a record in which the priority 1044 is set to "low."

[0031] Test status 1045 is information indicating the state before and after execution of a test, or the result of execution of a test. In the example shown in FIG. 8, "success" and "failure" in test status 1045 indicate that a test has been executed, and "not started" in test status 1045 indicates that a test has not been executed. "success" in test status 1045 indicates that a test that has been executed has been successful, and "failure" in test status 1045 indicates that a test that has been executed has failed. In this embodiment, a successful test means that the execution result of the software Src to be tested was as intended by the developer. In this embodiment, a failed test means that the execution result of the software Src to be tested was not as intended by the developer.

[0032] FIG. 9 is a diagram showing an example of the score tally information 105. The score tally information 105 is a table showing the tally results of the failure cause scores for each test and the estimated causes of failure. The score tally information 105 is generated by the score calculation unit 151 and the estimation unit 152. The score tally information 105 is composed of a plurality of records, and each record has fields of a test execution ID 1051, a failure cause score 1052, a rule ID 1053, and an estimated failure cause 1054. The test execution ID 1051 is the same as the test execution ID 1041 of the test execution information 104. The failure cause score 1052 is a value calculated by the score calculation unit 151 based on the score calculation rule information 102. The rule ID 1053 is an enumeration of the rule ID 1021 input by the score calculation unit 151. The estimated failure cause 1054 is the cause of failure estimated by the estimation unit 152.

[0033] 10 is a diagram showing an example of the name correspondence information 106. The name correspondence information 106 is a correspondence table between symbols and names, and specifically indicates the correspondence between the type ID of a test failure cause and the type of failure cause, that is, the type of configuration.

[0034] FIG. 11 is a diagram showing an example of output to the output device 3. Specifically, it shows an example of an analysis result for one test execution result. In the example shown in FIG. 11, the result of the test execution ID "E1" is shown, and the test execution ID and the ID of the test case in that test are shown at the top. A list of failure factors, configuration ID, version, cause of failure, score, and rule ID is displayed in the center of the screen. The failure factors store the names of the test script Ts, test data Dat, test target software Src, and test environment Env. The configuration ID and version display the configuration ID and version of the target test, in this example, the test with the test execution ID "E1".

[0035] In the cause of failure section, "YES" is displayed for the item that is presumed to be the cause of failure. In the score and rule ID section, the cause of failure score and the rule ID that contributed to that score value are displayed. In the example shown in Figure 11, only the rule with rule ID "R1" was applicable, and only the cause of failure score of test script Ts was added. Therefore, since test script Ts has the highest score out of the four, "YES" is displayed in the cause of failure column for test script. Note that if there are multiple rules that are applied, multiple rule IDs will be listed in the rule ID column.

[0036] 12 is a flowchart showing the processing of the score calculation unit 151. The processing of the score calculation unit 151 described below is started in a state where the execution of one or more tests is completed and one or more records in the test execution information 104 are completed. In this case, the score tally information 105 does not include records of the test execution IDs included in the test execution information 104. For example, the processing of the score calculation unit 151 described below is started in a state where three records E1 to E3 are recorded in the test execution information 104 and no information is recorded in the score tally information 105.

[0037] In step S301, the score calculation unit 151 adds a record to the score aggregation information 105 and enters the test execution ID to be processed. In the following step S302, the score calculation unit 151 reads one record whose test status 1045 is "Failed". If there are multiple records whose test status 1045 is "Failed", it reads any one of the records that has not yet been read. Hereinafter, the record read in this step is referred to as the "target test record".

[0038] In the next step S303, the score calculation unit 151 reads a record whose usage flag 1024 is "TRUE" from the score calculation rule information 102. If there are multiple records whose usage flag 1024 is "TRUE", the score calculation unit 151 reads any one of the records that have not yet been read. Hereinafter, the record read in this step is referred to as the "target rule record". In the next step S304, the score calculation unit 151 executes a score calculation process using the target test record and the target rule record. Details of the score calculation process will be described later.

[0039] In the next step S305, the score calculation unit 151 refers to the score calculation rule information 102 and judges whether or not there is a record whose usage flag 1024 is "TRUE" that is not the target rule record. If it is judged that there is still another record whose usage flag 1024 is "TRUE", the process returns to step S303, and if it is judged that there is no other record whose usage flag 1024 is "TRUE", the process proceeds to step S306.

[0040] In step S306, the score calculation unit 151 judges whether or not there is a record whose test status 1045 is "Failed" that has not yet been set as the target test record. If it is judged that there is still another record whose test status 1045 is "Failed", the process returns to step S302, and if it is judged that there is no other record whose test status 1045 is "Failed", the process ends the process shown in Fig. 12. For example, if the test execution information 104 has three records whose test status 1045 is "Failed" and the score calculation rule information 102 has five records whose usage flag 1024 is "True", the score calculation process in step S304 is executed 15 times, which is the product of 3 and 5.

[0041] Fig. 13 is a flowchart for explaining the details of step S304 in Fig. 12, that is, the score calculation process. In step S311, the score calculation unit 151 evaluates the condition expression in the target rule record. Specifically, the score calculation unit 151 calculates the value of the condition expression and determines whether the condition is satisfied. In the following step S312, the score calculation unit 151 determines whether the condition in the target rule record is satisfied according to the result of the evaluation in step S311. If the score calculation unit 151 determines that the condition is satisfied, the process proceeds to step S313, and if it determines that the condition is not satisfied, the process shown in Fig. 13 ends.

[0042] In step S313, the score calculation unit 151 adds the rule ID 1021 of the target rule record to the rule ID 1053 of the record added in step S301 of Fig. 12 in the score aggregation information 105. However, it is not added to failure causes for which the added value 1023 is zero. For example, when the target test record is the record "E1" shown in the top row of Fig. 7 and the target rule record is the record "R1" shown in the top row of Fig. 5, the values ​​are zero except for the test script Ts, so that "R1" is entered only in the "Ts" column of the rule ID 1053 of the score aggregation information 105, and not entered in the other columns.

[0043] In the next step S314, the score calculation unit 151 adds the additional value 1023 of the target rule record to the failure cause score 1052 of the score tally information 105, and ends the process shown in Fig. 13. However, since the initial value of the score is zero, if only one rule applies, the value of the failure cause score 1052 becomes the additional value 1023 of the applicable rule itself. When the process by the score calculation unit 151 is completed, each record of the score tally information 105 contains the test execution ID 1051, the failure cause score 1052, and the rule ID 1053, and the estimated failure cause 1054 remains blank.

[0044] The estimation unit 152 enters an estimated failure cause 1054 for each record of the score aggregation information 105 generated by the score calculation unit 151. Specifically, the estimation unit 152 compares the values ​​of the failure cause score 1052 in each record of the score aggregation information 105, and enters the name of the failure cause having the maximum value in the estimated failure cause 1054. If there are multiple failure causes with the maximum value, the estimation unit 152 enters all of them in the estimated failure cause 1054.

[0045] The display generation unit 153 generates an image of the display screen shown in Fig. 11 by referring to the test execution information 104 and the score tally information 105, and outputs it to the input / output interface 85. First, the display generation unit 153 starts processing when the test execution ID of the test to be displayed is specified from outside. The display generation unit 153 writes the specified test execution ID in the upper right corner of the screen, for example, as "E1".

[0046] Next, the display generation unit 153 identifies a record in the test execution information 104 that corresponds to the specified test execution ID. Then, the display generation unit 153 transcribes the values ​​of the test case ID 1042, and the configuration ID and version 1043 described in the identified record onto the screen. Furthermore, the display generation unit 153 identifies a record in the score aggregation information 105 that corresponds to the specified test execution ID, and transcribes the values ​​of the failure cause score 1052 and the rule ID 1053 onto the screen. Finally, the display generation unit 153 enters "YES" in the failure cause column that corresponds to the estimated failure cause 1054, and ends the process.

[0047] According to the above-described first embodiment, the following advantageous effects can be obtained. (1) The analysis device 1 identifies the cause of a failure of a test of the software to be tested Src in a test environment Env, which is an environment in which the test is executed. In this test, the software to be tested is executed and the execution result is confirmed by a test script Ts without human intervention. The test data Dat is data that is read into the software to be tested Src and the test script Ts when the test is executed. The analysis device 1 includes a score calculation unit 151 that calculates a failure cause score 1052 for each of the software to be tested Ts, the test data Dat, the test script Ts, and the test environment Env based on the presence or absence of changes, and an estimation unit 152 that identifies which of the software to be tested Src, the test data Dat, the test script Ts, and the test environment Env is the cause of the test failure based on the failure cause score 1052. Therefore, it is possible to identify which of the software to be tested Src, the test data Dat, the test script Ts, and the test environment Env is the cause of the test failure.

[0048] Software testing is performed with the aim of improving the quality of the software under test Src, but sometimes the test fails due to other elements that make up the test, namely the test data Dat, the test script Ts, and the test environment Env. An expert who is familiar with the entire test, including the test environment Env, can quickly identify the cause of the test failure, but it is not easy for an inexperienced person to identify the cause of the test failure. However, by using the analysis device 1, even an expert can easily identify the cause of the test failure, improving the efficiency of software development.

[0049] (2) The analysis device 1 includes a storage unit 84 in which score calculation rule information 102 including a plurality of combinations of a conditional expression 1022 and an additional value 1023 of the failure cause score that is added when the conditional expression 1022 is satisfied is stored. The score calculation unit 151 calculates a failure cause score 1052 based on the score calculation rule information 102. The analysis device 1 includes a display generation unit 153 that generates image information, such as that shown in FIG. 11, including the failure cause score 1052 and a rule ID 1053 that is information indicating the conditional expression used to calculate the failure cause score 1052, and an input / output interface 85 that outputs the image information generated by the display generation unit 153 to the output device 3. Therefore, the analysis device 1 can present the reason why the failure cause was identified to the user.

[0050] (3) Conditional formula 1022 is related to changes in detailed components that are components of the test target software Src, the test data Dat, the test script Ts, and the test environment Env. Therefore, the analysis device 1 can identify the cause of the failure using objective facts.

[0051] (Variation 1) In the above-described first embodiment, the analysis device 1 has been described as a single hardware device. However, the analysis device 1 may be configured as an analysis system in which the same functions as those of the analysis device 1 are shared and realized by a plurality of hardware devices.

[0052] --Second embodiment-- A second embodiment of the analysis device will be described with reference to Figs. 14 to 31. In the following description, the same components as in the first embodiment are given the same reference numerals, and differences will be mainly described. Points that are not particularly described are the same as in the first embodiment. This embodiment differs from the first embodiment mainly in generating score calculation rule information 102.

[0053] FIG. 14 is a functional configuration diagram of an analysis device 1A in the second embodiment. In addition to the configuration of the first embodiment, the analysis device 1A in this embodiment further includes a rule generating unit 161, a test addition executing unit 162, a priority changing unit 163, and a failure cause correcting unit 164. The storage unit M further stores element definition information 111, element related information 112, a conditional expression initial value 113, detailed configuration information 114, and detailed related information 115. The storage unit M further stores extended test case information 101A instead of the test case information 101, score calculation extended rule information 102A instead of the score calculation rule information 102, and extended score tally information 105A instead of the score tally information 105. Furthermore, the operations of the score calculation unit 151A and the display generating unit 153A are different from those of the first embodiment.

[0054] In the first embodiment, most of the information stored in the storage unit 84 is created in advance, but this is not the case in the present embodiment. In the present embodiment, the configuration information 100, the change history information 103, the name correspondence information 106, the element definition information 111, the element related information 112, the condition expression initial value 113, the detailed configuration information 114, and the detailed related information 115 are created in advance. Some records of the extended test case information 101A are added by the test addition execution unit 162. The score calculation extended rule information 102A is generated by the rule generation unit 161. However, some records may be recorded in the score calculation extended rule information 102A in advance. Some records of the test execution information 104 are written by the test addition execution unit 162, and the priorities 1044 of some records are written by the priority change unit 163. Some fields of the extended score tally information 105A are written by the failure cause correction unit 164.

[0055] FIG. 15 is a diagram showing an example of extended test case information 101A. In extended test case information 101A, a field for confirmation 1013 is added to each record of test case information 101 in the first embodiment. Confirmation 1013 indicates whether the test result is determined by a computer or an operator. For test cases where confirmation 101 is "automatic", the test script Ts determines whether the test result is a success or failure. For test cases where confirmation 101 is "manual", the operator determines whether the test result is a success or failure.

[0056] FIG. 16 is a diagram showing an example of the score calculation extension rule information 102A. In the score calculation extension rule information 102A, fields of a detailed state 1025 and a reliability 1026 are added to each record of the score calculation rule information 102 in the first embodiment. The detailed state 1025 is rewritten in the process of creating a detailed rule based on an existing rule. Specifically, the detailed state 1025 transitions from the initial state of "not detailed", through "being detailed" and "difference value not calculated", to the final state of "completed". The reliability 1026 is a value between 0 and 1.0 indicating the reliability of the rule, and the initial value is "1.0" indicating the highest reliability.

[0057] The score calculation extension rule information 102A includes the following conditional expressions that are not included in the kick score calculation rule information 102 of the first embodiment. The function "IsChangedT" is a function that outputs whether or not a specific type of test configuration has changed from a first time to a second time. If there has been a change, it outputs "true", and if there has been no change, it outputs "false". The function "IsChangedT" has a first argument that represents the type of test configuration, a second argument that represents a numeric value that indicates the first time, and a third argument that represents a numeric value that indicates the second time. The function "IsSuccess" has an argument that is a conditional expression, and outputs whether or not the evaluation result of the conditional expression of the argument is "true".

[0058] 17 is a diagram showing an example of the extended score tally information 105A. In the extended score tally information 105A, a field for user input 1055 is added to each record of the score tally information 105 in the first embodiment. The user input 1055 is a cause of failure of a test input by a user, and is any one of a test script Ts, test data Dat, software to be tested Src, and a test environment Env. When a test is executed, at least one of the fields for the estimated cause of failure 1054 and the user input 1055 is input.

[0059] FIG. 18 is a diagram showing an example of the element definition information 111. The element definition information 111 indicates element types obtained by subdividing classification for each type of component. That is, at the beginning of the first embodiment, it was explained that each test is composed of four components, namely, a test script Ts, test data Dat, software to be tested Src, and a test environment Env, and the element types are obtained by subdividing the types of each component. The element definition information 111 is composed of a plurality of records, and each record has fields for a type 1111, an element type 1112, and a configuration name 1113. The type 1111 indicates which type of component is the test. The element type 1112 is the type of the subdivided configuration. The configuration name 1113 is the name of the detailed configuration. In the following, the element type is also called a "detailed component".

[0060] In the example shown in Fig. 18, Src is broken down into an interface SrcIf, a constant / variable SrcV, processing logic SrcLogic, a comment SrcCmt, and a component SrcComp. Note that in the example shown in Fig. 18, each type is broken down into two or more configurations, but it is not essential to break down all types; it is sufficient if at least one type of configuration is broken down. Note that, as will be described in more detail later, element types are merely classifications and do not indicate specific configurations. For this reason, element types are associated with specific configurations by combining them with individual IDs.

[0061] FIG. 19 is a diagram showing an example of element relation information 112. The element relation information 112 indicates the relation between detailed configurations. The element relation information 112 is composed of one or more records, and each record has fields for element A1121, element B1122, and relation 1123. Element A1121 and element B1122 indicate one of the detailed configurations. Relation 1123 indicates the relationship between element A1121 and element B1122. In the example shown in the first record of FIG. 19, since relation 1123 is "one-way from A to B", it is shown that a change in element A1121 affects element B1122, but a change in element B1122 does not affect element A1121.

[0062] Relationship 1123 in the second record is "bidirectional," which indicates that changing either one will affect the other. Note that elements A and B are for convenience only, and it would mean the same thing if elements A and B in the first record were swapped and relationship 1123 were reversed to "unidirectional from B to A."

[0063] 20 is a diagram showing an example of a conditional expression initial value 113. The conditional expression initial value 113 is an initial value that is read by the rule generating unit 161 and is used to generate a plurality of conditional expressions 1022. The conditional expression initial value 113 is made up of one or more records, and each record includes an initial expression ID 1131 and a conditional expression 1132. The initial expression ID 1131 is an identifier for the initial expression. The conditional expression 1132 is an initial conditional expression. As will be described in more detail later, the rule generating unit 161 creates various conditional expressions using this conditional expression 1132.

[0064] FIG. 21 is a diagram showing an example of detailed configuration information 114. Detailed configuration information 114 is information showing the correspondence between a configuration ID for each type of configuration and a specific configuration of the detailed configuration, that is, a combination of an element type and an individual ID. Although detailed classifications for each type of test configuration have been described in element definition information 111, these are merely classifications and not specific configurations. Therefore, in this embodiment, a specific configuration is associated with a combination of an element type and an individual ID. Detailed configuration information 114 shows the correspondence between configuration 1141 and detailed configuration 1142, and specifically shows the correspondence between a configuration ID for each type and a combination of an element type and an individual ID.

[0065] FIG. 22 is a diagram showing an example of the detailed related information 115. The detailed related information 115 is a relationship between specific configurations, and is different from the general correlation between types shown in the element related information 112 in that it is more specific. The detailed related information 115 is composed of one or more records, and each record has fields for element X1151, element Y1152, relationship 1153, and similarity 1154. The element X1151 and the element Y1152 are combinations of element types and individual IDs that identify specific configurations. The relationship 1153 is a relationship between the element X1151 and the element Y1152, and is one of "similar," "X depends on Y," and "Y depends on X." The similarity 1154 is a degree of similarity between the element X1151 and the element Y1152, and is expressed by a value of 0 to 1. Note that a value is set to the similarity 1154 only when the relationship 1153 is "similar."

[0066] FIG. 23 is a diagram showing an example of output to the output device 3 by the display generation unit 153A. Specifically, an example of an analysis result for one test execution result and an input by a user is shown. The display generation unit 153A performs the process of the display generation unit 153 in the first embodiment, and further reflects an input operation by a user using the input device 2 on the output device 3. The difference from FIG. 10 in the first embodiment is that the column of the cause of failure is subdivided into "calculation" and "user specification". Of "calculation" and "user specification", the "calculation" column shows the cause of failure estimated by the estimation unit 152 with "YES" as in the first embodiment. A value is input by a user's input into the "user specification" column.

[0067] In the "user designation" field, radio buttons are arranged as shown by reference symbol G21, and the user can select only one of four options. After the user selects one option, the user can select the button shown by reference symbol G22, and the value input by the user is input to the analysis device 1 as the value of user input 1055 of extended score compilation information 105A. In the example shown in Fig. 23, non-selection is represented by a white circle, and selection is represented by a black circle.

[0068] 24 and 25 are flowcharts showing the processing of the rule generating unit 161. At the time when the rule generating unit 161 starts the processing, some records may exist in the score calculation extension rule information 102A, or no records may exist at all. First, in step S341, the rule generating unit 161 reads the initial formula 113, transcribes all the conditional formulas 1132 into the column of the conditional formula 1022 of the score calculation extension rule information 102A, and sets initial values ​​to the detailed state 1025, the reliability 1026, and the use flag 1024. Specifically, the detailed state 1025 is set to "unrefined", the reliability 1026 is set to "1.0", and the use flag 1024 is set to "True", and the process proceeds to the next step S342.

[0069] In the following step S342, the rule generation unit 161 arbitrarily selects one record whose detailing status 1025 is "not detailed" in the score calculation extension rule information 102A. Hereinafter, the record selected in step S342 is referred to as the "record to be detailed." In the following step S343, the rule generation unit 161 changes the detailing status 1025 of the record to be detailed selected in step S342 to "being detailed." In the following step S344, the rule generation unit 161 executes a condition expression addition process, which will be described later, on the record to be detailed, and in the following step S345, the rule generation unit 161 changes the detailing status 1025 of the record to be processed to "difference value not calculated."

[0070] In the following step S346, the rule generation unit 161 judges whether or not a record whose detailing state 1025 is "not detailed" exists in the score calculation extension rule information 102A. If the rule generation unit 161 judges that a record whose detailing state 1025 is "not detailed" still exists, the rule generation unit 161 returns to step S342, and if it judges that no record whose detailing state 1025 is "not detailed" exists any more, the rule generation unit 161 proceeds to step S350 in FIG. 25 via circled A.

[0071] In step S350 of FIG. 25, the rule generation unit 161 arbitrarily selects one record in the score calculation extension rule information 102A whose detailing status 1025 is "difference value not calculated." Hereinafter, the record selected in step S342 will be referred to as the "difference target record." In the following step S343, the rule generation unit 161 changes the detailing status 1025 of the difference target record to "refining." In the following step S344, the rule generation unit 161 executes a difference calculation process, which will be described later, on the difference target record, and in the following step S345, the rule generation unit 161 changes the detailing status 1025 of the difference target record to "completed."

[0072] In the following step S354, the rule generation unit 161 judges whether or not a record whose detailing state 1025 is "difference value not calculated" exists in the score calculation extension rule information 102A. If the rule generation unit 161 judges that a record whose detailing state 1025 is "difference value not calculated" still exists, it returns to step S350, and if it judges that a record whose detailing state 1025 is "difference value not calculated" no longer exists, it proceeds to step S355.

[0073] In step S355, the rule generating unit 161 changes the usage flag 1024 of the record in the score calculation extension rule information 102A in which all four scores of the addition value 1023 are less than a predetermined threshold, for example, 0.5, to "False." In the following step S356, the rule generating unit 161 changes the usage flag 1024 of the record in the score calculation extension rule information 102A in which the value of the reliability 1026 is less than a predetermined threshold, for example, 0.4, to "False," and ends the process shown in FIG.

[0074] Figures 26 and 27 are flow charts showing the details of the process of step S344 in Figure 24, that is, the conditional expression addition process. Although not explained individually below, the conditional expressions generated in Figures 26 and 27 are added to a new record in the Suo Kakei extended rule information 102A, and the detailed state 1025 of this record is set to "Complete" from the beginning. Also, the reliability 1026 is set to "1" and the use flag 1024 is set to "True", as in the previous example.

[0075] As described above, the record to be detailed is determined before the conditional expression adding process is executed. First, in step S361, the rule generation unit 161 judges whether the conditional expression in the record to be detailed includes any of the superordinate concepts, i.e., Ts, Dat, Src, and Env. If the rule generation unit 161 judges that any of the superordinate concepts are included, the process proceeds to step S362, and if the rule generation unit 161 judges that none of the superordinate concepts are included, the process proceeds to step S363.

[0076] In step S362, the rule generating unit 161 refers to the element definition information 111, adds the conditional expressions in which the superordinate concepts are replaced with the respective subordinate concepts to a new record in the score calculation extension rule information 102A, and proceeds to step S363. For example, if the conditional expression of the record to be detailed includes Src, five conditional expressions are added in which Src in the conditional expression is replaced with SrcIf, SrcV, SrcLogic, SrcCmt, and SrcComp, respectively. Note that the detailed state 1025 of the added record and other items are set to their initial values, i.e., the detailed state 1025 is set to "not detailed", the reliability 1026 is set to "1.0", and the use flag 1024 is set to "True".

[0077] In step S363, rule generation unit 161 determines whether or not the conditional expression in the record to be detailed and the conditional expression added in step S362 include any of the element types included in element association information 112. If rule generation unit 161 determines that they include any of the element types included in element association information 112, it proceeds to step S364, and if it determines that they do not include any of the element types included in element association information 112, it proceeds to FIG. 27 via circled B.

[0078] Hereinafter, an element type that is paired with an element type determined in step S363 to be included in element association information 112 will be referred to as a "correlation element." For example, if TsAs is included in either the conditional expression in the record to be detailed or the conditional expression added in step S362, SrcLogic will be referred to as a correlation element based on the description of the second record in the example of FIG.

[0079] In step S364, the rule generating unit 161 adds a conditional expression that performs a logical AND on the fact that there is a change in the correlation element to a conditional expression that includes any of the element types included in the element related information 112. For example, if either the conditional expression in the record to be detailed or the conditional expression added in step S362 is "IsChangedV(TsAs_x, -1) = true", the following conditional expression is generated in step S364. That is, when it is found from the detailed related information 115 that "TaAs" included in the existing conditional expression depends on "SrcLogic", a conditional expression "IsChangedV(TsAs_x, -1) = true && IsChangedV(SrcLogic_x, -1) = true" that is a logical AND with "IsChangedV(SrcLogic_x, -1) = true", which is a conditional expression indicating that there is a change in "SrcLogic", is added. This conditional expression is indicated by the rule ID "R15" in the score calculation extension rule information 102A.

[0080] In the next step S365, the rule generation unit 161 adds a conditional expression that performs a logical product of no change in the correlation element to a conditional expression that includes any element type included in the element relation information 112. That is, a logical product is taken with the opposite condition to that in step S363. That is, under the same premise as the example in step S363, a conditional expression "IsChangedV(TsAs_x, -1) = true && IsChangedV(SrcLogic_x, -1) = false" is added in this step. This conditional expression is indicated by the rule ID "R16" in the score calculation extension rule information 102A. When the processing in step S365 is completed, the rule generation unit 161 proceeds to FIG. 27 via circled B.

[0081] 27, the rule generation unit 161 first determines in step S371 whether or not either the conditional expression in the record to be detailed or the conditional expression added in step S362 includes a time specification. If it is determined that either conditional expression includes a time specification, the process proceeds to step S372, and if it is determined that neither conditional expression includes a time specification, the process proceeds to step S373. In step S372, the rule generation unit 161 adds a conditional expression in which the time specified in the expression including a time specification is replaced with its complement to a new record in the score calculation extension rule information 102A, and then proceeds to step S373.

[0082] In step S373, the rule generation unit 161 judges whether or not either the conditional expression in the record to be detailed or the conditional expression added in step S362 includes a version specification. If it is judged that either conditional expression includes a version specification, the process proceeds to step S374, and if it is judged that neither conditional expression includes a version specification, the process proceeds to step S375. In step S374, the rule generation unit 161 adds a conditional expression in which the version specified in the expression including a version specification is replaced with a complement to a new record in the score calculation extension rule information 102A, and then proceeds to step S375.

[0083] In step S375, the rule generation unit 161 judges whether or not either the conditional expression in the record to be detailed or the conditional expression added in step S362 includes a time specification or a version specification. If it is judged that either the conditional expression includes a time specification or a version specification, the process proceeds to step S376, and if it is judged that none of the conditional expressions includes a version specification, the process shown in FIG. 27 ends.

[0084] In step S376, the rule generation unit 161 adds a conditional expression that takes the success of the test in the immediately previous version as an additional logical product to the conditional expression including the time specification or the version specification, and ends the process shown in Fig. 27. For example, if either the conditional expression in the record to be detailed or the conditional expression added in step S362 is "IsChangedV(TsAs_x, -1) = true", a conditional expression "IsChangedV(TsAs_x, -1) = true && IsSuccess(Replace(Ts, Version(Ts, -1)) = true" is added, which is a logical product with the conditional expression "IsSuccess(Replace(Ts, Version(Ts, -1)) = true" that indicates that the test in the immediately previous version was successful.

[0085] Fig. 28 is a flowchart showing details of the difference value calculation process in step S352 in Fig. 25. First, in step S381, the rule generation unit 161 initializes variables rTotal, rTs, rDat, rSrc, and rEnv to zero. These variables are counters used to count the total number of cases, Ts, Dat, Src, and Env.

[0086] In the next step S382, the rule generation unit 161 selects one record whose test status 1045 is "success" or "failure" from the test execution information 104. Hereinafter, the record selected in this step is referred to as a "selected record." In the next step S383, the rule generation unit 161 judges whether the condition of the conditional expression in the selected record is satisfied by referring to the detailed configuration information 114, the detailed related information 115, and the change history information 103 as necessary. If the rule generation unit 161 judges that the conditional expression is satisfied, the process proceeds to step S384, and if the rule generation unit 161 judges that the conditional expression is not satisfied, the process proceeds to step S386. In step S384, the rule generation unit 161 increases the value of the variable rTotal by "1," and in the next step S385, the rule generation unit 161 increases the value of one of the corresponding rTs, rDat, rSrc, and rEnv by "1" based on the cause of failure input by the user, and proceeds to step S386.

[0087] In step S386, the rule generation unit 161 judges whether or not all records whose test status 1045 is "success" or "failure" have been selected. If it is judged that all have been selected, the process proceeds to step S387, and if it is judged that there are records that have not been selected, the process returns to step S382. In step S387, the rule generation unit 161 divides each of rTs, rDat, rSrc, and rEnv by rTotal. In the following step S388, the rule generation unit 161 enters the calculation results of step S387 in each field of the addition value 1023 of the score calculation extension rule information 102A, and ends the difference value calculation process.

[0088] 29 and 30 are flowcharts showing the process of the score calculation unit 151A in the second embodiment. The difference from the process shown in Figs. 12 and 13 in the first embodiment is that a process is added between step S303 and step S304. In this embodiment, when the process of step S303 ends, the process shown in Fig. 30 is performed via the circled C, and when the process is completed, the process from step S304 onwards is executed.

[0089] In Fig. 30, the score calculation unit 151A judges whether or not all tests necessary for evaluating the conditional expression have been executed. If the score calculation unit 151A judges that all tests have been executed, the process of Fig. 30 ends and returns to Fig. 29, and if the score calculation unit 151A judges that even one test necessary for evaluating the conditional expression has not been completed, the process proceeds to step S392. In step S392, the score calculation unit 151A judges whether or not there is a test that is necessary for evaluating the conditional expression but is not scheduled to be executed. If the score calculation unit 151A judges that there is a test that is not scheduled to be executed, the process proceeds to step S393, and if the score calculation unit 151A judges that all necessary tests are scheduled to be executed, the process proceeds to step S394.

[0090] In step S393, the score calculation unit 151A causes the test addition execution unit 162 to generate a test item for executing a test that is necessary for evaluating the conditional expression but is not scheduled to be executed, and proceeds to step S394. At this time, the test addition execution unit 162 adds a new record with the priority 1044 set to "high" to the test execution information 104. In step S394, the score calculation unit 151A calls a test execution unit (not shown) and waits for a predetermined waiting time, and then returns to step S391. This completes the description of the process shown in FIG. 30. Note that in this flowchart, the score calculation unit 151A calls the test execution unit, but the test addition execution unit 162 may call the test execution unit.

[0091] The test execution unit executes the test according to the description in the test execution information 104. However, since the content of the actual test to be executed is described in the corresponding test script Ts, detailed explanation of the operation of the test execution unit will be omitted.

[0092] 31 is a flowchart showing the processing of the test addition execution unit 162. In step S401, the test addition execution unit 162 reads one record with a test status 1045 of "not started" in the test execution information 104, starting with the record with the highest priority 1044. The record read in this step is referred to as an "execution record" in the following description of this flowchart. In the following step S402, the test addition execution unit 162 executes the corresponding test script to execute the test of the execution record read in step S401. For example, when a record having a test execution ID of "E3" in FIG. 7 is read, a test script with a configuration ID of "K2" and a version of "V1" is executed.

[0093] In the next step S403, the test addition execution unit 162 refers to the extended test case information 101A and determines whether the value of the confirmation 1013 of the test to be executed is "automatic" or "manual." If the test addition execution unit 162 determines that the confirmation 1013 is "automatic," the process proceeds to step S404, and if the test addition execution unit 162 determines that the confirmation 1013 is "manual," the process proceeds to step S405. In step S404, the test addition execution unit 162 records the test result output by the test script, i.e., "success" or "failure," in the test status 1045 of the execution record in the test execution information 104, and proceeds to step S406.

[0094] In step S405, the test addition execution unit 162 records the test result in the test status 1045 of the execution record in the test execution information 104 based on the user's input from the input device, and proceeds to step S406. In step S406, the test addition execution unit 162 determines whether or not there is a record whose test status 1045 is "not started." If the test addition execution unit 162 determines that there is a "not started" record, it returns to step S401, and if it determines that there is no "not started" record, it ends the processing shown in FIG.

[0095] According to the above-described second embodiment, the following advantageous effects can be obtained. (4) Conditional formula 1022 relates to changes in detailed components that are components of the test target software Src, the test data Dat, the test script Ts, and the test environment Env, dependencies between the detailed components, and similarities between the detailed components. Therefore, the analysis device 1 can identify the cause of failure from multiple perspectives.

[0096] (5) The analysis device 1A further includes a rule generating unit 161 that adds new conditional expressions in which the test target software Src, the test data Dat, the test script Ts, and the test environment Env in the conditional expressions of the score calculation rule information are replaced with detailed components to the score calculation extension rule information 102A. This process is shown in steps S361 to S362 in Fig. 26. Therefore, the user does not need to prepare all of the score calculation rule information 102 as in the first embodiment.

[0097] (6) The rule generation unit 161 further adds a new conditional expression based on the association between the detailed components, as shown in S363 to S365 of Fig. 26. Therefore, the user does not need to prepare all of the score calculation rule information 102 as in the first embodiment.

[0098] (7) The rule generating unit 161 generates a new conditional expression by changing the time or version included in the conditional expression of the score calculation extension rule information 102A as shown in S371 to S376 of Fig. 27, and adds the new conditional expression to the score calculation extension rule information 102A. Therefore, it is not necessary for the user to prepare all of the score calculation rule information 102 as in the first embodiment.

[0099] (8) The analysis device 1A includes a test addition execution unit 162 that generates and executes new test items required to evaluate the conditional expression 1022. This makes it possible to execute additional tests required to evaluate the conditional expression.

[0100] In each of the above-mentioned embodiments and modifications, the configuration of the functional blocks is merely an example. Some functional configurations shown as separate functional blocks may be integrated, or a configuration shown in one functional block diagram may be divided into two or more functions. In addition, some of the functions of each functional block may be provided by other functional blocks.

[0101] The above-mentioned embodiments and modifications may be combined with each other. Although various embodiments and modifications have been described above, the present invention is not limited to these. Other aspects that are conceivable within the scope of the technical idea of ​​the present invention are also included in the scope of the present invention. [Explanation of symbols]

[0102] 1, 1A…Analysis device 84...Storage section 85... Input / Output Interface 101...Test case information 101A…Extended Test Case Information 102...Score calculation rule information 102A…Score calculation expansion rule information 103...Change history information 104...Test execution information 105…Score tally information 105A…Extended score summary information 106...Name correspondence information 151, 151A…Score calculation section 152...Estimation part 153, 153A...display generation section 161...Rule generation unit 162…Additional test execution unit 163…Priority change section 164…Failure cause correction section

Claims

1. An analysis device for identifying a cause of a failure of a test of software under test in a test environment in which the test is executed, comprising: In the test, the execution of the software under test and the confirmation of the execution results are performed by a test script without human intervention; test data is data that is read into the software under test and the test script when the test is executed; a score calculation unit that calculates a failure cause score based on the presence or absence of changes for each of the software under test, the test data, the test script, and the test environment; and an estimation unit that estimates whether the cause of the test failure is the software under test, the test data, the test script, or the test environment, based on the failure cause score.

2. 2. The analysis device according to claim 1, A storage unit is further provided in which score calculation rule information including a plurality of combinations of a conditional expression and an additional value of the failure cause score that is added when the conditional expression is satisfied is stored, The score calculation unit calculates the failure cause score based on the score calculation rule information, a display generating unit that generates image information including information indicating the failure cause score and the conditional expression used in calculating the failure cause score; The analysis device further includes an input / output interface that outputs the video information generated by the display generation unit to a display unit.

3. 2. The analysis device according to claim 1, A storage unit is further provided in which score calculation rule information including a plurality of combinations of a conditional expression and an additional value of the failure cause score that is added when the conditional expression is satisfied is stored, The conditional expressions relate to changes in detailed components which are components of the software under test, the test data, the test script, and the test environment, dependencies between the detailed components, and similarity relationships between the detailed components, according to an analysis device.

4. 4. The analysis device according to claim 3, The analysis device further includes a rule generation unit that adds a new conditional expression to the score calculation rule information in which the test target software, test data, test script, and test environment in the conditional expression of the score calculation rule information are replaced with the detailed components.

5. 5. The analysis device according to claim 4, The rule generating unit further adds a new conditional expression based on the association between the detailed components.

6. 5. The analysis device according to claim 4, The rule generation unit generates a new conditional expression by changing the time or version included in the conditional expression of the score calculation rule information, and adds the new conditional expression to the score calculation rule information.

7. 4. The analysis device according to claim 3, The analysis device further includes a test addition execution unit that generates new test items necessary for evaluating the conditional expressions and executes the new test items.

8. An analysis method for a computer to identify a cause of a test failure of a test target software in a test environment, the method comprising: In the test, the execution of the software under test and the confirmation of the execution results are performed by a test script without human intervention; test data is data that is read into the software under test and the test script when the test is executed; Calculating a failure cause score based on the presence or absence of changes to each of the software under test, the test data, the test script, and the test environment; and estimating, based on the failure cause score, whether the cause of the test failure is the software under test, the test data, the test script, or the test environment.

Images