Information processing apparatus, refactoring effect evaluation apparatus, and information processing method

Abstract

To efficiently and appropriately estimate a test execution time and to appropriately evaluate an effect obtained by refactoring.SOLUTION: An information processing apparatus obtains a metric value of a function of a source code of software for learning, obtains a test execution time required for testing the function by analyzing a test code and a test execution log of the software, and generates a test execution time calculation model from learning data, in which the metric value is an explanatory variable and the test execution time is an objective variable. The information processing apparatus obtains the cost of not applying refactoring to the target software to be evaluated for the effect of refactoring based on the test execution time obtained by inputting the metric value for each function of the source code of the target software into a test execution time calculation model, and obtains the cost of applying refactoring based on a test execution time for each function obtained by inputting the metric value for each function of the source code of the applied code into the test execution time calculation model.SELECTED DRAWING: Figure 12

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a refactoring effect evaluation device, and an information processing method. [Background technology]

[0002] In recent years, various technologies have been proposed to support software development and maintenance.

[0003] For example, Patent Document 1 describes an application improvement support device configured to support improvements at the architecture level of an entire application. The application improvement support device analyzes the control structure and data dependency structure of the application, generates an application model based on the analysis results of the control structure and data dependency structure of the application and the source code, searches the application model for improvement points that will reduce architectural complexity using refactoring procedure information, and outputs a list of improvement points.

[0004] Furthermore, for example, Patent Document 2 describes a development support device configured for the purpose of identifying the most cost-effective parts when refactoring in development. The development support device stores development information that is changed during the development process, generates difference information before and after each change in the development information, receives search conditions for searching for difference information required during the development process, searches for difference information that satisfies the received search conditions, displays the part of the development information that corresponds to the searched difference information, and determines the most frequently displayed part of the development information based on displayed development information that indicates the part of the displayed development information.

[0005] For example, Patent Document 3 describes a software performance prediction system designed to improve software performance and quality. The software performance prediction system includes a database for registering software, is connectable to multiple terminals, accepts input of initial source code or modified source code associated with software development from at least one of the multiple terminals, and registers the input as source code information in the database. The system compares the performance of the software based on the modified source code with the performance of the software based on the old source code included in the registered source code information, calculates a performance reduction ratio between the new and old source code, determines whether the performance reduction ratio exceeds a predetermined value, and notifies an external device of the determination result if the performance reduction ratio exceeds the predetermined value. The database stores source code information (such as source code, function structure information, and function-specific assembler instruction counts), and stores change information (such as the name of the modified function, the number of modified function assembler instructions, and performance increase / decrease ratio information). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Publication No. 2009 / 011056 [Patent Document 2] International Publication No. 2014 / 222440 [Patent Document 3] Japanese Patent Application Laid-Open No. 2012-234448 Summary of the Invention [Problem to be solved by the invention]

[0007] Due to the nature of software, repeated maintenance can lead to a complex structure, which in turn reduces readability and increases management costs. For this reason, software that has been in operation for a long time, especially large-scale software, is refactored from time to time. Appropriate software refactoring can improve software readability and localize the impact of maintenance, thereby improving maintenance efficiency and reducing maintenance costs.

[0008] However, because refactoring is also a type of software change, it can sometimes increase costs. Furthermore, refactoring carries the risk of introducing bugs and other defects into the software. Therefore, before implementing refactoring, it is necessary to verify the effectiveness of the refactoring. Furthermore, when verifying refactoring, it is necessary to properly evaluate the cumulative costs required for future repeated maintenance of the software, both with and without refactoring.

[0009] In the above-mentioned Patent Document 1, the effect of refactoring is evaluated based on the change in architectural complexity (impact scale) before and after refactoring. However, the technology described in this document only evaluates the effect of refactoring based on the analysis results of a single version of source code, and does not evaluate the cumulative cost required for repeated software maintenance after refactoring. Furthermore, there is no mention of using test execution time to evaluate refactoring.

[0010] Furthermore, Patent Document 2 states that the part of development information that is changed most frequently during the development process is the part that is most cost-effective when refactoring during development. However, the technology described in this document does not evaluate the cumulative costs required for repeated software maintenance after refactoring. Furthermore, there is no mention of using test execution time to evaluate refactoring.

[0011] When evaluating refactoring, it is effective to use the time required to test the software (hereinafter referred to as "test execution time") as an indicator of cost, but in this case, it is necessary to efficiently and appropriately estimate the test execution time of the software before and after refactoring.

[0012] In the above Patent Document 3, the performance increase / decrease ratio is calculated based on the number of assembler instructions of the changed function, the difference in the number of assembler instructions per function, the changed function impact index, and the number of function calls, but there is no mention of using test execution time to evaluate refactoring.

[0013] The present invention has been made in consideration of the above background, and aims to provide an information processing device, a refactoring effect evaluation device, and an information processing method that are capable of efficiently and appropriately estimating test execution time and appropriately evaluating the effects obtained by refactoring. [Means for solving the problem]

[0014] One aspect of the present invention for achieving the above object is a system having a processor and a storage device, which stores source code of software for use in learning, test code created for the source code, and a test execution log obtained by executing the test code, and analyzes the source code to determine metric values ​​of functions written in the source code, and analyzes the test code and the test execution log to determine the test execution time, which is the time required to test the functions, and stores a test execution time calculation model, which is a machine learning model trained using learning data in which the metric values ​​are explanatory variables and the test execution time is a target variable, and calculates the time required to execute the software based on the time required to execute tests for each function obtained by inputting the metric values ​​for each function written in the source code of the software for which the time required to execute tests is to be estimated into the test execution time calculation model.

[0015] Other problems and solutions disclosed in the present application will be made clear in the detailed description and drawings. [Effects of the Invention]

[0016] According to the present invention, it is possible to efficiently and appropriately estimate the test execution time and appropriately evaluate the effect obtained by refactoring. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a diagram illustrating the general functions of an evaluation system. [Figure 2] FIG. 1 is a diagram illustrating a schematic configuration of an evaluation system. [Figure 3A] FIG. 2 is a system flow diagram illustrating the main functions of the evaluation device. [Figure 3B] FIG. 2 is a functional block diagram illustrating main functions of the evaluation device. [Figure 4A] FIG. 10 is a diagram illustrating a method for calculating costs when a non-application is performed. [Figure 4B] FIG. 10 is a diagram illustrating a method for calculating application costs. [Figure 5] FIG. 10 is a diagram illustrating a process performed by the evaluation device when applying refactoring. [Figure 6] 10 is a flowchart illustrating a refactoring effect calculation process. [Figure 7] 10 is a flowchart illustrating a cost calculation process when each maintenance is not applied. [Figure 8] 10 is a flowchart illustrating a cost calculation process when each maintenance is applied. [Figure 9] 10 is a flowchart illustrating a refactoring application process. [Figure 10] 1 is an example of a hardware configuration of an information processing device used to realize the evaluation system. [Figure 11] FIG. 1 illustrates an example of a DevOps environment. [Figure 12] FIG. 10 is a functional block diagram illustrating the main functions of an evaluation device according to a second embodiment. [Figure 13A] This is an example of test code. [Figure 13B] 10 is an example of a test execution log. [Figure 14A] FIG. 10 is a diagram illustrating a method for learning a test execution time estimation model. [Figure 14B] FIG. 10 is a diagram illustrating preprocessing of explanatory variables and response variables. [Figure 15A] FIG. 11 is a diagram illustrating a method for calculating a cost when no application is performed in the second embodiment. [Figure 15B] FIG. 10 is a diagram illustrating a method for calculating application costs in the second embodiment. [Figure 16] FIG. 10 illustrates an example of refactoring. [Figure 17A] 10 is a flowchart illustrating a test execution time estimation model generation process. [Figure 17B] 10 is a flowchart illustrating a test execution time calculation process. [Figure 18] 10 is a flowchart illustrating a test execution time estimation process. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, an embodiment of the present invention will be described with reference to the drawings as appropriate. Note that in the following description, duplicated descriptions of identical or similar configurations may be omitted. In the following description, the letter "S" added before a reference numeral indicates a processing step.

[0019] In the following description, "refactoring" refers to changing the description or structure of software without changing its behavior.

[0020] In the following description, "maintenance" of software refers to optimizing the configuration of the software over its life cycle.

[0021] In the following description, "modification" of software refers to improvements, corrections, and extensions of the software.

[0022] In the following description, the term "version" refers to the state (generation, version, region, etc.) of software at a certain point in time.

[0023] In the following description, "code" refers to software elements that can be subject to maintenance (source code, definition information of variables and functions, etc.).

[0024] In the following explanation, "maintenance history information" refers to information showing the history of maintenance performed on each version of software. The maintenance history information includes the code of each version (hereinafter referred to as "code history information") and information showing the differences in the software code before and after maintenance (hereinafter referred to as "maintenance difference information").

[0025] [First embodiment] 1 is a diagram illustrating the general functions of an information processing system (hereinafter referred to as "evaluation system 1") according to one embodiment of the present invention. The evaluation system 1 evaluates the effect of refactoring performed on software (hereinafter referred to as "refactoring effect") and provides the evaluation results to a user 2.

[0026] The evaluation system 1 receives from the user 2 the specification of the software to be evaluated for refactoring effectiveness (hereinafter referred to as the "target software") and the specification of information indicating the content of the refactoring to be applied to the target software (hereinafter referred to as the "refactoring method").

[0027] The evaluation system 1 acquires maintenance history information (code history information 31, maintenance difference information 32) of the target software received from the user 2, and uses the acquired maintenance history information to determine the refactoring effect when refactoring using the refactoring method received from the user 2 is applied to the target software received from the user 2, and presents the determined refactoring effect to the user 2 as the evaluation result.

[0028] As shown in the figure, the evaluation system 1 calculates the cumulative total (hereinafter referred to as the "cost when not applied") of the costs (hereinafter referred to as the "cost when not applied") incurred for each of the maintenance operations (three maintenance operations A to C in the figure) actually performed on the target software (target software to which the refactoring method has not been applied; hereinafter referred to as the "unapplied software") received from the user 2.

[0029] Next, the evaluation system 1 calculates the cumulative total (hereinafter referred to as the "application cost") of the costs required for each maintenance (hereinafter referred to as the "application cost per maintenance") if the same maintenance A to C as above were to be performed on the software to which refactoring has been applied (hereinafter referred to as the "applied software") received from the user 2.

[0030] Then, the evaluation system 1 calculates the refactoring effect by subtracting the cost when the refactoring is applied from the cost when the refactoring is not applied, that is, the cost reduction, and presents this to the user 2.

[0031] In this way, according to the evaluation system 1, the user 2 can easily know the refactoring effect when the target software is refactored using the refactoring method, simply by specifying the target software and the refactoring method to the evaluation system 1.

[0032] Furthermore, the exact effect of refactoring cannot be known until maintenance has been performed on the software to which the refactoring has been applied, but because the evaluation system 1 calculates the refactoring effect from the cost before and the cost after refactoring, which can be calculated from maintenance history information, User 2 can check the refactoring effect before the refactoring is applied to the target software. Therefore, for example, User 2 can appropriately determine in advance whether or not to perform refactoring by referring to the refactoring effect presented by the evaluation system 1.

[0033] FIG. 2 is a diagram showing a schematic configuration of the evaluation system 1. As shown in the figure, the evaluation system 1 includes an evaluation device 100, a user terminal 20, and a maintenance history management device 30. The evaluation device 100, the user terminal 20, and the maintenance history management device 30 are all configured using information processing devices (computers). The evaluation device 100, the user terminal 20, and the maintenance history management device 30 are communicably connected to each other via wired or wireless communication means. The communication means is, for example, a LAN (Local Area Network), a wireless LAN, the Internet, a serial communication device, a public communication network, a dedicated line, etc.

[0034] The evaluation device 100 receives designation of target software and a refactoring method from a user 2 via a user terminal 20. The evaluation device 100 calculates the refactoring effect when refactoring according to the received refactoring method is applied to the received target software, and presents the calculated refactoring effect to the user 2 via the user terminal 20.

[0035] The maintenance history management device 30 manages the above-mentioned maintenance history management information (code history information 31, maintenance difference information 32). The maintenance history management device 30 manages the maintenance history management information in a database in a state where it can be searched by the evaluation device 100 via a communication means, for example.

[0036] The user terminal 20 has a user interface for the user 2 to specify the target software and the refactoring method. The user interface is, for example, a screen that accepts the specification of the target software and the refactoring method. The user terminal 20 also has a user interface for presenting the refactoring effects sent from the evaluation device 100 to the user 2. The user interface is, for example, a screen that describes the refactoring effects. These user interfaces are provided, for example, as functions of an integrated development environment (IDE).

[0037] Fig. 3A is a system flow diagram explaining the main functions of the evaluation device 100. Fig. 3B is a functional block diagram explaining the main functions of the evaluation device 100. As shown in these figures, the evaluation device 100 includes a storage unit 120 (not shown in Fig. 3A), a user specification receiving unit 121, a maintenance history information acquiring unit 122, a refactoring application unit 123, a cost calculation unit 124, a refactoring effect calculation unit 125, and a refactoring effect presentation unit 126.

[0038] The storage unit 120 stores the following as main information (data): target software specification information 101, refactoring method specification information 102, target software specification information 101, refactoring method specification information 102, code information for each target software version 103, difference information between target software versions 104, code information for each applied software version 105, cost when not applied 106, cost when applied 107, and refactoring effect 108.

[0039] The user specification receiving unit 121 receives information specifying the target software (hereinafter referred to as "target software specification information 101") and information specifying the refactoring method (hereinafter referred to as "refactoring method specification information 102") from the user 2 via the user terminal 20. The target software specification information 101 includes a specification of the target software (for example, a specification of the source code to be refactored). The refactoring method specification information 102 includes a specification of the refactoring method and a specification of the scope to be refactored (hereinafter referred to as the "scope of application") (for example, a specification of a predetermined description range of the source code of the target software).

[0040] The maintenance history information acquisition unit 122 acquires the code for each version (each generation) of the target software specified in the target software designation information 101 from the code history information 31, and manages the acquired code for each version as target software code information for each version 103. In addition, the maintenance history information acquisition unit 122 acquires maintenance difference information between versions of the target software (between versions before and after maintenance) from the maintenance difference information 32, and manages the acquired maintenance difference information between versions as target software version difference information 104.

[0041] The refactoring application unit 123 applies the refactoring method specified in the refactoring method specification information 102 to the target software, and manages the resulting refactored code (hereinafter referred to as "applied code") as applied software version-specific code information 105. When applying a refactoring method, the refactoring application unit 123 refers to the target software version-specific code information 103 and target software version difference information 104. The application of a refactoring method to the target software will be described in detail later.

[0042] The cost calculation unit 124 calculates the cost for each maintenance performed on the unapplied software (cost before maintenance is applied) based on the code information 103 for each target software version, and calculates the cost before maintenance is applied 106 by accumulating the calculated costs.

[0043] In addition, the cost calculation unit 124 calculates the cost per maintenance (cost at the time of application for each maintenance) that would be required if the maintenance that was actually performed on the applied software were performed based on the code information 105 for each applied software version and the difference information 104 between the target software versions, and accumulates the calculated costs at the time of application for each maintenance to calculate the cost at application 107.

[0044] The refactoring effect calculation unit 125 calculates a value obtained by subtracting the applied cost 107 from the unapplied cost 106 as an evaluation index of the refactoring effect, and manages the calculated value as the refactoring effect 108 .

[0045] The refactoring effect presentation unit 126 presents the refactoring effect 108 to the user 2 via the user terminal 20.

[0046] <How costs are calculated> Fig. 4A is a diagram explaining how the cost calculation unit 124 calculates the unapplied cost 106. Fig. 4B is a diagram explaining how the cost calculation unit 124 calculates the applied cost 107. These figures illustrate a case where the code of the target software is source code and user 2 specifies the function "foo()" of version r4 as the target software. These figures also illustrate a case where maintenance A to C have been performed on the target software.

[0047] 4A, when the cost calculation unit 124 calculates the cost when the software is not applied, the maintenance history information acquisition unit 122 first acquires code information for each target software version 103 of the software that has not been applied and target software difference information between versions 104. For example, the maintenance history information acquisition unit 122 acquires a snapshot of the source code of the target software immediately after each maintenance as the target software version code information 103. In addition, the maintenance history information acquisition unit 122 acquires information indicating the difference in the source code before and after each maintenance (for example, the number of lines added, changed, or deleted due to maintenance) as the target software difference information between versions 104.

[0048] The cost calculation unit 124 calculates the cost of each maintenance operation based on, for example, the metrics values ​​of the source code (such as the cyclomatic number and the number of lines of code) and the number of lines added, changed, or deleted by the maintenance. In this example, for version r1, line 100 of the function "foo()" was changed by maintenance A, so the cost calculation unit 124 calculates a metric value of "140" calculated for the entire function "foo()". For version r2, line 3 of the function "foo()" was changed by maintenance B, so the cost calculation unit 124 calculates a metric value of "141" calculated for the entire function "foo()". For version r3, line 300 of the function "foo()" was changed by maintenance C, so the cost calculation unit 124 calculates a metric value of "143" calculated for the entire function "foo()". Then, the cost calculation unit 124 calculates the cumulative total of the costs of each of the maintenance A to C (cost before maintenance is applied) "424" as the cost before maintenance is applied.

[0049] As shown in Figure 4B, when the cost calculation unit 124 calculates the cost at the time of application, the refactoring application unit 123 applies refactoring using the refactoring method specified by user 2 to the code of each version of the code information for each target software version 103 acquired by the maintenance history information acquisition unit 122, and generates code information for each applied software version 105.

[0050] The code information 103 for each target software version shown in the figure is the same as the maintenance history information acquisition unit 122 shown in Fig. 4A. In this example, the refactoring application unit 123 generates, as the code information 105 for each applied software version, versions r1' to r4' of applied software in which refactoring has been applied to each of versions r1 to r3 of unapplied software.

[0051] The cost calculation unit 124 calculates the cost required for each maintenance based on the applied software version-specific code information 105 and the maintenance difference information 32. Specifically, the cost calculation unit 124 calculates the cost (cost when each maintenance is applied) required if the corresponding maintenance (maintenance A for version r1', maintenance B for version r2', and maintenance C for version r3') is hypothetically performed on each of versions r1' to r3', which are the applied code of the applied software.

[0052] In this example, for version r1', the description of the external function "bar()" of the function "foo()" was changed by maintenance A, so the cost calculation unit 124 calculates a metric value of "50" calculated for the function "bar()" as the cost. For version r2', the description of the function "foo()" was changed by maintenance B, so the cost calculation unit 124 calculates a metric value of "70" calculated for the entire function "foo()". For version r3', the description of the function "bar()" written within the function "bar()" was changed by maintenance C, so the cost calculation unit 124 calculates a metric value of "72" calculated for the entire function "bar()". The cost calculation unit 124 then calculates the cumulative total of the costs of maintenance A to C (the cost at the time of application of each maintenance), "192", as the cost at the time of application.

[0053] <Refactoring> Figure 5 is a diagram explaining the process in which the refactoring application unit 123 shown in Figure 3 applies refactoring using the refactoring method specified by user 2 to the code of each version of the code information for each target software version 103 to generate code information for each applied software version 105.

[0054] The figure illustrates an example in which user 2 specifies a predetermined description range (in the figure, the range from start line (line 56) to end line (line 65)) of the latest version of the source code of the target software as the application range in the refactoring method specification information 102. Note that user 2 specifies the application range, for example, by specifying the name (identifier) ​​of a function written in the source code. In this case, the range in which the function in question is written in the source code is specified as the application range.

[0055] In this example, the refactoring application unit 123 first identifies the application range of each version of the source code before the latest version that corresponds to the application range by tracing back sequentially from the specified latest version of the source code to the previous version (generation) using the target software version difference information 104.

[0056] In the example shown in the figure, the refactoring application unit 123 identifies "start line (line 56) to end line (line 63)" as the application range of the source code of the version before the latest version, which corresponds to the application range of the source code of the latest version.

[0057] The refactoring application unit 123 identifies the scope of application of the source code of the version immediately preceding the latest version of the source code based on the relationship between the start line and end line of the scope of application of the latest version of the source code and the top end (hereinafter referred to as the "top end of the changed line") and bottom end (hereinafter referred to as the "bottom end of the changed line") of the line changed by maintenance, which is obtained from the target software version difference information 104.

[0058] More specifically, the refactoring application unit 123 calculates the number of lines added (hereinafter referred to as the "number of added lines") and the number of lines deleted (hereinafter referred to as the "number of deleted lines") due to the maintenance performed on the previous version of the source code, and uses the calculated number of added lines and number of deleted lines to identify the start line and end line of the application range of the previous version of the source code based on the following determination rule.

[0059] If the bottom of the line being changed is less than the starting line: Start line = Start line - (Number of added lines - Number of deleted lines) End line = End line - (Number of added lines - Number of deleted lines)

[0060] If top of change line < start line < bottom of change line: Start line = top of change line End line = End line - (Number of added lines - Number of deleted lines)

[0061] If start line < top of change line < bottom of change line < end line: start line = start line End line = End line - (Number of added lines - Number of deleted lines)

[0062] If the top of the changed line < the end line < the bottom of the changed line: start line = start line End line = bottom of changed line

[0063] If end line < top of change: start line = start line End line = End line

[0064] The refactoring application unit 123 identifies the scope of application of each version by repeatedly performing the above procedure, going back through the versions in order. By repeatedly performing the above procedure in this way, it is possible to efficiently identify the scope of application of each version.

[0065] Each time the refactoring application unit 123 identifies the application scope of each version, it determines whether the contents of the application scope of each version satisfy the conditions (hereinafter referred to as "preconditions") for applying refactoring using the refactoring method specified by the user 2. The refactoring application unit 123 does not identify the application scope of versions (generations) earlier than the version whose contents of the application scope satisfy the preconditions, and does not include them in the calculation of the cost of application per maintenance. In this way, if the contents of the application scope do not satisfy the preconditions, they are not included in the calculation of the cost of application per maintenance, so that the cost of application per maintenance can be appropriately calculated within the range of versions that satisfy the preconditions.

[0066] The above preconditions are, for example, that the content corresponding to the application range is described, and that the description of the application range is complete (e.g., accessing an inaccessible function). Whether the content of the application range satisfies the preconditions may be determined, for example, by a function for determining whether the preconditions are satisfied that is provided in a general integrated development environment.

[0067] Next, the refactoring application unit 123 applies the refactoring specified in the refactoring method specification information 102 to the specified application range of the source code of each version, and generates the source code of each version to which the refactoring has been applied as code information 105 for each applied software version.

[0068] Furthermore, the cost calculation unit 124 calculates the cost at the time of application of each maintenance, for example, for the description range that has actually been maintained in the past and is included in the application range identified for each version of the source code (the description range that has been maintained for each version, identified from the target software version difference information 104).

[0069] <Processing explanation> 6 is a flowchart explaining the main processing (hereinafter referred to as "refactoring effect calculation processing S600") performed by the cost calculation unit 124 when calculating the refactoring effect. The refactoring effect calculation processing S600 will be explained below with reference to FIG. 6. In the following explanation, it is assumed that the evaluation device 100 has already received the target software and refactoring method specifications from the user 2 via the user terminal 20, and that the received contents are managed as target software specification information 101 and refactoring method specification information 102, respectively.

[0070] As shown in the figure, when calculating the refactoring effect, the cost calculation unit 124 first performs a process of calculating the cost when each maintenance is not applied (hereinafter referred to as "a cost calculation process when each maintenance is not applied S611"). Details of the cost calculation process when each maintenance is not applied S611 will be described later.

[0071] Next, the cost calculation unit 124 performs a process of calculating the cost at the time of application of each maintenance (hereinafter referred to as "a cost calculation process at the time of application of each maintenance S612"). Details of the cost calculation process at the time of application of each maintenance S612 will be described later.

[0072] Next, the cost calculation unit 124 aligns (sets to the smaller number) the number of versions for which the cost per maintenance not applied is calculated (the number of versions for which the maintenance not applied cost is calculated) and the number of versions for which the cost per maintenance applied is calculated (the number of versions for which the maintenance applied cost is calculated) (S613). Note that this process is necessary to exclude versions earlier than the version whose application range content does not satisfy the preconditions from being included in the calculation of the cost per maintenance applied.

[0073] Next, the cost calculation unit 124 accumulates the costs for each maintenance period when no maintenance is applied to obtain the cost for when no maintenance is applied (S614).

[0074] Next, the cost calculation unit 124 accumulates the application costs for each maintenance to obtain the application cost (S615).

[0075] Next, the cost calculation unit 124 calculates the refactoring effect by subtracting the applied cost from the unapplied cost (S616).

[0076] 7 is a flowchart for explaining the details of the cost calculation process S611 when maintenance is not applied. The cost calculation process S611 when maintenance is not applied will be explained below with reference to the same figure.

[0077] First, the cost calculation unit 124 selects the target software specified in the target software specification information 101 as the target version (S711). In this example, the latest version of the source code is specified in the target software specification information 101, and the latest version is selected as the target version, but the target software specification information 101 may specify an older version (generation) of source code as the target software. User 2 can specify any range of versions (generations) of source code as the source code to be used in calculating the costs (cost when not applied, cost when applied).

[0078] The subsequent processing of S712s to S712e is a loop process that is repeatedly performed by going back one version at a time to each version of the target software.

[0079] In S713, the cost calculation unit 124 determines whether the application range (for example, the range in which a specified function is written) specified in the refactoring method specification information 102 exists in the target version. If the cost calculation unit 124 determines that the application range exists in the target version (S713: Yes), the process proceeds to S714. On the other hand, if the cost calculation unit 124 determines that the application range does not exist in the target version (S713: No), the cost calculation process for when maintenance is not applied S611 ends, and the process proceeds to S612 in FIG. 6.

[0080] In S714, the cost calculation unit 124 identifies the range of maintenance performed on the version immediately before the target version (the range of descriptions that were subject to maintenance in the source code of the version immediately before) based on the target software version difference information 104.

[0081] Next, the cost calculation unit 124 calculates the cost for each maintenance not applied for the range of maintenance identified in S714 (S715).

[0082] Next, the cost calculation unit 124 resets the target version to the version immediately preceding the current target version (S716). Note that unless the determination in S713 is "No," the cost calculation unit 124 repeats the loop processing (S712s to S712e) until there is no previous target version.

[0083] 8 is a flowchart for explaining the details of the maintenance application cost calculation process S612. The maintenance application cost calculation process S612 will be explained below with reference to the same figure.

[0084] First, the cost calculation unit 124 selects the target software (in this example, the latest version of the source code) specified in the target software specification information 101 as the target version (S811).

[0085] The subsequent processing of S812s to S812e is a loop process that is repeatedly performed by going back one version at a time to each version of the target software.

[0086] In S813, the cost calculation unit 124 determines whether the application range (for example, the range in which a specified function is written) specified in the refactoring method specification information 102 exists in the target version. If the cost calculation unit 124 determines that the application range exists in the target version (S813: Yes), the process proceeds to S814. On the other hand, if the cost calculation unit 124 determines that the application range does not exist in the target version (S813: No), the per-maintenance application cost calculation process S612 ends, and then the process proceeds to S613 in FIG. 6.

[0087] In S814, the cost calculation unit 124 performs a process of applying the refactoring specified in the refactoring method specification information 102 to the target version (hereinafter referred to as "refactoring application process S814"). The refactoring application process S814 will be described in detail later.

[0088] In S815, the cost calculation unit 124 determines whether or not a refactoring has been applied to the target version in the refactoring application process S814, based on the return value of the refactoring application process S814. If the cost calculation unit 124 determines that a refactoring has been applied to the target version (S815: Yes), the process proceeds to S816. On the other hand, if the cost calculation unit 124 determines that a refactoring has not been applied to the target version (S815: No), the per-maintenance application cost calculation process S612 ends, and then the process proceeds to S613 in FIG. 6.

[0089] In S816, the cost calculation unit 124 identifies the scope of maintenance performed on the version immediately before the target version (the description scope that was the subject of maintenance in the source code of the immediately previous version) based on the target software version difference information 104.

[0090] Next, the cost calculation unit 124 calculates the cost for each maintenance application for the scope of maintenance identified in S816 (S817).

[0091] Next, the cost calculation unit 124 resets the target version to the version immediately preceding the current target version (S818). Note that the cost calculation unit 124 repeats the loop process (S812s to S812e) until there is no previous target version.

[0092] Fig. 9 is a flowchart for explaining the details of the refactoring application process S814 in Fig. 8. The refactoring application process S814 will be explained below with reference to this figure.

[0093] First, the cost calculation unit 124 determines whether the contents of the refactoring application scope specified in the refactoring method specification information 102 satisfy the preconditions for applying the refactoring method specified in the refactoring method specification information 102 (S911). The application scope is specified by the "start line" and "end line" of the source code. If the cost calculation unit 124 determines that the contents of the application scope satisfy the preconditions (S911: Yes), the process proceeds to S912. On the other hand, if the cost calculation unit 124 determines that the contents of the application scope do not satisfy the preconditions (S911: No), the refactoring application process S814 ends, and then the process proceeds to S815 in FIG. 8. In this case, the return value of the refactoring application process S814 indicates that the refactoring was not applied.

[0094] In S912, the cost calculation unit 124 applies the refactoring specified in the refactoring method specification information 102 to the application scope.

[0095] Next, the cost calculation unit 124 acquires the differences between the target version and the previous version (differences in source code (number of added lines, number of deleted lines)) from the target software version difference information 104 (S913).

[0096] Next, the cost calculation unit 124 identifies the application range of the previous version that corresponds to the application range of the target version based on the difference acquired in S913, and updates the start line and end line of the application range currently held (S914 to S919). As described above, the cost calculation unit 124 identifies the application range of the source code of the previous version based on the relationship between the start line and end line of the application range of the target version and the top end (top end of changed line) and bottom end (bottom end of changed line) of the lines changed by maintenance.

[0097] For example, if the relationship "bottom of changed line < starting line" holds, the cost calculation unit 124 updates the starting line to "number of added lines - number of deleted lines" and updates the value of the ending line to the value obtained by subtracting "number of added lines - number of deleted lines" from the current value of the ending line (S915).

[0098] Also, for example, if the relationship "top of changed line < start line < bottom of changed line" holds, the cost calculation unit 124 updates the value of the start line to "top of changed line" and updates the value of the end line to the value obtained by subtracting "number of added lines - number of deleted lines" from the current value of the end line (S916).

[0099] Also, for example, if the relationship "start line < top of changed line < bottom of changed line < end line" holds, the cost calculation unit 124 leaves the value of the start line as it is, and updates the value of the end line to the value obtained by subtracting "number of added lines - number of deleted lines" from the current value of the end line (S917).

[0100] Also, for example, if the relationship "top of changed line < end line < bottom of changed line" holds, the cost calculation unit 124 leaves the value of the start line as the current value and updates the value of the end line to "bottom of changed line" (S918).

[0101] Also, for example, if the relationship "end line<changed upper end" holds, the cost calculation unit 124 does not update the values ​​of either the start line or the end line (S919).

[0102] After the processes of S912 to S919, the refactoring application process S814 ends, and the process proceeds to S815 in Fig. 8. In this case, the return value of the refactoring application process S814 indicates that the refactoring has been applied.

[0103] As described above, according to the evaluation system 1 of this embodiment, the user 2 can easily know the effect of refactoring by specifying the target software and the refactoring method to the evaluation system 1. Furthermore, the evaluation system 1 calculates the refactoring effect from the cost before application and the cost after application, which can be calculated from the maintenance history information, so the user 2 can confirm the refactoring effect before applying the refactoring to the target software. Therefore, for example, if the refactoring presented by the evaluation system 1 has a low refactoring effect, the user 2 can avoid incurring unnecessary costs by not performing the refactoring.

[0104] Furthermore, User 2 can appropriately determine in advance whether or not to perform refactoring by referring to the refactoring effects presented by the evaluation system 1. Therefore, for example, User 2 can prevent unnecessary costs by not performing refactorings with low refactoring effects. Furthermore, by implementing refactorings with high refactoring effects presented by the evaluation system 1, User 2 can improve software maintenance efficiency and reduce future maintenance costs. For example, in software testing during maintenance (white-box unit testing, regression testing, etc.), the number of test cases can be reduced while maintaining code coverage, allowing for more efficient testing. Furthermore, for example, the efficiency of builds and tests in so-called DevOps environments (CI / CD environments) can be improved.

[0105] <Hardware> FIG. 10 shows an example of the hardware configuration of an information processing device (computer) used to realize the evaluation system 1 described above.

[0106] The illustrated information processing device 10 includes a processor 11, a main storage device 12 (memory), an auxiliary storage device 13 (external storage device), an input device 14, an output device 15, and a communication device 16. These are communicatively connected via a bus, a communication cable, etc. Examples of the information processing device 10 include a personal computer, a server device, a smartphone, a tablet, an office computer, a general-purpose machine (mainframe), etc.

[0107] All or part of the information processing device 10 may be realized using virtual information processing resources provided using virtualization technology, process space separation technology, or the like, such as a virtual server provided by a cloud system. All or part of the functions provided by the information processing device 10 may be realized by a service provided by a cloud system via an API (Application Programming Interface), for example. All or part of the functions provided by the information processing device 10 may be realized using, for example, Software as a Service (SaaS), Platform as a Service (PaaS), Infrastructure as a Service (IaaS), or the like.

[0108] The processor 11 is configured using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), an AI (Artificial Intelligence) chip, etc.

[0109] The main memory device 12 is a device used by the processor 11 when executing a program, and is, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a non-volatile memory (NVRAM (Non Volatile RAM)), etc. Various functions realized in the evaluation system 1 are realized by each processor 11 reading out programs and data stored (memorized) in the auxiliary memory device 13 into the main memory device 12 and executing them.

[0110] The auxiliary storage device 13 is a device that stores programs and data, and can be configured, for example, by an SSD (Solid State Drive), a hard disk drive, an optical storage device (e.g., a CD (Compact Disc) or a DVD (Digital Versatile Disc)), a storage system, a read / write device for non-transitory recording media such as an IC card, an SD card or an optical recording media, or a non-transitory storage area of ​​a cloud server. Programs and data can be read into the auxiliary storage device 13 from other information processing devices equipped with non-transitory recording media or non-transitory storage devices via a recording media reader or a communication device 16. The programs and data stored in the auxiliary storage device 13 are read into the main storage device 12 as needed.

[0111] The input device 14 is an interface that accepts input of information from the outside, and is, for example, a keyboard, a mouse, a touch panel, a card reader, a pen-input tablet, a voice input device, or the like.

[0112] The output device 15 is an interface that outputs various information such as the progress of processing and the results of processing to the outside. The output device 15 is, for example, a display device (liquid crystal monitor, LCD (Liquid Crystal Display), graphic card, etc.) that visualizes the various information described above, a device that converts the various information described above into audio (audio output device (speaker, etc.)), or a device that converts the various information described above into text (printer, etc.). Note that, for example, the information processing device 10 may be configured to input and output information to and from other devices via the communication device 16.

[0113] The input device 14 and the output device 15 constitute a user interface that realizes interactive processing with the user (receiving information, providing information, etc.).

[0114] The communication device 16 is a device that realizes communication with other devices. The communication device 16 is a wired or wireless communication interface that realizes communication with other devices via the communication medium 5, and is, for example, a NIC (Network Interface Card), a wireless communication module, a USB module, or the like.

[0115] The information processing device 10 may be equipped with, for example, an operating system, a file system, a DBMS (DataBase Management System) (relational database, NoSQL, etc.), a KVS (Key-Value Store), etc.

[0116] [Second embodiment] In the first embodiment, the evaluation system 1 calculates the pre-application cost 106 and the application cost 107 using metrics values ​​(cyclomatic number, number of lines, etc.) of the source code of the pre-application software and the application software. In the second embodiment, the evaluation system 1 calculates the pre-application cost 106 and the application cost 107 based on the time required to test the software (pre-application software, application software).

[0117] For example, in a DevOps environment (CI / CD environment) like the one shown in Figure 11, when a developer creates and commits source code, builds and tests are automatically executed. In order to properly cover all branches and execution paths in the above tests (achieve appropriate code coverage (hereinafter also referred to as "coverage")), test cases with a variety of input conditions are required. When complex or large functions are modified or changed in particular, a large number of test cases must be created and executed, which takes a lot of time to perform testing and increases maintenance costs. Therefore, the time required for software testing is an effective indicator for estimating the effectiveness of software refactoring (the difference between the cost when refactoring is not applied (106) and the cost when refactoring is applied (107)).

[0118] The evaluation system 1 of the second embodiment will be described below. Note that the basic configuration of the evaluation system 1 of the second embodiment is common to that of the evaluation system 1 of the first embodiment, and therefore the following description will focus on the differences from the evaluation system 1 of the first embodiment.

[0119] 12 is a functional block diagram illustrating the main functions of the evaluation device 100 in the evaluation system 1 of the second embodiment. As shown in the figure, the evaluation device 100 of the second embodiment includes a test execution time estimation model generation unit 130 and a test execution time estimation unit 140 in addition to the functions included in the evaluation device 100 of the first embodiment shown in FIG.

[0120] Furthermore, the storage unit 110 of the evaluation device 100 of the second embodiment stores, in addition to the main information stored in the storage unit 110 of the evaluation device 100 of the first embodiment, source code 109, test code 111, test execution log 112, learning data 113, test execution time estimation model 114, and test execution time estimation result 115.

[0121] Of these, the source code 109 is source code used for learning the test execution time estimation model 114, which is a machine learning model, and is acquired from a version management system such as the maintenance history management device 30, for example.

[0122] The test code 111 includes test code created for the source code 109, obtained from the test environment. The test code 111 includes, for example, code (hereinafter referred to as "test function") for testing one or more functions to be tested (hereinafter referred to as "test target function"). The test code 111 is created and managed, for example, as a file in a file system.

[0123] 13A shows an example of test code 111. The illustrated test code 111 includes test code (test functions) for test target functions "foo()" and "hoge()" described in source code 109. The illustrated test code 111 is managed as a file with the file name "test / foo_test.c".

[0124] 12, the test execution log 112 is an execution log (execution history, execution result) of the test (test code 111) generated by executing the test code 111. The test execution log 112 includes information such as the name of the test target function that was the subject of the test (the name of the test target function (hereinafter referred to as the "test target function name")), the test execution time (Duration) (the test execution time for each test function (or each test target function)), the test execution result (Result) (the execution result (success / failure) for each test function (or each test target function)), and statistical information (summary information) of the test execution results (for example, the total test execution time and test success rate based on the execution results of each test for multiple test functions (or test target functions) described in one file).

[0125] FIG. 13B shows an example of the test execution log 112. The illustrated test execution log 112 contains information based on the test execution history in report format. The test execution log 112 may also include standard output and function call history (trace information). The test execution log 112 is created as a file written in a format such as CSV (Comma Separated Values) or XML (Extensible Markup Language). The illustrated test execution log 112 is managed as a file with the file name "test / foo_test.c test report."

[0126] Returning to FIG. 12, the test execution time estimation model generation unit 131 generates the test execution time estimation model 114, which is a machine learning model used to calculate the time required for software testing.

[0127] When the test execution time estimation model 114 receives the metrics values ​​obtained for the functions described in the source code 109 as explanatory variables, it outputs the predicted value of the time required to test the functions (hereinafter referred to as "test execution time") as the objective variable.

[0128] Here, the above metric values ​​are, for example, metric values ​​in the Maintainability Index (MI), such as the number of lines of source code, the cyclomatic number, the halstead volume, the number of lines of written comments, the number of written arguments, the number of written function calls, etc.

[0129] The algorithm of the test execution time estimation model 114 is not necessarily limited, and general machine learning algorithms such as Random Forest and XGBoost can be used. The test execution time estimation model 114 may also be configured using a statistical method of multivariate analysis such as multiple regression analysis. However, since the accuracy of multiple regression analysis decreases due to multicollinearity when explanatory variables are highly correlated, when explanatory variables are highly correlated, preprocessing is required, such as converting highly correlated explanatory variables into independent explanatory variables or narrowing down the explanatory variables (such as excluding one of the highly correlated explanatory variables).

[0130] The test execution time estimation unit 132 shown in the figure generates a test execution time estimation result 115, which is information used to estimate the time required to test software (unapplied software, applied software) using a test execution time estimation model 114. The test execution time estimation result 115 includes, for example, information indicating the test execution time for each function described in the source code of the software and the total test execution time for each function. The cost calculation unit 124 of the evaluation device 100 of the second embodiment calculates an unapplied cost 106 and an applied cost 107 based on the test execution time estimation result 115.

[0131] FIG. 14A is a diagram for explaining a method for learning the test execution time estimation model 114 by the test execution time estimation model generation unit 131.

[0132] As shown in the figure, the test execution time estimation model generation unit 131 acquires metric values ​​for each function described in the learning source code 109 acquired from a version control system or the like, and generates explanatory variables by performing preprocessing S1450 on the acquired metric values.

[0133] Furthermore, the test execution time estimation model generation unit 131 generates a target variable by performing preprocessing S1450 on the test execution time of the test execution log 112 of the test code 111 acquired from the test environment or the like.

[0134] Then, the test execution time estimation model generation unit 131 performs machine learning using the generated explanatory variables and sets (combinations) of objective variables corresponding to the explanatory variables as learning data 113 (S1460), thereby generating a test execution time estimation model 114.

[0135] Fig. 14B is a diagram for explaining the details of the pre-processing shown in Fig. 14A. As shown in Fig. 14B, the test execution time estimation model generation unit 131 performs the following pre-processing steps: identifying a function to be tested (S1451), filtering invalid tests (S1452), and correcting the test execution time (S1453).

[0136] In step S1451 of specifying the test target function, the test execution time estimation model generation unit 131 analyzes the test function of the test code 111 to specify the test target function called by the test function.

[0137] A test execution log does not usually include information that directly indicates the function to be tested (such as the name of the function to be tested). On the other hand, the test execution log 112 includes information that identifies the corresponding test code (test function) (such as the path name of the source file and the name of the test function (hereinafter referred to as the "test function name").

[0138] Therefore, in identifying the test target function S1451, the test execution time estimation model generation unit 131 analyzes the test code 111 to find the correspondence (call relationship) between the test function (test function name) and the test target function (test target function name), and obtains the test execution time for each test target function.

[0139] However, depending on the type of test, for example, the call stack (call sequence, call hierarchy) of the function may be included in the test execution log 112. In that case, the function to be tested can be identified from the call stack, and the above correspondence can be obtained from the call stack without analyzing the test code 111.

[0140] In invalid test filtering S1452 shown in the figure, the test execution time estimation model generation unit 131 excludes information that is determined to be inappropriate for use in generating the learning data 113 of the test execution time estimation model 114.

[0141] For example, for a test whose result in the test execution log 112 is "failed," the test execution time recorded in the test execution log 112 may not necessarily be an appropriate value, for example, because the test was aborted midway.

[0142] Therefore, information (such as test execution time) obtained by executing a test function for a test target function whose test result is a failure in the test execution log 112 is excluded from the data to be generated for the learning data 113 (hereinafter referred to as "first filtering").

[0143] Generally, the test execution log 112 includes information indicating the success or failure of the test for each test function in the test code 111, and information on failed test functions is excluded from the target for generating the learning data 113 by utilizing this information.

[0144] By performing the first filtering, it is possible to improve the accuracy of the test execution time prediction by the test execution time estimation model 114.

[0145] Furthermore, if the test cases described in the test code 111 are too poor and the coverage is too low (for example, if the coverage is below a preset threshold (such as 30%, hereinafter referred to as the "first threshold")), it is highly likely that the test code 111 does not include a description that dominates the test execution time (a description of a test function).

[0146] Therefore, information (such as test execution time) acquired by executing a test function for such a function to be tested is excluded from the data to be generated for the learning data 113 (hereinafter referred to as "second filtering").

[0147] By performing the second filtering, the accuracy of the test execution time estimation model 114 in predicting the test execution time can be improved.

[0148] In correction of test execution time S1453 shown in the figure, the test execution time estimation model generation unit 131 corrects the test execution time entered as a test result in the test execution log 112.

[0149] For example, if the coverage is equal to or greater than the first threshold but the test code 111 does not necessarily contain enough test cases (if it is less than a preset threshold (hereinafter referred to as the "second threshold" (>first threshold))), the test execution time will be short relative to the complexity and size of the function, and the above correction will be necessary.

[0150] If the coverage is equal to or greater than the first threshold and less than the second threshold, the test execution time estimation model generation unit 131 corrects the test execution time used as the objective variable of the training data 113 so that it becomes the time required for testing when the coverage is 100%. For example, the test execution time estimation model generation unit 131 corrects the test execution time based on the following equation. [Number 1] Corrected test execution time = Test execution time in the test execution log × (required test cases / actual test cases)

[0151] By correcting the test execution time in this manner, it is possible to improve the accuracy of the test execution time prediction by the test execution time estimation model 114.

[0152] The coverage calculation method (coverage standard) used in the pre-processing S1450 described above includes, for example, the following. C0 coverage (instruction coverage): If all instructions are executed, the coverage is 100% C1 coverage (branch coverage): 100% coverage if all branch directions are executed C2 coverage (condition coverage): If all conditional expressions are met, the coverage is 100% Path coverage: 100% coverage if all combinations of multiple conditional branches are executed

[0153] In addition to what is shown in the same figure, the test execution time estimation model generation unit 131 appropriately performs general preprocessing as needed, such as excluding outliers and filling in missing data, which is performed when learning a machine learning model.

[0154] <How costs are calculated> Fig. 15A is a diagram illustrating a method for calculating the unapplied cost 106 by the cost calculation unit 124 of the second embodiment. Fig. 15B is a diagram illustrating a method for calculating the applied cost 107 by the cost calculation unit 124 of the second embodiment. These figures illustrate a case where the code of the target software is source code and user 2 specifies the function "foo()" of version r4 as the target software. These figures also illustrate a case where maintenance A to C have been performed on the target software.

[0155] 15A, when the cost calculation unit 124 calculates the cost when the software is not applied, the maintenance history information acquisition unit 122 first acquires code information for each target software version 103 and target software version difference information 104 for the unapplied software. The maintenance history information acquisition unit 122 acquires, for example, a snapshot of the source code of the target software immediately after each maintenance as the target software version code information 103. The maintenance history information acquisition unit 122 also acquires information indicating the difference in the source code before and after each maintenance (for example, the number of lines added, changed, or deleted due to maintenance) as the target software version difference information 104.

[0156] The cost calculation unit 124 calculates the cost of each maintenance based on the test execution time of the unapplied software. In this example, for version r1, line 100 of function "foo()" was changed by maintenance A, and the cost calculation unit 124 estimates the test execution time of function "foo()" to be "15 minutes" as the cost. For version r2, line 3 of function "foo()" was changed by maintenance B, and the cost calculation unit 124 estimates the test execution time of function "foo()" to be "17 minutes" as the cost. For version r3, line 300 of function "foo()" was changed by maintenance C, and the cost calculation unit 124 estimates the test execution time of function "foo()" to be "23 minutes" as the cost. The cost calculation unit 124 then calculates the cumulative cost of each maintenance A to C (the cost when each maintenance is not applied) to be "55 minutes" as the cost when the maintenance is not applied.

[0157] As shown in Figure 15B, when the cost calculation unit 124 calculates the application cost, the refactoring application unit 123 applies refactoring using the refactoring method specified by user 2 to the code of each version of the code information for each target software version 103 acquired by the maintenance history information acquisition unit 122, to generate code information for each applied software version 105.

[0158] The code information 103 for each target software version shown in the figure is the same as the maintenance history information acquisition unit 122 shown in Fig. 4A. In this example, the refactoring application unit 123 generates, as the code information 105 for each applied software version, versions r1' to r4' of applied software in which refactoring has been applied to each of versions r1 to r3 of unapplied software.

[0159] The cost calculation unit 124 calculates the cost required for each maintenance based on the applied software version-specific code information 105 and the maintenance difference information 32. Specifically, the cost calculation unit 124 calculates the cost (cost when each maintenance is applied) required if the corresponding maintenance (maintenance A for version r1', maintenance B for version r2', and maintenance C for version r3') is hypothetically performed on each of versions r1' to r3', which are the applied code of the applied software.

[0160] The cost calculation unit 124 calculates the cost of each maintenance based on the test execution time of the applied software. In this example, for version r1', the description of the external function "bar()" of the function "foo()" was changed by maintenance A, and the cost calculation unit 124 estimates the test execution time of the function "foo()" to be "5 minutes" as the cost. For version r2', the description of the function "foo()" was changed by maintenance B, and the cost calculation unit 124 estimates the test execution time of the function "foo()" to be "8 minutes" as the cost. For version r3, the description of the function "bar()" written within the function "bar()" was changed by maintenance C, and the cost calculation unit 124 estimates the test execution time of the function "foo()" to be "12 minutes" as the cost. The cost calculation unit 124 then calculates the cumulative total of the costs of each maintenance A to C (the cost at the time of application of each maintenance) to be "25 minutes" as the cost at the time of application.

[0161] <Examples of the effects of refactoring> Figure 16 shows an example of refactoring (an example of refactoring in which the cost when applied is lower than the cost when not applied (reducing test execution time)). In this example, a single function "foo()" is split into multiple functions "foo()" and "bar()" through refactoring (method extraction, etc.). The figure also shows an example of test cases that will result in 100% C0 coverage.

[0162] As shown in the figure, for the function "foo()" before refactoring, three test cases must be implemented to achieve 100% C0 coverage. On the other hand, for the functions "foo()" and "bar()" after refactoring, two test cases must be implemented each. However, if only the function "foo()" is modified, then only the function "foo()" needs to be tested, and only two test cases are required, thereby reducing the time required for testing.

[0163] <Processing explanation> 17A is a diagram for explaining the process (hereinafter referred to as "test execution time estimation model generation process S1700") that the test execution time estimation model generation unit 131 performs when generating the test execution time estimation model 114. Below, the test execution time estimation model generation process S1700 will be explained with reference to this diagram.

[0164] First, the test execution time estimation model generation unit 131 analyzes the learning source code 109 to obtain the metric values ​​for each function to be used as explanatory variables of the learning data 113 (S1711).

[0165] Next, the test execution time estimation model generation unit 131 calculates the test execution time for each function to be used as a target variable of the learning data 113, based on the source code 109, the test code 111 created for the source code 109, and the test execution log 112 of the source code 109 generated by the test code 111 (S1712). Details of this process (hereinafter referred to as "test execution time calculation process S1712") will be described later.

[0166] Next, the test execution time estimation model generation unit 131 generates the learning data 113 by associating the test execution time (objective variable) calculated for each of the metrics values ​​(explanatory variables) for each function acquired in S1711 with the corresponding metrics values ​​(explanatory variables) in S1712 (S1713).

[0167] Next, the test execution time estimation model generation unit 131 performs machine learning using the generated training data 113 to generate the test execution time estimation model 114 (S1714).

[0168] This completes the test execution time estimation model generation process S1700.

[0169] Fig. 17B is a flowchart for explaining the test execution time calculation process S1712 in Fig. 17A. Below, the test execution time calculation process S1712 will be explained with reference to this figure.

[0170] As shown in the figure, the test execution time estimation model generation unit 131 analyzes the test code 111 and generates information (hereinafter referred to as "first correspondence") indicating the calling relationship (the relationship between the test function name and the test target function name) in the test code 111 (S1751).

[0171] In addition, the test execution time estimation model generation unit 131 analyzes the test execution log 112 to generate information (hereinafter referred to as "second correspondence") indicating the execution result for each test function (information associating the test function name, test execution time, and test result (success / failure)) (S1752).

[0172] Next, the test execution time estimation model generation unit 131 generates the execution result for each test target function (information associating the test function name, test target function name, test execution time, and test result (success / failure)) based on the call relationship (first correspondence relationship) generated in S1751 and the information indicating the execution result for each test function (second correspondence relationship) generated in S1752 (S1753).

[0173] In addition, the test execution time estimation model generation unit 131 analyzes the source code 109 and generates information indicating the number of test cases required for each function of the source code 109 (information associating the function name with the number of test cases required for the function (number of required test cases)) (S1754).

[0174] Furthermore, the test execution time estimation model generation unit 131 performs the above-mentioned filtering (first filtering and second filtering, which corresponds to S1452 in FIG. 14B) on the execution results for each test target function generated in S1753, and generates valid execution results for each test target function (information associating the test function name, the test target function name, and the test execution time) (S1755).

[0175] The test execution time estimation model generation unit 131 performs the first filtering by referring to the test result (success / failure) of the execution result for each function to be tested. Also, the test execution time estimation model generation unit 131 performs the second filtering based on the information indicating the number of test cases required for each function generated in S1754.

[0176] Next, the test execution time estimation model generation unit 131 corrects the test execution time of the valid execution results for each function generated in S1754 based on the information indicating the number of test cases required for each function generated in S1755 (corresponding to S1453 in FIG. 14B), and generates a test execution time for each function (S1756).

[0177] 18 is a flowchart explaining the process (hereinafter referred to as "test execution time estimation process S1800") in which the test execution time estimation unit 132 calculates the test execution time used to calculate the applied cost and the unapplied cost by using the test execution time estimation model 114 generated by the test execution time estimation model generation process S1700 in FIG. 17A. The test execution time estimation process S1800 will be explained below with reference to this figure.

[0178] First, the test execution time estimation unit 132 acquires metrics values ​​for each function described in the source code for which the test execution time is to be estimated from the code information for each target software version 103 (S1811). Note that the above functions (functions used to estimate the test execution time) may be all functions described in the source code, or may be only some of the functions (for example, functions specified by the user via a user interface).

[0179] Next, the test execution time estimation unit 132 inputs the metrics values ​​of each function acquired in S1811 into the test execution time estimation model 114, and stores the test execution time for each function and the total test execution time for each function output by the test execution time estimation model 114 in the memory unit 110 as the test execution time estimation result 115 (S1812).

[0180] This completes the test execution time estimation process S1800.

[0181] <Effects> As described above, the evaluation device 100 of the second embodiment calculates the pre-refactoring cost 106 and the pre-refactoring cost 107 based on the time required to test the software (pre-refactoring software, pre-refactoring software). Therefore, it is possible to evaluate the effect of refactoring from the perspective of the time cost required to test the software.

[0182] Furthermore, the evaluation device 100 of the second embodiment calculates the test execution time for each function using the test execution time estimation model 114 learned from learning data 113 generated based on existing information (source code 109, test code 111, test execution log 112), and therefore can calculate the test execution time for each function efficiently and appropriately.

[0183] Furthermore, the evaluation device 100 of the second embodiment performs a first filtering process to exclude information acquired by executing the test code 111 for a function for which the test result of the test code 111 is a failure in the test execution log 112 from the data to be generated for the learning data 113. This improves the accuracy of prediction of the test execution time by the test execution time estimation model 114.

[0184] Furthermore, when the coverage of the test code 111 for the function to be tested is less than a preset first threshold, the evaluation device 100 of the second embodiment performs second filtering to exclude information acquired by executing the test code 111 for the function to be tested from the information to be generated for the learning data 113. This improves the accuracy of prediction of the test execution time by the test execution time estimation model 114.

[0185] Furthermore, when the coverage of the test code 111 for the function to be tested is less than a preset second threshold (≧first threshold), the evaluation device 100 of the second embodiment corrects the test execution time of the function to be tested according to the coverage and sets the corrected time as the objective variable of the learning data 113. This makes it possible to improve the prediction accuracy of the test execution time by the test execution time estimation model 114.

[0186] Furthermore, the evaluation device 100 corrects the above test execution time to the time required for testing assuming 100% coverage, thereby improving the accuracy of the test execution time prediction by the test execution time estimation model 114.

[0187] Moreover, the evaluation device 100 of the second embodiment analyzes the test code 111 to obtain a first correspondence, which is information indicating the correspondence between the test function described in the test code 111 and the test target function, which is the function to be tested by the test function, and analyzes the test execution log 112 to obtain a second correspondence, which is the correspondence between the test function and the test execution time, and determines the correspondence between the test target function and the test execution time of the test target function based on the first correspondence and the second correspondence. Therefore, for example, even if the test execution log 112 does not include information directly indicating the test target function, it is possible to obtain the correspondence (call relationship) between the test function and the test target function, which is necessary for generating the learning data 113.

[0188] Although the embodiments have been described above, the present invention is not limited to the above-described embodiments, and various modifications are included, and the present invention is not necessarily limited to those including all of the configurations described. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

[0189] For example, it is possible to determine both the effect of refactoring based on the metrics values ​​of the source code of unapplied software and applied software in the first embodiment, and the effect of refactoring based on the time cost required for software testing in the second embodiment, and evaluate the effect of refactoring comprehensively / from multiple perspectives based on both effects. [Explanation of symbols]

[0190] 1 Evaluation system, 2 User, 20 User terminal, 30 Maintenance history management device, 31 Code history information, 32 Maintenance difference information, 100 Evaluation device, 101 Target software specification information, 102 Refactoring method specification information, 103 Code information for each target software version, 104 Difference information between target software versions, 105 Code information for each applied software version, 106 Cost when not applied, 107 Cost when applied, 108 Refactoring effect, 111 Test code, 112 Test execution log, 113 Learning data, 114 Test execution time estimation model, 115 Test execution time estimation result, 121 User specification reception unit, 122 Maintenance history information acquisition unit, 123 Refactoring application unit, 124 Cost calculation unit, 125 Refactoring effect calculation unit, 126 Refactoring effect presentation unit, 131 Test execution time estimation model generation unit, 132 Test execution time estimation unit

Claims

1. A processor and a storage device are included. Source code for educational software, test code generated for said source code; And, a test execution log obtained by executing the test code; Remember, determining metric values ​​of functions described in the source code by analyzing the source code; determining a test execution time, which is a time required to test the function, by analyzing the test code and the test execution log; storing a test execution time calculation model, which is a machine learning model trained using training data in which the metric value is an explanatory variable and the test execution time is an objective variable; calculating a time required for execution of the software based on a time required for execution of a test for each function obtained by inputting a metric value for each function described in a source code of the software for which a time required for execution of a test is to be estimated into the test execution time calculation model; Information processing device.

2. 2. The information processing device according to claim 1, information acquired by executing the test code for the function for which a test result by the test code is a failure in the test execution log is excluded from the target for generating the learning data; Information processing device.

3. 2. The information processing device according to claim 1, determining the number of test cases required for the function by analyzing a source code of the training software; determining coverage of the test code for the function based on the number of required test cases; If the coverage is less than a preset first threshold, information acquired by executing the test code for the function is excluded from targets for generating the learning data. Information processing device.

4. 2. The information processing device according to claim 1, determining the number of test cases required for the function by analyzing a source code of the training software; determining coverage of the test code for the function based on the number of required test cases; if the coverage is less than a second threshold value set in advance, correcting the test execution time of the function in accordance with the coverage; Information processing device.

5. 5. The information processing device according to claim 4, correcting the test execution time of the function to a time required for testing when the coverage is set to 100%; Information processing device.

6. 6. The information processing device according to claim 3, The coverage is any one of C0 coverage, C1 coverage, C2 coverage, and path coverage. Information processing device.

7. 2. The information processing device according to claim 1, The metric value is a metric value in a Maintainability Index (MI), Information processing device.

8. 2. The information processing device according to claim 1, By analyzing the test code, a first correspondence relationship is obtained, which is information indicating a correspondence between a test function described in the test code and a test target function that is a function to be tested by the test function; acquiring a second correspondence between the test functions and the test execution times by analyzing the test execution log; obtaining a correspondence between the test target function and the test execution time of the test target function based on the first correspondence relationship and the second correspondence relationship; Information processing device.

9. A refactoring effect evaluation device that evaluates the effect of a refactoring applied to software, the refactoring effect evaluation device being configured using the information processing device according to claim 1, Multiple versions of the source code of the target software to be evaluated for the effectiveness of refactoring, A refactoring method which is information indicating the content of the refactoring; maintenance difference information indicating differences between versions of the source code; Remember, generating applied code, which is source code corresponding to each of the multiple versions of the source code, by applying refactoring by the refactoring method to the target software; calculating a cost for each maintenance not applied, which is a cost required for the maintenance performed in generating each of the source codes of the multiple versions of the target software, based on the source code of the multiple versions of the target software and the maintenance difference information; Calculating a per-maintenance application cost, which is a cost required if the maintenance performed in generating the corresponding version of the target software is performed on each of the applied codes; A cost when the maintenance is not applied is calculated as a cumulative total of the costs when the maintenance is not applied for each maintenance. Calculate the application cost, which is the cumulative total of the application costs for each maintenance, obtaining information based on a difference between the not-applied cost and the applied cost as an evaluation index for the effect of the refactoring; The non-application cost for each maintenance is calculated based on the time required to execute a test for each function, which is obtained by inputting a metric value for each function described in the source code of the target software into the test execution time calculation model; The cost per maintenance application is calculated based on the time required to execute a test for each function, which is obtained by inputting a metric value for each function described in the source code of the applied code into the test execution time calculation model. Refactoring effectiveness evaluation device.

10. An information processing device having a processor and a storage device, Source code for educational software, test code generated for said source code; And, a test execution log obtained by executing the test code; storing the determining metric values ​​of functions described in the source code by analyzing the source code; determining a test execution time, which is a time required to test the function, by analyzing the test code and the test execution log; storing a test execution time calculation model, which is a machine learning model trained using training data in which the metric value is an explanatory variable and the test execution time is an objective variable; And, a step of calculating a time required for execution of the software based on a time required for execution of a test for each function obtained by inputting a metric value for each function described in a source code of the software for which a time required for execution of a test is to be estimated into the test execution time calculation model; An information processing method.

11. 11. The information processing method according to claim 10, a step of excluding, by the information processing device, information acquired by executing the test code for the function for which a test result by the test code is a failure in the test execution log, from targets for generating the learning data; The information processing method further comprises:

12. 11. The information processing method according to claim 10, The information processing device, determining the number of test cases required for the function by analyzing a source code of the training software; determining coverage for the function of the test code based on the number of required test cases; And, If the coverage is less than a predetermined first threshold, excluding information acquired by executing the test code for the function from targets for generating the learning data; The information processing method further comprises:

13. 11. The information processing method according to claim 10, The information processing device, determining the number of test cases required for the function by analyzing a source code of the training software; determining coverage for the function of the test code based on the number of required test cases; if the coverage is less than a second threshold value set in advance, correcting the test execution time of the function in accordance with the coverage; The information processing method further comprises:

14. 11. The information processing method according to claim 10, The information processing device, acquiring a first correspondence relationship, which is information indicating a correspondence between a test function described in the test code and a test target function that is a function to be tested by the test function, by analyzing the test code; acquiring a second correspondence between the test functions and the test execution times by analyzing the test execution log; And, obtaining a correspondence between the test target function and the test execution time of the test target function based on the first correspondence relationship and the second correspondence relationship; The information processing method further comprises:

15. A method for evaluating the effectiveness of refactoring applied to software, which is performed using the information processing method according to claim 10, comprising the steps of: The information processing device, Multiple versions of the source code of the target software to be evaluated for the effectiveness of refactoring, A refactoring method which is information indicating the content of the refactoring; maintenance difference information indicating differences between versions of the source code; storing the generating applied code, which is source code corresponding to each of the multiple versions of the source code, by applying refactoring by the refactoring method to the target software; A step of calculating a cost for each maintenance not applied, which is a cost required for the maintenance performed in generating each of the source codes of the multiple versions of the target software, based on the source codes of the multiple versions of the target software and the maintenance difference information; determining a cost per application of maintenance, which is a cost required if maintenance performed in generating a version of the target software corresponding to each of the applied codes is performed; A step of calculating a cost before the maintenance is not applied, which is a cumulative total of the costs before the maintenance is not applied for each maintenance; A step of calculating an application cost which is a cumulative total of the application costs for each maintenance; obtaining information based on a difference between the not-applied cost and the applied cost as an evaluation index for the effect of the refactoring; A step of calculating the cost when the maintenance is not applied based on the time required for executing the test for each function, which is obtained by inputting a metric value for each function described in the source code of the target software into the test execution time calculation model; And, determining the cost per maintenance application based on the time required for executing a test for each function, the time being obtained by inputting a metric value for each function described in the source code of the applied code into the test execution time calculation model; A method for evaluating the effectiveness of refactoring.

Images