Test execution support device and test execution support method

Abstract

A test execution support device 1 comprises: a waiting unit 14 that accumulates and holds, as needed, requests in a waiting state for testing a software program using a modified file; a monitoring unit 16 that, when the total amount of computational resources required to process a plurality of requests in the waiting state exceeds a threshold value, or when the deadline for processing the plurality of requests in the waiting state arrives, searches for a plurality of computers having the amount of free computational resources required to perform a plurality of test executions associated with the plurality of requests; a calculation unit 17 that calculates the environmental load amount generated by the test execution on each of the plurality of computers when the plurality of test executions are performed at the same time; and an output unit 18 that outputs computer information in ascending order of the environmental load amounts for the test executions.

Description

Test Execution Support Device and Test Execution Support Method

[0001] The present disclosure relates to a test execution support device and a test execution support method.

[0002] Reduction of the power consumption and carbon dioxide emissions of servers is required. However, CI (Continuous Integration) / CD (Continuous Delivery) tools used in the development of software programs consume a large amount of power (see Non-Patent Document 1). Therefore, methods for improving the efficiency of the CI / CD process have been proposed (see Non-Patent Documents 2 and 3).

[0003] The CI / CD tool is a tool for automating the testing of software programs. In CI, files such as modified code are automatically analyzed, built, and tested. In CD, the built software program is automatically deployed to the execution environment server.

[0004] Ameya Bhave, “Greening IT - Sustainable Software Architecture”, [online], [searched on September 2, Reiwa 6], <URL: https: / / www.linkedin.com / pulse / greening-sustainable-software-architecture-ameya-bhave> Milos Gligoric, et al. 2, “Practical Regression Test Selection with Dynamic File Dependencies”, Proc. ISSTA 2015, 2015, pp. 211 - 222 Jonathan Bell, et al. 1, “VMVM: Unit Test Virtualization for Java”, Proc. ICSE Companion’14, 2014, pp. 576 - 579

[0005] Server power consists of fixed power and power that fluctuates depending on the utilization rate of computing resources. Power efficiency can be improved by increasing the utilization rate of computing resources. However, conventionally, multiple tests were executed at arbitrary times regardless of the utilization rate of computing resources, resulting in fixed power consumption each time a test was run, thus consuming excessive power.

[0006] This disclosure is made in view of the above circumstances, and its purpose is to provide a technology that can help reduce server power consumption and carbon dioxide emissions during test execution on CI without impairing development efficiency.

[0007] A test execution support device according to one aspect of the present disclosure includes: a standby unit that continuously stores and keeps on standby requests for testing a software program with a modified file; a monitoring unit that searches for multiple computers with available computing resources necessary for multiple test executions related to the multiple requests when the total amount of computing resources required to process the multiple standby requests exceeds a threshold, or when the deadline for processing the multiple standby requests arrives; a calculation unit that calculates the amount of environmental load for each of the multiple computers generated when the multiple test executions are performed at once; and an output unit that outputs computer information in order of the lowest environmental load for each test execution.

[0008] A test execution support method according to one aspect of the present disclosure is a test execution support method performed by a test execution support device, in which requests for testing a software program with a modified file are accumulated and kept on standby as needed, when the total amount of computing resources required to process the multiple standby requests exceeds a threshold, or when the deadline for processing the multiple standby requests arrives, multiple computers with available computing resources required for the multiple test executions related to the multiple requests are searched for, the amount of environmental load for each of the multiple computers when the multiple test executions are performed at once is calculated, and computer information is output in order of the smallest environmental load for each test execution.

[0009] According to this disclosure, a technology is available that can help reduce server power consumption and carbon dioxide emissions during test execution on a CI (Computer Interface) without compromising development efficiency.

[0010] Figure 1 shows an example of the functional block configuration of the test execution support device. Figure 2 shows a test execution support method. Figure 3 shows an example of a received PR. Figure 4 shows an example of a PR in standby. Figure 5 shows an example of a classification of PRs in standby. Figure 6 shows the relationship between CPU usage and power consumption at deployment candidate 1. Figure 7 shows an example of the calculation results of power consumption and carbon dioxide emissions. Figure 8 shows an example of the hardware configuration of the test execution support device.

[0011] Embodiments of this disclosure will be described below with reference to the drawings. In the drawings, the same parts are denoted by the same reference numerals and their descriptions are omitted.

[0012] [Summary of this disclosure] This disclosure relates to the testing of software programs, particularly to reducing the environmental impact of CI (Computer Intelligence).

[0013] If a fixed power consumption is 200W and a variable power consumption of 10W is generated for every 10% increase in CPU usage, then running two separate tests at 10% CPU usage will consume 420W, while running them simultaneously in parallel at 20% CPU usage will consume 220W. Therefore, the latter consumes less power.

[0014] Therefore, this disclosure records the amount of computing resources required to run tests on file changes associated with pull requests (hereinafter referred to as PRs), and supports parallel and simultaneous processing of test execution by accumulating PRs until the amount is sufficient to efficiently utilize the computing resources of the execution environment server.

[0015] Furthermore, this disclosure aims to reduce power consumption and carbon dioxide emissions associated with test execution by supporting the selection of execution environment servers based on computing resource utilization and carbon intensity (carbon dioxide emissions per unit of energy consumption).

[0016] This disclosure reduces the total power and carbon dioxide emissions associated with unit tests across the entire project or organization, rather than the power consumption associated with individual program tests corresponding to file changes.

[0017] [Configuration of the Test Execution Support Device] In this embodiment, the device supports parallel and simultaneous processing of tests for changes to software program files that are of low urgency, and also supports the reduction of power consumption and carbon dioxide emissions associated with test execution by recommending the location of the test execution (location of the changed files and software program).

[0018] Figure 1 shows an example of the functional block configuration of the test execution support device 1 according to this embodiment.

[0019] As shown in Figure 1, the test execution support device 1 includes, as functional units, an input unit 11, a selection unit 12, a determination unit 13, a standby unit 14, a classification unit 15, a monitoring unit 16, a calculation unit 17, and an output unit 18.

[0020] Furthermore, as shown in Figure 1, the test execution support device 1 includes a dependency DB 21, an urgency criterion DB 22, an execution history DB 23, a local work history DB 24, and a deployment location information DB 25 as its storage unit.

[0021] The dependency database 21 has the function of associating and storing each file, such as code, that makes up a software program with various test cases used to test the software program in each file.

[0022] The urgency criteria DB22 has a function to store criteria information for determining the urgency of test execution.

[0023] The execution history DB23 has the function of storing past test execution history (for example, type of computing resource, resource usage, execution time) when a software program is tested on each file on multiple execution environment servers 3.

[0024] The local work history DB24 has the function of acquiring and storing the development test work history (for example, the type of computing resource, resource usage, and execution time) when the program is tested for each file from the local environment server 2 where the developer is working.

[0025] The deployment location information DB25 has the function of acquiring various information (for example, the amount of each computing resource installed, real-time resource usage, a function representing the relationship between resource usage and power consumption, and carbon intensity) from multiple execution environment servers 3 and storing it as deployment location information.

[0026] The input unit 11 has a function to accept PRs and waiting periods. A PR is a request to test execute a program in response to a file change.

[0027] The selection unit 12 has the function of referring to the dependency DB 21 and selecting the necessary test cases for the changes in the files associated with the received PR.

[0028] The determination unit 13 has the function of referring to the urgency criteria DB 22 and determining whether the received PR is highly urgent or not.

[0029] The standby unit 14 has a function to read the computational resources, resource usage, and execution time required for the test execution of PRs that have been determined to be of low urgency from past test execution history in the execution history DB 23 or from the development test work history in the local work history DB 24, and to store them in a waiting state associated with the PR, thereby putting the PR into standby mode.

[0030] The classification unit 15 has a function to classify the accumulated and standby PRs according to the main type of computing resources required for test execution.

[0031] The monitoring unit 16 has a function to search for one or more candidate deployment locations (execution environment servers 3) with available computing resources necessary for test execution from the deployment location information DB 25 when the total amount of computing resources required for each classified PR group exceeds a threshold, or when a predetermined date arrives.

[0032] The calculation unit 17 has a function to calculate the power consumption and carbon dioxide emissions for each candidate deployment location when test executions are performed all at once (parallel processing / simultaneous processing) at each candidate deployment location.

[0033] The output unit 18 has a function to output the name and ID of each candidate placement location, along with the calculated power consumption and carbon dioxide emissions for the test run, to a screen or the like, in order of increasing power consumption or carbon dioxide emissions.

[0034] Note that electricity consumption and carbon dioxide emissions are examples of environmental impact.

[0035] [Operation of the Test Execution Support Device] Figure 2 shows the test execution support method.

[0036] Step S1; The input unit 11 accepts PRs and waiting periods (completion deadlines). Each PR is linked to a task management ticket that describes the corresponding function changes, etc.

[0037] Step S2; The selection unit 12 refers to the dependency DB 21 and selects the necessary test cases for the changes in the function (file) associated with the received PR.

[0038] Step S3; The determination unit 13 refers to the urgency criteria DB 22 and determines whether the received PR requires a high level of urgency for test execution. If the test execution is deemed to be highly urgent, the test is executed immediately without performing this process.

[0039] Step S4; If the urgency of test execution is low, the standby unit 14 stores the received PR and the selected test case in relation to each other and waits for the PR to be processed.

[0040] Furthermore, the standby unit 14 obtains the main types of computing resources, resource usage, and execution time required for test execution in response to changes in the function (file) associated with the PR from the execution history DB 23, and records them in association with the PR. If there is no corresponding execution history, that is, if there is no execution history on the CI and it is the first time a test is being executed, the standby unit 14 obtains them from the local work history DB 24.

[0041] Describe specific examples of steps S3 and S4.

[0042] When the PR received in step S1 is as shown in FIG. 3, the determination unit 13 determines that "PJ1-1" with a due date within today has a high urgency. For "PJ1-2", "PJ1-3", and "PJ2-1" with due dates two days later or later, the waiting unit 14 accumulates each PR together with the test case, and obtains and records the type of main computing resource consumed, the resource usage amount, and the execution time for each from the execution history DB 23 (see FIG. 4). The test function shown in FIG. 4 is a test case.

[0043] Here, the due date of the task management ticket is used as an example for explanation, but the urgency may also be determined based on the number of tickets, release plans, etc.

[0044] Step S5; The classification unit 15 classifies the PRs according to the type of computing resource mainly used for test execution. For example, the classification unit 15 classifies the accumulated and waiting PRs into those mainly consuming the CPU and those mainly consuming the memory. Classification examples of the accumulated and waiting PRs are shown in FIG. 5. FIG. 5(a) is a group of PRs mainly consuming the CPU, and FIG. 5(b) is a group of PRs mainly consuming the CPU.

[0045] Step S6; The monitoring unit 16 determines whether the total resource usage amount of each PR group exceeds the threshold value or whether each PR group has passed the waiting period. If the total resource usage amount of each PR group does not exceed the threshold value or each PR group has not passed the waiting period, the process returns to step S1.

[0046] Step S7; For the PR group whose total resource usage amount has exceeded the threshold value or whose waiting period has passed, the monitoring unit 16 refers to the resource utilization status of each execution environment server (EES) 3 from the destination information DB 25, and narrows down a plurality of execution environment servers 3 where the change file, software program, and test case of the PR group are to be arranged as destination candidates.

[0047] Describe specific examples of steps S6 and S7.

[0048] For example, the resource utilization threshold is set to 40% of the default execution server for both CPU and memory. In this case, when the total CPU utilization of the PR group exceeds 40%, the monitoring unit 16 refers to the resource utilization status of all deployment locations from the deployment location information DB 25 and searches for a deployment location with available CPU resources necessary for the PR group's test execution as a candidate deployment location.

[0049] Step S8; The calculation unit 17 uses equations (1) and (2) to calculate the power consumption and carbon dioxide emissions associated with the test execution of the PR group when the PR group is placed in each candidate location.

[0050]

[0051]

[0052] For example, suppose destination candidate 1, destination candidate 2, and default execution server are found. If the PR group is placed at destination candidate 1, the CPU usage of destination candidate 1 will be 70%, with the PR group accounting for 40% of that. The relationship between CPU usage and power consumption at destination candidate 1 is shown in Figure 6. The carbon intensity of destination candidate 1 is 0.3 kg-CO2. ２ Let's assume the power consumption is / kWh. In this case, the calculation results for the power consumption and carbon dioxide emissions related to the test execution of the PR group are given by equations (3) and (4).

[0053]

[0054]

[0055] The calculation unit 17 performs calculations similarly for the candidate deployment location 2 and the default execution server.

[0056] Step S9; The output unit 18 displays the calculation results on the screen. The calculation results are shown in Figure 7. Figure 7(a) shows the calculation results for the PR group that mainly consumes CPU resources, and Figure 7(b) shows the calculation results for the PR group that mainly consumes memory resources. Multiple placement candidates are displayed in order of lowest power consumption and lowest carbon dioxide emissions, respectively.

[0057] Finally, we compared this to the case where each PR test is performed separately, as in the conventional technology. When the power consumption of the PR group that mainly consumes CPU is performed separately, the power consumption is calculated according to Figure 6 as 30W × 0.5h × 4 = 60Wh. Compared with the calculation results shown in Figure 7(a), in this embodiment, power consumption can be reduced by approximately 50%-70% compared to the conventional technology.

[0058] [Effects] As described above, according to this embodiment, PRs for testing the software program with the modified file are accumulated and kept on standby as needed. When the total value of resource usage required to process multiple PRs that are being accumulated and on standby exceeds a threshold, or when the deadline for processing multiple PRs that are being accumulated and on standby arrives, multiple candidate deployment locations with sufficient available resources for multiple test executions related to multiple PRs are searched for. When multiple test executions are performed at once, the power consumption and carbon dioxide emissions for each of the candidate deployment locations are calculated, and the deployment locations are output in order of lowest power consumption and carbon dioxide emissions for each test execution. Therefore, in test execution on CI, this technology can provide support for reducing server power consumption and carbon dioxide emissions without impairing development efficiency.

[0059] This allows for the estimation of the computing resources required for test execution, enabling the efficient use of computing resources by guiding the selection of test volumes and execution locations to facilitate parallel and simultaneous execution, thereby supporting power consumption optimization. For development projects or projects / organizations using the same test environment, it becomes possible to select execution environments with lower power consumption and carbon dioxide emissions during test execution on the CI (Computer Interface).

[0060] [Other] This disclosure is not limited to the embodiments described above. This disclosure can be modified in numerous ways within the scope of the gist of this disclosure.

[0061] The test execution support device 1 of this embodiment described above can be realized using a general-purpose computer system, for example, as shown in Figure 8, which includes a CPU 901, a memory 902, a storage 903, a communication device 904, an input device 905, and an output device 906. The memory 902 and the storage 903 are storage devices. In this computer system, the CPU 901 executes a predetermined program loaded onto the memory 902, thereby realizing each function of the test execution support device 1.

[0062] The test execution support device 1 may be implemented on a single computer. The test execution support device 1 may be implemented on multiple computers. The test execution support device 1 may also be a virtual machine implemented on a computer.

[0063] The program for the test execution support device 1 can be stored on a computer-readable recording medium such as an HDD, SSD, USB memory, CD, or DVD. A computer-readable recording medium is, for example, a non-transitory recording medium. The program for the test execution support device 1 can also be distributed via a communication network.

[0064] 1 Test execution support device 2 Local environment server 3 Execution environment server 11 Input unit 12 Selection unit 13 Judgment unit 14 Standby unit 15 Classification unit 16 Monitoring unit 17 Calculation unit 18 Output unit 21 Dependency DB 22 Urgency criterion DB 23 Execution history DB 24 Local work history DB 25 Deployment information DB 901 CPU 902 Memory 903 Storage 904 Communication device 905 Input device 906 Output device

Claims

1. A test execution support device comprising: a waiting unit that continuously accumulates and holds requests for testing a software program with a modified file; a monitoring unit that searches for multiple computers with available computing resources necessary for multiple test executions related to the multiple requests when the total amount of computing resources required to process the multiple pending requests exceeds a threshold, or when the deadline for processing the multiple pending requests arrives; a calculation unit that calculates the amount of environmental load for each of the multiple computers when the multiple test executions are performed at once; and an output unit that outputs computer information in order of the lowest environmental load for the test executions.

2. The test execution support device according to claim 1, wherein the standby unit obtains the amount of computing resources required to execute a test related to a pending request from past test execution history and stores the obtained amount of computing resources in association with the request.

3. The test execution support device according to claim 1, wherein the environmental load is power consumption and carbon dioxide emissions.

4. A test execution support method performed by a test execution support device, wherein requests for testing a software program with a modified file are accumulated and kept on standby as needed, when the total amount of computing resources required to process the multiple standby requests exceeds a threshold, or when the deadline for processing the multiple standby requests arrives, multiple computers with available computing resources required for the multiple test executions related to the multiple requests are searched for, the amount of environmental load for each of the multiple computers when the multiple test executions are performed at once is calculated, and the computer information is output in order of the lowest environmental load for each test execution.

Images