Program development support system and program development support method

Abstract

We provide a program development support system that enables the efficient use of resources. [Solution] Multiple functions to be tested are registered in an execution order queue. The automated test execution unit tests the function registered at the top of the execution order queue. A function association table associates each of the multiple functions to be tested with a submodule used when implementing the function or a test item to confirm the function's functionality. The execution order control unit has a function to change the order of testing of the functions registered in the execution order queue based on the function that did not meet the expected result and the function association table, if the test result from the automated test execution unit does not meet the expected result.

Description

Technical Field

[0006] , ,

[0001] The present invention relates to a program development support system and a program development support method.

Background Art

[0002] In recent years, test automation using a continuous integration (CI) system has become widespread in the program development field. In the development of web-based programs, CI systems are common, but CI systems are also beginning to be introduced in the development of device embedded programs.

[0003] Generally, the CI system operates triggered by a developer committing (uploading) source code to a software configuration management server (hereinafter referred to as an SCM server), and automatically executes program tests. When multiple developers develop multiple functions, the CI system is shared by multiple developers. Therefore, there is a possibility that multiple developers commit source code for realizing each of multiple functions at the same time. In such a case, generally, tests of the source code for realizing each function (hereinafter sometimes simply referred to as function tests) are executed in the order of commitment.

[0004] In the case of a mechanism that performs function tests in the order of commitment, the function tests committed later are not executed until the function tests committed earlier are completed. Even if a function to be tested first (a function with a high priority) is committed, the test of the function with a high priority is waited until the test of the function committed earlier is completed.

[0005] A technique for associating a priority attribute with each function and swapping the order of functions registered in an execution order queue so that functions with a high priority are tested first is known (Patent Document 1).

Prior Art Documents

Patent Documents

[0006] [Patent Document 1] Patent No. 7246301 [Overview of the project] [Problems that the invention aims to solve]

[0007] If the test results for a particular function do not meet the expected results, it is highly likely that testing other functions that use the same submodule used in the testing of that function will also result in test results that do not meet the expected results. Furthermore, if the expected results are not obtained for a test item of a particular function, the expected results will not be obtained by running the same test item on other functions. Thus, running tests on functions that are known to fail to meet the expected results leads to a decrease in resource efficiency.

[0008] The object of the present invention is to provide a program development support system and a program development support method that enable the effective use of resources. [Means for solving the problem]

[0009] According to one aspect of the present invention, An execution order queue where multiple functions to be tested are registered, An automated test execution unit that tests the function registered at the top of the execution order queue, A function association table is used to associate each of the multiple functions to be tested with the submodules used in the implementation of the function or with test items to verify the function's functionality. If the test results from the automated test execution unit do not meet the expected results, the execution order control unit has a function to change the order of testing of the functions registered in the execution order queue based on the function that did not meet the expected results and the function association table. A program development support system equipped with these features is provided.

[0010] According to another aspect of the present invention, For each function to be tested, create a function association table that associates it with the submodules used to implement the function or test items to verify the function's functionality. The function registered at the top of the execution order queue, in which multiple functions to be tested are registered in the order in which they will be tested, will be tested. A program development support method is provided that, if the test results do not meet the expected results, changes the order of testing of the functions registered in the execution order queue based on the function that did not meet the expected results and the function association table. [Effects of the Invention]

[0011] If the test results for a feature registered at the top of the execution order queue are likely to not meet the expected results, the test order can be changed to avoid running unnecessary tests. This allows for more efficient use of resources. [Brief explanation of the drawing]

[0012] [Figure 1] Figure 1 is a block diagram of the program development support system according to the first embodiment. [Figure 2] Figure 2 is a diagram showing an example of a function association table 16. [Figure 3] Figures 3A and 3B show examples of images displayed on the display screen 15A of the input device 15 operated by the software administrator. [Figure 4] Figure 4 is a flowchart showing the procedure executed by the execution order control unit 12. [Figure 5] Figures 5A and 5B show examples of information displayed on the display screen 26 of terminal 25 (Figure 1). [Figure 6] Figure 6 is a diagram showing an example of a function association table 16 of the program development support system according to the second embodiment. [Modes for carrying out the invention]

[0013] Referring to FIGS. 1 to 5B, a program development support system according to the first embodiment will be described. FIG. 1 is a block diagram of a program development support system according to the first embodiment. Each function of the program development support system is realized by a computer executing a program stored in a storage unit. A plurality of developers create source codes for realizing their respective functions using their respective terminals 25. In the SCM server 20, source codes for realizing each of the plurality of functions, common sub-modules used in the plurality of functions, and the like are accumulated. The SCM server 20 performs change history and version management of source codes for each function.

[0014] When the source code created for each function is committed (uploaded) from the terminal 25 to the SCM server 20, the plurality of functions are registered in the execution order queue 11 in the order in which they are committed. FIG. 1 shows an example in which source codes 30A, 30B, and 30C for realizing three functions A, B, and C, respectively, are committed to the SCM server 20. Functions A, B, and C are registered in this order in the execution order queue 11. Information for identifying a function can be extracted from a changed folder or file, for example, by a change history managed by the SCM server 20.

[0015] The automatic test execution unit 10 tests the function registered at the head of the execution order queue 11. In the example shown in FIG. 1, the automatic test execution unit 10 compiles the source code 30A for realizing the function A registered at the head and the source code of the common sub-module to create an executable file and executes it.

[0016] Next, the function association table 16 stored in the storage unit of the program development support system will be described. The function association table 16 associates a sub-module used when realizing each of the plurality of functions to be tested. The function association table 16 is created, for example, by a software administrator operating an input device 15 described later.

[0017] FIG. 2 is a chart showing an example of the function association table 16. Each row in FIG. 2 corresponds to a function, and each column corresponds to a sub-module. The circled symbols in the chart of FIG. 2 indicate that the sub-module is used to implement the function, and the hyphens indicate that the sub-module is not used. In the example shown in FIG. 2, sub-module X is used to implement functions A and B, and sub-modules Y and Z are not used. To implement function C, sub-modules Y and Z are used, and sub-module X is not used.

[0018] Next, the function association setting unit 13 (FIG. 1) will be described. When developing a new function, the software administrator operates the input device 15 to associate the new function to be tested with the sub-module used when implementing the function. Next, referring to FIGS. 3A and 3B, the operation of associating the function with the sub-module will be described.

[0019] FIGS. 3A and 3B are diagrams showing an example of an image displayed on the display screen 15A of the input device 15 operated by the software administrator. As shown in FIG. 3A, a chart associating functions A, B, C with sub-modules X, Y, Z is displayed, and further, "Function Addition", "Sub-module Addition", "Settings", and "Cancel" buttons are displayed. When the software administrator selects the "Function Addition" button, as shown in FIG. 3B, a row indicating the new function D is added, and check boxes are displayed at locations corresponding to each module.

[0020] The software administrator checks the check box corresponding to the sub-module used when implementing function D to associate function D with the sub-module. FIG. 3B shows an example of associating sub-module Z with function D. When the "Settings" button is selected, the function association setting unit 13 newly adds the association between function D and sub-module Z to the function association table 16 based on the input information.

[0021] When the "Add Submodule" button is selected, a column is added showing the new submodule, and checkboxes appear in the corresponding locations for each function. When the "Cancel" button is selected, the association process between the function and the submodule is canceled.

[0022] Next, the execution order control unit 12 (Figure 1) will be described. If the test results from the automated test execution unit 10 do not meet the expected results, the execution order control unit 12 changes the order of testing for the functions registered in the execution order queue based on the function that did not meet the expected results and the function association table 16.

[0023] Next, the procedure performed by the execution order control unit 12 will be described with reference to Figure 4. Figure 4 is a flowchart showing the procedure performed by the execution order control unit 12. When the test by the automatic test execution unit 10 is completed for one function, the procedure shown in Figure 4 is started. The execution order control unit 12 determines whether the test result meets the expected result (step S1). If the test result meets the expected result (OK), the process is terminated. If the test result does not meet the expected result (NG), the number of functions registered in the execution order queue 11 is determined (step S2).

[0024] If the number of functions registered in the execution order queue 11 is 1 or less, the process is terminated. If the number of functions registered in the execution order queue 11 is 2 or more, the function association table 16 is referred to and the submodule associated with the function that did not meet the expected result is extracted (step S3). For example, in the example shown in Figure 2, if the test for function A is performed and the expected result is not met, submodule X is extracted. Next, from the submodules extracted in step S3, the submodule that caused the failure to meet the expected result is determined (step S4).

[0025] If a faulty submodule is found in step S4, the parameter n is initialized (set to n=0) (steps S5, S6). After initializing parameter n, it is checked whether the nth function from the beginning of the execution order queue 11 is using the faulty submodule determined in step S4 (step S7). The beginning of the execution order queue 11 is defined as the 0th position. When parameter n is zero, it is checked whether the function registered at the beginning of the execution order queue 11 (function B in the example shown in Figure 1) is using the faulty submodule.

[0026] For example, in the example shown in Figure 2, if submodule X associated with function A is determined to be a faulty submodule, then since function B is also associated with submodule X, function B is determined to be associated with the faulty submodule X.

[0027] If the function registered as the nth item from the beginning of the execution order queue 11 is associated with a faulty submodule (step S8), the function registered as the nth item from the beginning is moved to the end of the execution order queue 11 (step S9). In other words, the order in which the function associated with the faulty submodule is tested is shifted down. In the example shown in Figures 1 and 2, function B, which uses the faulty submodule X of function A, is moved to the end of the execution order queue 11. As a result, function C is registered at the beginning of the execution order queue 11.

[0028] The system determines whether there are any unverified functions in the execution order queue 11. If there are, it checks again whether the nth function from the beginning of the execution order queue 11 is using the faulty submodule determined in step S4 (step S7). In the example shown in Figures 1 and 2, since function C, currently registered at the top of the execution order queue 11, is not using the faulty submodule X, the parameter n is increased by 1 without changing the order of the functions registered in the execution order queue 11 (steps S8, S10). Then, step S11 is executed.

[0029] If no faulty submodule is found in step S4, steps S6 through S9 and S10 are not executed, and the process terminates. An example of a case where the test result is determined to be NG in step S1 and no faulty submodule is found in step S4 is when there are faults in the processing before or after calling various submodules.

[0030] For example, in the example shown in Figure 2, function B, which is registered at the top of the execution order queue 11, is using the malfunctioning submodule. However, the following describes an example where function B is not using the malfunctioning submodule. In this case, 1 is added to n in step S10 without changing the order of the functions registered in the execution order queue 11. As a result, after executing the procedure in step S11, in step S7, processing is performed for function C, which is registered as the first (after the top) function in the execution order queue 11.

[0031] If there are no unverified functions in the execution order queue 11, the process terminates (step S11).

[0032] Next, the functions of the order change notification unit 14 (Figure 1) will be explained with reference to Figures 5A and 5B. Figures 5A and 5B show examples of information displayed on the display screen 26 of the terminal 25 (Figure 1).

[0033] As shown in Figure 5A, for example, the execution procedure for testing function A is divided into multiple stages I to V, and the tests are executed in the order of the stages. The automated test execution unit 10 (Figure 1) displays the execution status (the stage currently being executed) with a progress bar. Furthermore, the functions currently registered in the execution order queue 11 (Figure 1) are displayed in the order they were registered, along with the estimated time required for each test. This allows developers to know the approximate waiting time until the function they have developed is tested.

[0034] Once the test is complete, the test result will be displayed as "OK" or "NG" at the right end of the progress bar, as shown in Figure 5B.

[0035] When the execution order control unit 12 (Figure 1) changes the order of functions registered in the execution order queue 11 (Figure 1), the order change notification unit 14 (Figure 1) displays information on the display screen 26 of the terminal 25 notifying that the execution order has been changed, as shown in Figure 5B. Figure 5B shows an example where the execution order of function B has been moved to the end. Function C is displayed first, along with the message, "Due to a malfunction found in another function, the test execution order of this function has been moved forward." Function B is then displayed, along with the message, "Due to a malfunction found in another function, the test execution order of this function has been moved back. Please wait a moment for the test to run."

[0036] Alternatively, instead of displaying the message on the terminal 25's display screen 26 (Figures 5A and 5B), or in addition to displaying the message, an email containing the same message may be sent to the email address provided to the developer.

[0037] Next, we will describe the excellent effects of the first embodiment. In the first embodiment, if the test results from the automated test execution unit 10 (Figure 1) do not meet the expected results, the test order of the functions registered in the execution order queue is changed based on the function that did not meet the expected results and the function association table 16 (Figure 1). For example, the test order of the function that uses the defective submodule among the submodules associated with the function that did not meet the expected results is moved down. In other words, the test order of other functions is moved up.

[0038] Functions that use a faulty submodule are highly likely to experience the same fault even after testing. This results in wasted resources used for testing. In the first embodiment, the testing order of functions using the faulty submodule is postponed, thereby suppressing resource monopolization by tests that are highly likely to experience the same fault. This allows for efficient use of resources and reduces the need for lengthy testing times.

[0039] Next, a modified example of the first embodiment will be described. In the first embodiment, step S4 (Figure 4) determines the faulty submodule from the submodules extracted in step S3. However, depending on the fault, it may not be possible to identify the faulty submodule. In such cases, it is advisable to adopt all of the submodules extracted in step S3 as the faulty submodules.

[0040] In the first embodiment, in step S9 (Figure 4), the nth submodule using the faulty submodule was moved to the end of the execution order queue 11 (Figure 1), but the test execution order may be changed by other procedures. For example, the order of the function registered at the top (n=0) may be shifted back by one position. Changing the test execution order of the function registered at the top is equivalent to shifting the order of the function registered at the top down.

[0041] Next, a program development support system according to the second embodiment will be described with reference to Figure 6. The following description will omit explanations of components common to the program development support system according to the first embodiment, which was described with reference to Figures 1 to 5B.

[0042] Figure 6 is a diagram showing an example of a function association table 16 of the program development support system according to the second embodiment. In the first embodiment (Figure 2), the function association table 16 associates each of the multiple functions with the submodules used when implementing the functions. In contrast, in the second embodiment, each of the multiple functions is associated with test items for confirming the function's functionality.

[0043] The procedure performed by the execution order control unit 12 (Figure 1) of the program development support system according to Embodiment 2 can be performed by replacing "submodules" with "test items" in the flowchart shown in Figure 4. For example, in step S3 (Figure 4), test items associated with a function are extracted. In step S4 (Figure 4), the test item in which a defect occurred is determined from the extracted test items. In step S7 (Figure 4), it is checked whether the nth function from the beginning of the execution order queue 11 is associated with the test item in which the defect occurred.

[0044] The display screen 15A of the input device 15, shown in Figures 3A and 3B, displays functions A, B, and C, along with multiple test items, in a table format.

[0045] Next, we will describe the excellent effects of the second embodiment. Running the same test item that caused the malfunction is likely to result in the same malfunction occurring. This leads to wasted resources used for testing. In the second embodiment, the order in which the function associated with the malfunctioning test item is tested is shifted back, thus avoiding the monopolization of resources by tests that are likely to cause similar malfunctions. This allows for efficient use of resources and suppresses the prolongation of test time.

[0046] Next, a modified example of the second embodiment will be described. In the second embodiment, a function association table 16 (Figure 6) that associates functions with test items is used, but the function association table 16 (Figure 2) used in the first embodiment, which associates functions with submodules, may also be used. For example, in step S3 (Figure 4), submodules and test items associated with functions are extracted, and in step S4 (Figure 4), the submodule where the defect occurs and the defect test item are determined.

[0047] In step S7 (Figure 4), it is checked whether the nth function from the beginning of the execution order queue 11 is associated with a faulty submodule or a faulty test item. If the nth function is associated with at least one of the faulty submodule and the faulty test item, in step S9 (Figure 4), the nth function is moved to the end of the execution order queue 11.

[0048] When associating functions with test items, one may first associate submodules with test items, and then associate functions with test items through the submodules.

[0049] The embodiments described above are illustrative, and it goes without saying that partial substitution or combination of the configurations shown in different embodiments is possible. Similar effects and benefits from similar configurations in multiple embodiments will not be mentioned sequentially for each embodiment. Furthermore, the present invention is not limited to the embodiments described above. For example, it will be obvious to those skilled in the art that various modifications, improvements, and combinations are possible. [Explanation of Symbols]

[0050] 10. Automated Test Execution Unit 11 Execution Order Queue 12 Execution Order Control Unit 13 Function association setting section 14. Order Change Notification Unit 15 Input device 15A Input device display screen 16. Function Association Table 20. Shiftware Configuration Management Server (SCM Server) 25 devices 26. Device display screen Source code to implement functions 30A, 30B, and 30C

Claims

1. An execution order queue where multiple functions to be tested are registered, An automated test execution unit that tests the function registered at the top of the execution order queue, A function association table is used to associate each of the multiple functions to be tested with the submodules used in the implementation of the function or with test items to verify the function's functionality. If the test results from the automated test execution unit do not meet the expected results, the execution order control unit has a function to change the order of testing of the functions registered in the execution order queue based on the function that did not meet the expected results and the function association table. A program development support system equipped with the following features.

2. The program development support system according to claim 1, wherein the execution order control unit extracts the faulty submodule or faulty test item that caused the failure to meet the expected result from the submodule or test item associated with the function that failed to meet the expected result, and delays the order in which the function associated with the faulty submodule or faulty test item is tested among the functions registered in the execution order queue.

3. moreover, The program development support system according to claim 1 or 2, further comprising a function association setting unit that adds new associations between functions and submodules or test items to the function association table based on the input information.

4. Furthermore, the program development support system according to claim 1 or 2, further comprising an order change notification unit that notifies when the execution order control unit changes the order of functions registered in the execution order queue, that the order of execution has been changed.

5. For each function to be tested, create a function association table that associates it with the submodules used to implement the function or test items to verify the function's functionality. The function registered at the top of the execution order queue, in which multiple functions to be tested are registered in the order in which they will be tested, will be tested. A program development support method that, if the test results do not meet the expected results, changes the order of testing of functions registered in the execution order queue based on the function that did not meet the expected results and the function association table.

6. A program development support method according to claim 5, comprising extracting the faulty submodule or faulty test item that caused the failure to meet the expected result from the submodule or test item associated with a function that failed to meet the expected result, and shifting the order in which to test the function associated with the faulty submodule or faulty test item among the functions registered in the execution order queue.

7. The program development support method according to claim 6, wherein when the order in which to test a function associated with the defect occurrence submodule or the defect occurrence test item among the functions registered in the execution order queue is postponed, the developer is notified that the order has been postponed.

Images