Software analysis systems, software analysis methods

Abstract

To quickly specify a version of software causing failure even when update histories of a plurality of pieces of software are individually managed.SOLUTION: In response to detecting degradation in performance of second software by a performance test, a software analysis system relating to the present invention specifies a candidate for first software causing the degradation by specifying a past version updated by merging the first software out of past versions of the second software or specifying reference information changed from the original, of the first software.SELECTED DRAWING: Figure 4B

Description

Technical Field

[0001] The present invention relates to a technique for analyzing the cause of software performance degradation.

Background Art

[0002] In an information system composed of a plurality of software, when performance degradation or a defect occurs in a software, it is possible that the cause is in another software. For example, an electronic control unit (ECU) that controls the operation of a vehicle operates in cooperation with a plurality of software such as application-side software (hereinafter referred to as ASW) and base system-side software (hereinafter referred to as BSW). For example, when performance degradation occurs in ASW, the cause may be due to the BSW side.

[0003] The following Patent Document 1 describes a technique for identifying the cause of a performance problem of a program. The technique described in this document is as follows: "The route determination device (1) includes a first difference extraction unit (21), a route extraction unit (22), and a determination unit (23). The first difference extraction unit (21) acquires a plurality of difference locations between the source code of the first program and the source code of the second program, and the first coverage information of the second program. The first difference extraction unit (21) refers to the first coverage information and extracts the first difference locations included in the first coverage information from the plurality of difference locations. The route extraction unit (22) extracts a plurality of affected routes (32) affected by the first difference locations from the source code of the second program for each first difference location (31). The determination unit (23) acquires the second coverage information of the second program and excludes the affected routes included in the second coverage information from the plurality of affected routes extracted by the route extraction unit (22)." (See the abstract).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

[0005] Software performance testing is not always performed frequently, and there may be intervals of a considerable period between tests. Conventional technologies, such as those described in Patent Document 1, seem to assume that the software changes causing performance degradation are relatively recent. In that case, if the software change that caused the problem occurred considerably earlier than the time of analysis, the changed parts may not be extracted as change differences, making it difficult to find the root cause.

[0006] Furthermore, if multiple software components within a system each have their own update history, the update history is managed separately for each software. Therefore, if performance degradation or other issues arise in software developed by a different developer, identifying the root cause can be difficult or time-consuming.

[0007] This invention has been made in view of the above problems, and aims to quickly identify the software version causing the malfunction, even when the update histories of multiple software programs are managed individually. [Means for solving the problem]

[0008] When a performance test detects a performance degradation in the second software, the software analysis system according to the present invention identifies a candidate for the first software causing the degradation by identifying a past version of the second software that was updated by merging with the first software, or by identifying a version of the first software whose reference information has been changed from the previous version. [Effects of the Invention]

[0009] According to the software analysis system of the present invention, even when the update histories of multiple software programs are managed individually, the software version causing the malfunction can be quickly identified. Other issues, configurations, and effects will be clarified in the following description of the embodiments. [Brief explanation of the drawing]

[0010] [Figure 1] This diagram schematically shows the software configuration installed in the ECU. [Figure 2] This is a schematic diagram illustrating continuous integration (CI) of software. [Figure 3] This diagram illustrates the challenges of having multiple software programs each under their own individual version control. [Figure 4A] This is a conceptual diagram illustrating the procedure for searching for the source of a malfunction in Embodiment 1. [Figure 4B] This is a conceptual diagram illustrating the procedure for searching for the source of a malfunction in Embodiment 1. [Figure 5] This is a conceptual diagram illustrating the procedure for searching for problematic areas on the BSW side using conventional binary search. [Figure 6A] This is a conceptual diagram illustrating the procedure for searching for the source of a defect in the software analysis method according to Embodiment 2. [Figure 6B] This is a conceptual diagram illustrating the procedure for searching for the source of a defect in the software analysis method according to Embodiment 2. [Figure 7] This is a diagram showing the configuration of the software analysis system 1 according to Embodiment 4. [Figure 8] This is a flowchart explaining the operation of the analytical device 200. [Figure 9] This is a flowchart explaining the operation of the analytical device 100. [Figure 10] This is an example of a screen interface that visualizes and presents the results of identifying the problematic areas causing performance degradation. [Modes for carrying out the invention]

[0011] <Embodiment 1> Figure 1 schematically shows the software configuration of an ECU. An ECU, for example, includes application-side software (ASW) and underlying system-side software (BSW). ASW and BSW can provide functions to each other (or one to the other) via a suitable interface. For example, ASW can utilize functions provided by BSW as software functions.

[0012] Figure 2 is a schematic diagram illustrating continuous integration (CI) of software. In CI, typically, a source code change history (e.g., the full source code and markings of the differences for each change) is accumulated within a source code version control tool. Registering source code in the version control tool (confirming the version at that time) is called a commit. Along with commits, regression tests and performance tests may be performed by running automated testing tools. These tests include static analysis of the source code and building (compiling the source code into an executable format) and actually running the program. The test results are notified to stakeholders (developers, etc.).

[0013] The above CI mechanism, through automated testing, reduces the time required to check for defects while ensuring that defects are reliably reported in the test results and maintaining consistent test quality by eliminating subjectivity in testing. However, some test items, such as performance tests, can be time-consuming, making it difficult to perform them after every commit. Therefore, they are typically performed at certain intervals (e.g., every two weeks, every month). For such test items, this time interval is necessary before defects are discovered. Traditional software analysis methods are considered to have room for improvement in this regard.

[0014] FIG. 3 is a diagram for explaining problems in the case where a plurality of software are each individually under version control. A version control tool generally has a function of searching for the source code of a problematic version (commit number) by, for example, binary search. In the example of FIG. 3, the source code committed first (C1) had no problem in the test result (good), but the source code committed the ninth time had a problem (bad). The version control tool retrieves and tests past versions (typically intermediate versions between C1 and C9, C5 in FIG. 3)追溯 from C9. If the test result is okay, the same test is performed for an intermediate version ahead (newer) than C5 (C7 in FIG. 3). By repeating this, it is possible to identify the oldest of the versions with a test problem (that is, the version causing the problem in C9). The test at this time may be either automatic or manual.

[0015] Searching for a problematic version by binary search as in FIG. 3 can efficiently identify the problem location. On the other hand, assume a case where a plurality of software operate in conjunction and each software is individually under version control. In this case, when a defect is discovered during testing of the software being developed, it is possible that the cause of the defect is in another software. Therefore, it is necessary to search within the version control tool of another software to identify the problem location. However, a method for efficiently searching for the problem location across version control tools has not been fully studied so far.

[0016] In view of such problems, the software analysis method according to Embodiment 1 of the present invention provides a method for efficiently searching for the cause location when the results of performance tests or the like are problematic in the case where a plurality of software operating in conjunction are each individually under version control.

[0017] Figures 4A and 4B are conceptual diagrams illustrating the procedure for investigating the cause of a malfunction in this embodiment. The developer has developed the ASW shown in Figure 1, and the ASW utilizes the functions provided by the BSW. The ASW and BSW are version-controlled independently of each other by separate version control tools (or separate instances within the same version control tool). Here, we assume a scenario in which a performance degradation is discovered during a performance test of the ASW, and the cause lies on the BSW side.

[0018] When ASW utilizes features from BSW, it may be necessary to incorporate BSW before building or testing ASW. For example, a specific version of the BSW source code (or compiled binaries, class libraries, etc., that enable BSW functionality during ASW execution) may be incorporated into the ASW development environment. The correspondence between the source BSW version and the target ASW version can be predetermined in the development plan. In this example, it is assumed that ASW commit C2 incorporates BSW commit C1 (shown in Figure 4B), and ASW commit C6 incorporates BSW commit C5 (shown in Figure 4B).

[0019] At the start of Figure 4A, it is assumed that the initial commit performed in the past passed the test without problems (good), and that a problem was found in the performance test performed during the ninth commit (bad). The source code version that caused the problem is searched for on the BSW side using the procedure described below. The search process on the ASW side is performed by the second analyzer, and the search process on the BSW side is performed by the first analyzer. The configuration of these analyzers will be described later. On the BSW side, as described later, it is assumed that commit C1 of the BSW had good performance test results.

[0020] In the first performance test, the second analysis unit identifies past versions of the ASW source code that incorporate BSW. In this case, commits C2 and C6 correspond to these. The second analysis unit then performs a performance test on the oldest of these (C2). It is assumed that the test results were favorable.

[0021] In the second performance test, the second analyzer will perform a performance test on the most recent version of the BSW that has been imported (in this case, since there are only two versions that have imported the BSW, C2 and C6, C6 is the most recent). It will be assumed that the test results were problematic. The performance test performed on the ASW side will only be for the version that imported the BSW. Therefore, the performance test on the ASW side is now complete.

[0022] On the ASW side, the test targets are limited to versions that incorporate BSW, but the criteria for selecting the search targets (test targets) within that range are based on binary search. In other words, the second analysis device uses binary search to select the ASW source code to be tested. For example, in Figure 4A, if there are past versions that incorporate BSW other than C2 and C6, the target of performance testing will be determined for those as well, using the same method as binary search.

[0023] In the third performance test (Figure 4B), the first analysis device performs a performance test on the latest version of the BSW source code that was incorporated into the ASW (in this case, commit C5 of the BSW). Here, we assume that the test results were problematic. In this case, we can infer that the cause of the problem occurred at some point between commits C2 and C5 on the BSW side. The following search procedure is the same as for binary search.

[0024] In the fourth performance test, the first analyzer performs a performance test on an intermediate version (in this case, C3) of commits C2-C4 in the BSW. Assume the test results are favorable. In the fifth performance test, the first analyzer performs a performance test on commit C4, which has not yet been tested among C2-C5. Assume the test results are problematic. Thus, the version C4 causing the problem on the BSW side has been identified.

[0025] Figure 5 is a conceptual diagram illustrating the procedure for searching for problem areas on the BSW side using conventional binary search. On the ASW side, the second analyzer searches for problem areas using a binary search starting from commit C9. Here, performance tests are performed in the order of C9=>C5=>C7=>C6. The procedure is the same as in Figures 4A and 4B. Unlike Figure 4A, the search target on the ASW side is not limited to the version merged with the BSW, so the performance test at C6 is performed for the third time. Therefore, since the number of performance tests performed is greater than in Figures 4A and 4B, it takes more time to find the problem areas. In this respect, this embodiment is advantageous over conventional binary search.

[0026] <Embodiment 1: Supplement> Unlike the above, if the performance test result for C2 is poor in the first performance test in Figure 4A, then it can be identified at that point that C2 is the oldest defective part on the ASW side. Therefore, the next step is to perform a performance test on commit C1 on the BSW side. If commit C1 on the BSW is found to be problematic, then that is the cause of the problem; if commit C1 on the BSW is not problematic, then commit C2 on the ASW is the cause of the problem. The software analysis system 1 (for example, one of the search units) outputs an estimated result to that effect.

[0027] Unlike the above, if the performance test result for C6 is good in the second performance test in Figure 4A, it means that the first problem on the ASW side occurred in one of the ASW commits C7 to C9. Since these do not incorporate the BSW, the cause of the problem lies on the ASW side. The software analysis system 1 (for example, one of the search units) outputs an estimated result to that effect. In this case, there is no need to search for the cause across version control tools, so the problem addressed by the present invention does not arise.

[0028] Unlike the above, if the performance test results for C5 are good in the third performance test in Figure 4B, it means there is no problem on the BSW side. This is because when the problem-free BSW source code C5 was incorporated into ASW's C6, ASW's C6 was found to have a problem. Software analysis system 1 (for example, one of the search units) outputs an estimated result to that effect.

[0029] <Embodiment 1: Summary> The software analysis method according to this embodiment automatically searches for the version causing performance degradation on the BSW side, using a method similar to binary search. This eliminates the reliance on individual skills for identifying problem areas and allows for the identification of problem areas without relying on the analyst's analytical skills.

[0030] The software analysis method according to this embodiment, when identifying the commits to be searched on the ASW side, uses a method similar to binary search, but only searches versions that incorporate the BSW. This allows the search target to be narrowed down, thus reducing the time required to identify the problem area compared to conventional methods. For example, while the conventional method shown in Figure 5 requires 6 performance tests, the embodiment shown in Figures 4A to 4B requires only 5.

[0031] The software analysis method according to this embodiment starts with the version in which the ASW (Application Software) incorporates the BSW (Web Software), and then searches for past versions of the BSW. This allows for the efficient identification of problem areas across version control tools, even when the ASW and BSW are individually version-controlled.

[0032] The software analysis method according to this embodiment does not require performance testing to be performed every time the source code is changed (each commit). If a problem is found in the performance test results at a certain point in time, it is sufficient to identify the problematic area by working backward from that point. For example, in Figures 4A and 4B, if a performance problem is found at commit C9, it is sufficient to perform performance testing by working backward to previous versions. Therefore, it is not necessary to prepare a physical device (e.g., an ECU in an in-vehicle system) for each performance test.

[0033] <Embodiment 2> Figures 6A and 6B are conceptual diagrams illustrating the procedure for searching for the source of a defect in the software analysis method according to Embodiment 2 of the present invention. In this embodiment, in addition to the procedure described in Embodiment 1, profile information of functions called within the ASW is used. Profile information is information that describes the statistics of the frequency of function calls, and if a version control tool has a function to acquire such statistics, this can be used. Alternatively, the calls of each function may be recorded during test execution. The rest is generally the same as Embodiment 1, so the prerequisites and other details are the same as in Embodiment 1.

[0034] At the start of Figure 6A, the second analyzer acquires profile information for ASW commit C9 (the latest version in which problems were found through performance testing). For convenience, function names provided by ASW are assumed to start with "asw_", and function names provided by BSW are assumed to start with "bsw_".

[0035] In the first performance test, the second analysis device tests commit C2 in the same manner as in Figure 4A. Here, we assume that the test results were satisfactory. The second analysis device obtains profile information for commit C2. Comparing the profile information for C9 and C2, we find that the call frequency of the function bsw_func2 changes significantly between C2 and C9, with an increase in frequency in C9. This suggests that this function may be the cause of the performance degradation in C9.

[0036] In the second performance test (Figure 6B), the first analyzer performs a performance test on the most recent version of the BSW (C5 in Figure 6B) imported into the ASW. Here, we assume that the test results were problematic. In this case, it is assumed that the cause of the performance degradation on the ASW side lies on the BSW side, and according to the profile information, the function bsw_func2 is presumed to be the cause. The first analyzer obtains the change history of the function bsw_func2 for BSW commits prior to commit C5. Here, we assume that there were no changes in commits C1 to C3, and that the function was modified in commit C4.

[0037] In the third performance test, the first analyzer performs a performance test on commit C4, which includes a modification to the function bsw_func2. Let's assume the test results are problematic. Therefore, this version is likely the cause of the performance degradation, and a detailed investigation of this version (especially the changes to the function bsw_func2) should be conducted.

[0038] In Figure 6A, the function bsw_func2 was estimated to be the cause of the performance degradation. However, depending on the structure of the function, the calling function that called it may also be the cause of the performance degradation. For example, suppose the calling function previously called a different function, but after a certain version, it was changed to call bsw_func2, and as a result, the frequency of calls to bsw_func2 on the ASW side increased. In this case, even though there is no problem with bsw_func2, the profiling information makes it appear as if the function is the cause of the degradation, which is not a valid analysis result. Therefore, in this embodiment, trace information may be additionally used to improve the accuracy of the search in such cases.

[0039] Trace information describes the order in which ASW (or BSW) calls functions. In the example shown in Figure 6B, the first analyzer obtains trace information for commit C5 during the second performance test. According to the trace information, the function bsw_func2 is called from the function bsw_init. Therefore, the first analyzer obtains the change history of bsw_init prior to commit C5. Here, we assume that there were no changes in commits C1 to C3, and that the function was modified in commit C4.

[0040] In the third performance test, the first analysis instrument performs a performance test on commit C4, which modifies the function bsw_init. Let's assume the test results are problematic. Therefore, this version is likely the cause of the performance degradation, and a detailed investigation of this version (especially the changes to the bsw_init function) should be conducted.

[0041] Investigating the change history of bsw_init is only necessary if there is no history of changes to bsw_func2. In that case, it is assumed that bsw_func2 is not the cause of the degradation. Alternatively, the change history of bsw_init may be investigated only if performance tests were successful in the version in which bsw_func2 was changed.

[0042] The software analysis method in this second embodiment requires only three performance tests, thus further reducing the number of tests compared to the first embodiment. Compared to the conventional method described in Figure 5, half the number of tests is sufficient.

[0043] <Embodiment 3> In Embodiment 2, it was explained that profile information or trace information is used as the basis for estimating the cause of ASW performance degradation. This is because changes in the order of function calls can cause performance degradation. Other parameters that can cause performance degradation include the following:

[0044] (Example of a causal parameter, part 1) When compiling source code, compilation options are specified. For example, optimization options may specify which of the following should be prioritized: reducing compilation time, reducing binary size, or improving execution performance. Changing compilation options can cause performance degradation at runtime.

[0045] (Example of a causative parameter #2) Changing the memory map (information about where to place binaries in memory) or the mutual exclusion level on the OS (Operating System) running the software can cause performance degradation during execution.

[0046] The first analysis device may use at least one of the two parameters described above, either in place of or in combination with the trace information described in Embodiment 2, when identifying past versions of the BSW to be tested. For example, if the test result for C5 in Figure 6B is problematic, the device obtains the time when the compilation options were changed among commits C2 to C4 via the change history on the version control tool. OS parameters can be obtained in the same way. The first analysis device performs a performance test on the commit at the time the parameters were changed. The process thereafter is the same as in Embodiment 2.

[0047] <Embodiment 4> Figure 7 is a diagram showing the configuration of a software analysis system 1 according to Embodiment 4 of the present invention. The software analysis system 1 is a system that implements the software analysis methods described in Embodiments 1 to 3, and is composed of an analysis device 100 and an analysis device 200. The analysis device 100 is a device that searches for commits on the BSW side and also serves as the BSW development environment. The analysis device 200 is a device that searches for commits on the ASW side and also serves as the ASW development environment.

[0048] The analysis device 200 (second analysis device) comprises an ASW repository 210 (second repository), a source code acquisition unit 220, a search unit 230 (second search unit), a BSW merge information acquisition unit 240, a test execution unit 250, a questionable commit information transmission unit 260, and a profile acquisition unit 270. The ASW repository 210 is a database in which the ASW version control tool stores past versions of the ASW source code. The source code acquisition unit 220 acquires each version of the ASW source code from the ASW repository 210. The search unit 230 performs a search for the test target version (ASW side) using binary search as described in Embodiments 1 to 3. The BSW merge information acquisition unit 240 identifies the commit that incorporated BSW from the past versions of ASW. The test execution unit 250 performs a performance test of ASW. The profile acquisition unit 270 acquires the profile information described in Embodiment 2. The doubt commit information transmission unit 260 notifies the analysis device 100 if there is a function in the profile information whose call frequency has changed by more than a certain threshold value (for example, in Figure 6A, C2 is ranked 10th, but in C9, the rank has risen by 5 or more places).

[0049] The analysis device 100 (first analysis device) comprises a BSW repository 110 (first repository), a source code acquisition unit 120, a search unit 130 (first search unit), an OS information acquisition unit 140, a test execution unit 150, and a trace acquisition unit 160. The BSW repository 110 is a database in which the BSW version control tool stores past versions of the BSW source code. The source code acquisition unit 120 acquires each version of the BSW source code from the BSW repository 110. The search unit 130 performs a search for the test target version (BSW side) using binary search, as described in Embodiments 1 to 3. The OS information acquisition unit 140 acquires OS parameters and their changes, as described in Embodiment 3. Compilation options and their changes may also be acquired. The test execution unit 150 performs a performance test of the BSW. The trace acquisition unit 160 acquires trace information, as described in Embodiment 3.

[0050] Figure 8 is a flowchart illustrating the operation of the analyzer 200. Here, we will describe the procedure described in Embodiment 2 as an example. This flowchart is performed starting from the beginning of Figure 4A (or Figure 6A). The following describes each step in Figure 8.

[0051] (Figure 8: Steps S801-S803) The BSW merge information acquisition unit 240 identifies the commit from past versions of ASW that incorporated BSW (S801). The search unit 230 determines the search target on the ASW side (the version to be investigated for the presence or absence of performance degradation causes by performing tests) (S802). Specifically, it determines the past versions of ASW that incorporated BSW as the search target. If the search target can be determined, proceed to S804 (S803:Yes); otherwise, skip to S809 (S803:No).

[0052] (Figure 8: Steps S804-S807) The source code acquisition unit 220 acquires the ASW source code of the target commit (S804). The test execution unit 250 sets any profile information that needs to be set prior to the test (S805) and then performs the test on the target commit (S806). Profile information acquisition can also be performed in S805. The test execution unit 250 performs S802 to S806 while changing the target commit until the test result changes (S807). Specifically, it performs a search in the manner of a binary search until it identifies the target commits that have no problems and those that have problems in the test results.

[0053] (Figure 8: Steps S808-S809) The search unit 230 performs steps S802 to S807 until it can identify a commit in which the call frequency in the profile information has changed by more than a certain threshold compared to that in C9 (S808). In the example in Figure 6A, steps S802 to S807 are performed until commit C2 is identified in which the call frequency of the function bsw_func2 has changed significantly compared to that in C9. The questionable commit information transmission unit 260 notifies the analysis device 100 of the fact that a commit has been identified and the name of the function in which the call frequency has changed significantly (S809).

[0054] (Figure 8: Step S809: Supplement) Embodiment 1: As explained in the supplement, there are cases where the search on the ASW side is completed and there is no need to perform any further search on the BSW side. In this case, the operation of the software analysis system 1 can be terminated by determining in S803 that there are no commits to search and notifying the BSW side of this fact in S809.

[0055] Figure 9 is a flowchart illustrating the operation of the analysis device 100. Here, as an example, the operation on the BSW side corresponding to Figure 8 (an example of operation using trace information) is explained. This flowchart starts in Embodiment 2 when the repository to be searched is switched from the ASW side to the BSW side (the second performance test in Figure 6B).

[0056] (Figure 9: Steps S901-S902) The search unit 130 determines the target of the search on the BSW side (the version to be investigated for the presence or absence of performance degradation through test execution) (S901). Specifically, when S901 is performed for the first time, the BSW version incorporated into the ASW version for which the test results were problematic is used as the starting point of the search. From the second time onward, the search for the version for which the test results were problematic is performed in the manner of a binary search. If a target commit is found, the search proceeds to S903; otherwise, it skips to S908 (S902).

[0057] (Figure 9: Steps S903-S907) The source code acquisition unit 120 acquires the BSW source code of the target commit (S903). The test execution unit 150 sets any trace information that should be set prior to the test (S904) and then performs the test on the target commit (S905). Trace information acquisition can also be performed in S904. The test execution unit 150 identifies the commit with the problem based on the test results (commit C5 in Figure 6B) and then performs S901 to S905 until it identifies the point in time when the calling function in the trace information (bsw_init in Figure 6B) was changed in a previous commit (S906). The search unit 130 sets the next target commit to search as the commit at the time when the calling function changed (S907). In Figure 6B, commit C4 is the next target of the search.

[0058] (Figure 9: Step S908) The search unit 130 identifies the problematic area (problematic version) on the BSW side that is causing performance degradation through the above procedure. The search unit 130 outputs data describing the identified problematic area, a screen interface, etc.

[0059] Figure 10 is an example of a screen interface that visualizes and presents the results of identifying the problematic area causing the performance degradation. The search unit 130 or 230 can visualize the problematic area (the version causing the problem on the BSW side) identified by the above procedure and the search process, and present it, for example, as a screen interface. Specifically, the visualization may display Figures 4A to 4B, Figures 6A to 6B as they are, or these processes may be presented in chronological order. Alternatively, data describing the visualization results may be output so that it can be reproduced on another device.

[0060] <Regarding variations of the present invention> The present invention is not limited to the embodiments described above, and includes various modifications. For example, the embodiments described above are described in detail to make the present invention easier to understand, and are not necessarily limited to those having all the configurations described. Furthermore, it is possible to replace parts of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add configurations from other embodiments to the configuration of one embodiment. In addition, it is possible to add, delete, or replace parts of the configuration of each embodiment with other configurations.

[0061] In the embodiments described above, an example of searching for source code versions using binary search was explained, but the present invention can also be applied when using other search methods. That is, by narrowing down the versions to be searched using the method described in the embodiments above, the same effects as in the embodiments can be achieved with search methods other than binary search.

[0062] In the embodiments described above, each functional unit of the analytical apparatus 100 and 200 can be configured by hardware such as circuit devices that implement the function, or by software that implements the function being executed by a computing device such as a CPU (Central Processing Unit).

[0063] In the embodiments described above, an ECU was used as an example of a system composed of multiple software components, but the present invention can also be applied to other software systems. [Explanation of Symbols]

[0064] 1: Software Analysis System 100: Analyzer 110: BSW Repository 130: Search Department 200: Analyzer 210: ASW Repository 230: Search Department

Claims

1. A software analysis system comprising a first analysis device for performing a performance test on first software, and a second analysis device for performing a performance test on second software, The first analytical apparatus is A first repository that stores software information, including update information for the first software, The system includes a first search unit that searches for a plurality of first software programs stored in the first repository based on update information of the first software, The second analytical apparatus is A second repository that stores software information, including update information for the second software, The system includes a second search unit that searches for a plurality of second software programs stored in the second repository based on update information of the second software, The second search unit is, If a performance degradation of the second software is detected by performing a performance test of the second software, Search the update information in the second repository, merge the update information of the first software to identify one or more older versions of the second software that have been updated, perform tests on the identified older versions of the second software, and identify the first software that contributed to the performance degradation of the second software. Alternatively, the update information in the second repository may be searched to identify one or more older versions of the second software and the first software in which the reference information of the first software has been updated. The information of the identified first software is transmitted to the first analyzer, The first search unit is, Based on the information of the identified first software transmitted from the second search unit, one or more older versions of the first software stored in the first repository are identified. By performing a performance test on the older version of the first software, the first software that has experienced performance degradation can be identified. A software analysis system characterized by the following features.

2. The second search unit performs tests only on older versions of the second software that have been updated by merging the update information of the first software, thereby identifying older versions of the second software that have degraded performance and versions that have not degraded performance. The first search unit identifies the versions of the first software that have been merged with the performance-degraded version and the performance-non-degraded version of the second software identified by the second search unit as the two ends of the first software version to search from the first repository. The first search unit identifies a version of the first software with degraded performance by performing performance tests on the versions of the first software located between the identified ends. The software analysis system according to claim 1, characterized in that it is as described above.

3. The software analysis system estimates that if an older version of the second software, which has been updated by merging the update information of the first software, does not experience performance degradation, then the cause of the performance degradation in the second software lies within the second software itself. The software analysis system according to claim 2, characterized in that it is as described above.

4. The software analysis system estimates that if neither of the versions of the first software at either end identified by the first search unit shows any performance degradation, the cause of the performance degradation in the second software lies within the second software itself. The software analysis system according to claim 2, characterized in that it is as described above.

5. The second search unit obtains statistics on the frequency of function calls made by the second software, The second search unit identifies, among the functions included in the statistics, those provided by the first software, and whose call frequency differs by a predetermined value or more between versions of the second software that have degraded performance and those that have not, as candidate functions for causing degradation. Based on the update history of the candidate degradation cause function, the first search unit identifies the version of the first software between the two ends in which the candidate degradation cause function has been changed from the previous version as a candidate cause of performance degradation in the second software. The software analysis system according to claim 2, characterized in that it is as described above.

6. The first search unit obtains trace information describing the order in which the first software calls functions, The first search unit identifies the calling function that invokes the candidate function for the cause of degradation based on the trace information, The first search unit identifies, from the update history of the calling function, versions of the first software between the two ends in which the calling function has been changed from the previous version as candidate causes of performance degradation in the second software. The software analysis system according to claim 5, characterized in that it is as described above.

7. If the candidate function causing degradation has been changed from the previous version in the first software version between the two ends, the first search unit searches for the first software version in which the candidate function causing degradation has been changed from the previous version, without searching for the first software version in which the calling function has been changed from the previous version. The first search unit searches for a version of the first software in which the calling function has been changed from the previous version, if the candidate function for the cause of degradation has not been changed from the previous version in the version of the first software between the two ends. The software analysis system according to claim 6, characterized in that it is as follows.

8. The reference information is at least one of the following: change information for the system call functions of the first software, change information for the OS referenced by the first software, or change information for the compiler of the first software. The software analysis system according to claim 1, characterized in that it is as described above.

9. The first search unit obtains at least one of the following as reference information: change information describing a change in the OS settings referenced by the first software, or change information describing a change in the compilation options of the compiler of the first software. The first search unit identifies, among the versions of the first software located between the two ends, those in which the change information describes a change in the OS settings or compilation options from the previous version, as candidate causes of performance degradation in the second software. The software analysis system according to claim 2, characterized in that it is as described above.

10. At least one of the first or second search unit outputs a result that visualizes at least one of the process of the search or the result of the test. The software analysis system according to claim 1, characterized in that it is as described above.

11. The first and second search units identify the first and second software by binary search. The software analysis system according to claim 1, characterized in that it is as described above.

12. A software analysis method comprising a first analysis device for performing a performance test on first software and a second analysis device for performing a performance test on second software, wherein the software is analyzed using these devices. The first analytical apparatus is Based on the update information of the first software, the following steps are performed to search for multiple first software programs stored in a first repository that stores software information including the update information of the first software: The second analytical apparatus is Based on the update information of the second software, the process involves searching for multiple second software programs stored in a second repository that stores software information including the update information of the second software. In the step of exploring the second software, If a performance degradation of the second software is detected by performing a performance test of the second software, Search the update information in the second repository, merge the update information of the first software to identify one or more older versions of the second software that have been updated, perform tests on the identified older versions of the second software, and identify the first software that contributed to the performance degradation of the second software. Alternatively, the update information in the second repository may be searched to identify one or more older versions of the second software and the first software in which the reference information of the first software has been updated. The information of the identified first software is transmitted to the first analyzer, In the step of exploring the first software, Based on the information of the first software identified and transmitted from the second analyzer, one or more older versions of the first software stored in the first repository are identified. By performing a performance test on the older version of the first software, the first software that has experienced performance degradation can be identified. A software analysis method characterized by the following features.

Images