Software analysis system and software analysis method

Abstract

The purpose of the present invention is to quickly determine the version of software that causes a failure even when a plurality of software update records are individually managed. A software analysis system according to the present invention, in a case where performance degradation of second software is detected by a performance test, specifies a version in which first software is merged and updated, or specifies a version in which reference information of the first software has been changed with respect to the previous version, from among the previous versions of the second software. As a result, a candidate for the first software, which causes the decrease, is identified (referring to Figure 4B).

Description

Technical Field

[0001] The present invention relates to a technique for analyzing the cause of software performance degradation. Background Art

[0002] In an information system composed of multiple software, when performance degradation or malfunction occurs in a certain software, it is also possible that the cause lies in other software. For example, the electronic control unit (ECU) that controls the movement of the vehicle, such as the application side software (hereinafter referred to as ASW) and the basic system side software (hereinafter referred to as BSW), works in coordination with multiple software. For example, when performance degradation occurs in ASW, there are cases where the cause is caused by the BSW side.

[0003] The following patent document 1 describes a technique for determining the cause of a performance problem of a program. The technique described in the document is as follows: "A path determination device (1) includes a first difference extraction unit (21), a path extraction unit (22), and a determination unit (23). The first difference extraction unit (21) obtains a plurality of difference locations between the source code of a first program and the source code of a second program, and first coverage information of the second program. The first difference extraction unit (21) refers to the first coverage information and extracts the first difference location included in the first coverage information from the plurality of difference locations. The path extraction unit (22) extracts a plurality of impact paths (32) affected by the first difference location from the source code of the second program for each first difference location (31). The determination unit (23) obtains the second coverage information of the second program and excludes the impact path included in the second coverage information from the plurality of impact paths extracted by the path extraction unit (22)." (See the abstract of the specification).

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent document 1: WO2021 / 124464 Summary of the invention

[0007] Problems to be solved by the invention

[0008] Software performance testing is not necessarily performed frequently, and there are cases where the interval between performance tests spans a certain period. It is conceivable that the prior art such as Patent Document 1 assumes that the software change that causes performance degradation is relatively recent. Therefore, it is conceivable that if the software change that becomes the cause is at a time point that is quite before the analysis time point, the changed part will not be extracted as the changed difference part, and it may be difficult to find the cause part.

[0009] Furthermore, when multiple software constituting the system each have update histories, the update histories need to be managed for each software. Therefore, when performance degradation or the like occurs in software managed by other developers, it may be difficult to identify the cause or a long time may be required.

[0010] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to be able to quickly identify the version of software that is the cause of a malfunction even when the update histories of a plurality of software are managed individually.

[0011] Means used to solve problems

[0012] When the software analysis system of the present invention detects a performance degradation of the second software through performance testing, it determines a version that is updated by merging the first software in past versions of the second software, or determines a version in which the reference information of the first software is changed relative to the previous version, thereby determining the candidate of the first software that is the cause of the degradation.

[0013] Effects of the Invention

[0014] According to the software analysis system of the present invention, even when the update histories of a plurality of software are managed individually, the version of the software causing the malfunction can be quickly identified. Other issues, configurations, effects, etc. than those described above will become clear from the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a diagram schematically showing the configuration of software mounted on the ECU.

[0016] Figure 2 It is a diagram illustrating continuous integration (CI) of software.

[0017] Figure 3 This is a diagram for explaining a problem in a case where a plurality of software items are individually version-managed.

[0018] Figure 4A This is a conceptual diagram for explaining the procedure of searching for the cause of a failure in the first embodiment.

[0019] Figure 4B This is a conceptual diagram for explaining the procedure of searching for the cause of a failure in the first embodiment.

[0020] Figure 5 This is a conceptual diagram for explaining the procedure of searching for a problem location on the BSW side by a conventional binary search.

[0021] Fig. 6A This is a conceptual diagram illustrating the order of searching for the cause of a malfunction in the software analysis method related to the second embodiment.

[0022] Figure 6B This is a conceptual diagram illustrating the order of searching for the cause of a malfunction in the software analysis method related to the second embodiment.

[0023] Figure 7 This is a diagram showing the configuration of a software analysis system 1 according to a fourth embodiment.

[0024] Figure 8 This is a flowchart illustrating the operation of the analysis device 200 .

[0025] Fig. 9 It is a flowchart explaining the operation of the analysis device 100 .

[0026] Fig.10 This is an example of a screen interface that visualizes and presents the result of identifying a problem location that is a cause of performance degradation. DETAILED DESCRIPTION

[0027] <Implementation Method 1>

[0028] Figure 1 This is a diagram schematically showing the composition of the software installed in the ECU. For example, the ECU is equipped with application-side software (ASW) and basic system-side software (BSW). The ASW and the BSW can provide functions to each other (or from one to the other) via appropriate interfaces. For example, the ASW can use the functions provided by the BSW as functions on the software.

[0029] Figure 2 This is a diagram illustrating the continuous integration (CI) of software. In CI, the change history of the source code (e.g., the full text of the source code for each change and the mark of the differential part) is typically stored in the source code version management tool. Registering the source code to the version management tool (determining the version at that point in time) is called commit. Along with the commit, there are also cases where regression testing or performance testing is implemented by executing automatic testing tools. The test includes static analysis of the source code, building (build, compiling the source code into an executable form) and actually executing the program. The test results are notified to relevant personnel (developers, etc.).

[0030] The above mechanism of CI can suppress the time required to check for bad conditions through the automatic testing function, while reliably reporting the bad condition parts as test results, and excluding the personal nature of the test to maintain stable test quality. However, since some test items such as performance testing take a long time, it is difficult to implement the test every time it is submitted, and it is generally implemented at a certain interval (for example, every two weeks, every month, etc.). For such test items, a time interval is required until the bad condition is discovered. It can be imagined that the previous software analysis methods have room for improvement in this regard.

[0031] Figure 3 This is a diagram that explains the problem when multiple software are version-managed separately. Version management tools generally have a function to search for the source code of the problematic version (submitted serial number) by, for example, binary search. Figure 3 In the example, the source code submitted initially (C1) has no problems (good) in the test results, but the source code submitted 9th time has problems (bad). The version management tool backtracks from C9 to obtain the previous version (typically the intermediate version between C1 and C9, in Figure 3 If the test result is no problem, then compare the previous (newer) intermediate version of C5 (in Figure 3 By this repeated process, the oldest version among the versions with problems in the test results (i.e., the version that caused the problem in C9) can be determined. The test at this time can be either automatic or manual.

[0032] like Figure 3 Searching for problematic versions through binary search can effectively identify the problem site. On the other hand, imagine a situation where multiple software works in conjunction and each software is version managed separately. In this case, when a bad situation is found during the testing of the software under development, it can be imagined that the cause of the bad situation may exist in other software. Therefore, it is necessary to search the version management tools of other software to identify the problem site. However, methods for effectively searching for problem sites across version management tools have not been fully studied so far.

[0033] In view of such problems, the software analysis method according to the first embodiment of the present invention provides a method for efficiently searching for the cause of a problem when the results of performance testing or the like indicate a problem, when a plurality of software programs operating in conjunction are individually version-managed.

[0034] Figure 4A to Figure 4B This is a conceptual diagram for explaining the order of searching for the cause of the fault in this embodiment. Figure 1ASW uses the functions provided by BSW. ASW and BSW are version-managed independently of each other by separate version management tools (or different instances of the same version management tool, etc.). Here, it is assumed that performance degradation is found in the performance test of ASW, and the cause lies on the BSW side.

[0035] In the case where ASW utilizes functions on the BSW side, when building or testing ASW, there are cases where BSW is imported and then implemented. For example, there are cases where a certain version of source code on the BSW side (or code such as compiled binaries, class libraries, etc., which is used to enable the use of BSW functions when executed on the ASW side) is imported into the development environment on the ASW side. The correspondence between the BSW version of the import source and the ASW version of the import destination can be pre-set in the development plan. In this example, it is assumed that it is pre-specified that ASW's submission C2 imports BSW's submission C1 ( Figure 4B As shown), ASW's submission C6 is taken into BSW's submission C5 ( Figure 4B shown).

[0036] Assume that Figure 4A At the start time point of the test, the initial submission implemented in the past had no problems (good) in the test results, and a problem (bad) was found in the performance test implemented at the 9th submission. In the order described below, the source code version that is the cause of the problem is searched on the BSW side. It is assumed that the second analysis device implements the search process on the ASW side, and the first analysis device implements the search process on the BSW side. The structure of these analysis devices will be described later. On the BSW side described later, it is also assumed that the performance test result of BSW's submission C1 is good.

[0037] In the first performance test, the second analysis device determines the version of BSW that has been imported from the past version of the source code of ASW. Here, submissions C2 and C6 correspond to this. The second analysis device performs a performance test on the oldest version (C2) among them. Here, it is assumed that the test result is good.

[0038] In the second performance test, the second analysis device performs a performance test on the latest version of the past versions of BSW (here, since only two versions of BSW, C2 and C6, are imported, C6 is the latest). Here, it is assumed that the test result is problematic. It is assumed that the performance test performed on the ASW side is only for the version imported into BSW. Therefore, the performance test on the ASW side is completed by the above.

[0039] The test object on the ASW side is limited to the version that has been imported into BSW, but the basis for selecting the search object (test object) is based on binary search. That is, the second analysis device selects the part that is the test object in the ASW source code through binary search. For example, Figure 4A If there are past versions of BSW imported in addition to C2 and C6, the objects of performance testing are determined by binary search.

[0040] In the third performance test ( Figure 4B ), the first analysis device performs performance testing on the latest version of the BSW source code imported into the ASW side (here, BSW submission C5) from the past versions of the BSW source code. Here, it is assumed that the test result is a problem. Therefore, it can be inferred that the cause of the problem occurred at a certain point in time in submissions C2 to C5 on the BSW side. The following search order is the same as the binary search.

[0041] In the fourth performance test, the first analysis device performs a performance test on an intermediate version (here C3) among the BSW submissions C2 to C4. Here, it is assumed that the test result is good. In the fifth performance test, the first analysis device performs a performance test on the untested submission C4 among C2 to C5. Here, it is assumed that the test result is problematic. Through the above, the version C4 that caused the problem on the BSW side was determined.

[0042] Figure 5 This is a conceptual diagram for explaining the order of searching for problem parts on the BSW side by conventional binary search. On the ASW side, the second analysis device searches for problem parts by binary search starting from submission C9. Here, the performance test is performed in the order of C9=>C5=>C7=>C6. The following order is consistent with Figure 4A to Figure 4B is the same. Figure 4A Unlike the previous version, the ASW side did not limit the search target to the version merged with the BSW side, so the performance test of C6 was the third time. Figure 4A to Figure 4B Compared with the performance test, the number of executions is greater, so it takes longer to find the problem location. In this respect, the present embodiment is more advantageous than the conventional binary search.

[0043] <Implementation method 1: Supplement>

[0044] Different from the above, Figure 4AIn the first performance test of the ASW, if the performance test result of C2 is bad, then it can be determined that the oldest bad part on the ASW side is C2 at this time point. Therefore, it is only necessary to perform a performance test on the submission C1 on the BSW side. On this basis, if the submission C1 of BSW is problematic, it is the cause of the problem, and if the submission C1 of BSW is not problematic, then the submission C2 of ASW is the cause of the problem. The software analysis system 1 (for example, a certain search unit) outputs a presumption result to this effect.

[0045] Different from the above, Figure 4A In the second performance test of , if the performance test result of C6 is good, then the problem first occurs on the ASW side in one of the submissions C7 to C9 on the ASW side. Since they are not taken into BSW, the cause of the problem lies on the ASW side. The software analysis system 1 (for example, a certain search unit) outputs a result of speculation to this effect. In this case, since there is no need to search for the cause across version management tools, the subject of the present invention does not occur.

[0046] Different from the above, Figure 4B In the third performance test of the BSW, if the performance test result of C5 is good, there is no problem on the BSW side. This is because the result of taking the BSW source code C5 without problem into C6 of ASW makes C6 of ASW have problems. The software analysis system 1 (for example, a certain search unit) outputs the inference result to this effect.

[0047] <Implementation 1: Summary>

[0048] The software analysis method according to this embodiment automatically searches for the version that causes performance degradation on the BSW side by binary search, thereby eliminating the human nature of the technology for identifying the problem site and identifying the problem site without relying on the analysis skills of the analyst.

[0049] The software analysis method of this embodiment determines the submission as the search object on the ASW side, based on the binary search, and only the version that has been imported into the BSW is set as the search object. In this way, the search objects can be narrowed down, so the time required to determine the problem location can be shortened compared to the past. For example, Figure 5 The conventional method shown in the figure performed 6 performance tests. Figure 4A to Figure 4B In the present embodiment shown, five times are sufficient.

[0050] The software analysis method according to this embodiment starts from the version of BSW imported into the ASW side and searches for the past version of the BSW side. Thus, even when the ASW and BSW are version-managed separately, the problem site can be efficiently identified across version management tools.

[0051] The software analysis method of this embodiment does not require performance testing every time the source code is changed (or submitted). If it is known that there is a problem with the performance test result at a certain point in time, it is sufficient to trace back from that point in time to determine the problem location. Figure 4A to Figure 4B If it is known that there is a performance problem at the time of submitting C9, it is sufficient to perform performance testing while backtracking to the previous version. Therefore, there is no need to prepare a real machine (such as an ECU in a vehicle system) for performance testing as many times as the performance test is performed.

[0052] <Implementation Method 2>

[0053] Figure 6A to Figure 6B This is a conceptual diagram for explaining the order of searching for the cause of a defect in the software analysis method according to Embodiment 2 of the present invention. In this embodiment, in addition to the order described in Embodiment 1, profile information of functions called in ASW is also used. Profile information is information that records the statistics of the frequency of function calls. For example, if the version management tool has the function of obtaining the statistics, the information can be used. Alternatively, the call of each function can be recorded when the test is executed. The rest is roughly the same as Embodiment 1, so the prerequisites, etc. are based on Embodiment 1.

[0054] exist Fig. 6A At the start time of the test, the second analysis device obtains the profile information of ASW's submission C9 (the latest version that found the problem through performance testing). For convenience, it is assumed that the function names provided by ASW start with "asw_" and the function names provided by BSW start with "bsw_".

[0055] In the first performance test, the second analysis device and Figure 4A The same test is conducted on submission C2. Here, it is assumed that the test result is that there is no problem. The second analysis device obtains profile information about submission C2. If the profile information of C9 is compared with the profile information of C2, the calling frequency of function bsw_func2 varies greatly between C2 and C9, and the frequency in C9 increases. Therefore, it can be inferred that this function may be the cause of the performance degradation in C9.

[0056] In the second performance test ( Figure 6B ), the first analysis device analyzes the latest version of the BSW in the past versions imported into the ASW side (in Figure 6BIn the example, the first analysis device performs a performance test on the BSW side (C5 in the example). Here, it is assumed that the test result is problematic. Therefore, it is assumed that the cause of the performance degradation on the ASW side is on the BSW side, and based on the profile information, it is inferred that the cause is function bsw_func2. The first analysis device obtains the change history of function bsw_func2 for submissions prior to BSW submission C5. Here, it is assumed that there is no change in submissions C1 to C3, and the function is changed in submission C4.

[0057] In the third performance test, the first analysis device performs a performance test on the commit C4 in which the function bsw_func2 is changed. Here, it is assumed that the test result is problematic. Therefore, it is highly likely that this version is the cause of the performance degradation, so a detailed investigation is only required on this version (especially the changed part of the function bsw_func2).

[0058] exist Fig. 6A In the analysis, it is inferred that the function bsw_func2 is the cause of the performance degradation. However, based on the structure of the function, it is also possible that the source function that called the function is the cause of the performance degradation. For example, suppose that the source function previously called other functions, but was changed to call bsw_func2 after a certain version, thereby increasing the calling frequency of bsw_func2 on the ASW side. In this case, although there is no problem in bsw_func2, it appears from the profile information that the function has become the cause of the degradation, so it is not appropriate as an analysis result. Therefore, in this embodiment, in order to improve the search accuracy in such a case, additional tracking information may also be used.

[0059] The trace information records the order in which the ASW (or BSW) calls the functions. Figure 6B In the example shown, the first analysis device obtains the trace information of submission C5 during the second performance test. According to the trace information, function bsw_func2 is called by function bsw_init. Therefore, the first analysis device obtains the change history of bsw_init before submission C5. Here, it is assumed that there is no change in submissions C1 to C3, and the function is changed in submission C4.

[0060] In the third performance test, the first analysis device performs a performance test on the commit C4 in which the function bsw_init is changed. Here, it is assumed that the test result is problematic. Therefore, since this version is likely to be the cause of the performance degradation, a detailed investigation is only required on this version (especially the changed part of the function bsw_init).

[0061] The investigation of the change history of bsw_init may be limited to the case where there is no change history of bsw_func2. This is because it is conceivable that bsw_func2 is not the cause of the degradation in this case. Alternatively, the investigation of the change history of bsw_init may be limited to the case where the performance test shows no problem in the version with bsw_func2 changed.

[0062] Since the software analysis method of the second embodiment performs the performance test three times, the number of tests can be further reduced compared with the first embodiment. Figure 5 Compared with the conventional method described in , half the number of times is sufficient.

[0063] <Implementation Method 3>

[0064] In Embodiment 2, it is described that profile information or trace information is used as a basis for inferring the cause of performance degradation of ASW. This is because changes related to the order of function calls may become the cause of performance degradation. Other parameters that may become the cause of performance degradation include the following.

[0065] (Example 1 of the cause parameter) When compiling source code, compile options are specified. For example, as an optimization option, it is sometimes specified whether to emphasize shortening the compile time / reducing the binary size / improving the execution performance, etc. If the compile option is changed, it may cause performance degradation during execution.

[0066] (Example 2 of Cause Parameter) If the memory map (information about where binary numbers are placed on the memory) or the exclusive control level on the OS (Operating System) that executes the software is changed, it may cause performance degradation during execution.

[0067] When determining the past version of the BSW to be tested, the first analyzing device may use at least one of the two parameters instead of or in addition to the trace information described in the second embodiment. Figure 6B If the test result of C5 is problematic, the time point at which the compile options were changed in commits C2 to C4 is obtained through the change history on the version management tool. The OS parameters can also be obtained. The first analysis device performs performance testing on the commit at the time point when the parameters were changed. The subsequent processing is the same as that of implementation method 2.

[0068] <Implementation Method 4>

[0069] Figure 7This is a configuration diagram of a software analysis system 1 according to Embodiment 4 of the present invention. The software analysis system 1 is a system for implementing the software analysis method described in Embodiments 1 to 3, and is composed of an analysis device 100 and an analysis device 200. The analysis device 100 is a device for searching submissions on the BSW side and also serves as a development environment for the BSW. The analysis device 200 is a device for searching submissions on the ASW side and also serves as a development environment for the ASW.

[0070] The analysis device 200 (second analysis device) includes an ASW repository 210 (second repository), a source code acquisition unit 220, a search unit 230 (second search unit), a BSW merge information acquisition unit 240, a test execution unit 250, a questionable submission information sending unit 260, and a profile acquisition unit 270. The ASW repository 210 is a database in which the ASW version management tool stores past versions of the source code of the ASW. The source code acquisition unit 220 acquires various versions of the source code of the ASW from the ASW repository 210. The search unit 230 implements the search for the test object version by binary search described in Implementation Examples 1 to 3 (ASW side). The BSW merge information acquisition unit 240 identifies the submissions that have been imported into the BSW in the past versions of the ASW. The test execution unit 250 implements the performance test of the ASW. The profile acquisition unit 270 acquires the profile information described in Implementation Example 2. The questionable submission information sending unit 260 sends a query if there is a call frequency change in the profile information that exceeds the baseline value (for example, in Fig. 6A In the case of a function where the rank of a function is 10th in C2 but rises by 5 or more places in C9), the analysis device 100 is notified of the message.

[0071] The analysis device 100 (first analysis device) includes a BSW repository 110 (first repository), a source code acquisition unit 120, a search unit 130 (first search unit), an OS information acquisition unit 140, a test execution unit 150, and a trace acquisition unit 160. The BSW repository 110 is a database in which the BSW version management tool stores past versions of the source code of the BSW. The source code acquisition unit 120 acquires each version of the source code of the BSW from the BSW repository 110. The search unit 130 implements the search for the test object version by binary search described in Embodiments 1 to 3 (BSW side). The OS information acquisition unit 140 acquires the OS parameters and their change points described in Embodiment 3. The compile options and their change points may also be acquired together. The test execution unit 150 implements the performance test of the BSW. The trace acquisition unit 160 acquires the trace information described in Embodiment 3.

[0072] Figure 8 1 is a flowchart for explaining the operation of the analysis device 200. Here, the procedure described in Embodiment 2 is described as an example. Figure 4A (or Fig. 6A ) is implemented as the initial state. Figure 8 The following describes each step.

[0073] ( Figure 8 :Steps S801~S803)

[0074] The BSW merge information acquisition unit 240 identifies the submission that has taken in the BSW in the past versions of the ASW (S801). The search unit 230 determines the search target on the ASW side (the version to be investigated by testing whether there is a cause of performance degradation) (S802). Specifically, the version that has taken in the BSW in the past versions of the ASW is determined as the search target. If the search target is determined, the process proceeds to S804 (S803: Yes). If not, the process jumps to S809 (S803: No).

[0075] ( Figure 8 :Steps S804~S807)

[0076] The source code acquisition unit 220 acquires the ASW source code submitted by the search object (S804). After the test execution unit 250 sets the profile information that should be set before the test (S805), it implements the test submitted by the search object (S806). The profile information acquisition can also be implemented in S805. The test execution unit 250 implements S802 to S806 while changing the search object submission until the test result changes (S807). Specifically, the search is implemented in a binary search method until the test results are determined to be submissions without problems and submissions with problems in the search object submissions.

[0077] ( Figure 8 : Steps S808-S809)

[0078] The search unit 230 performs S802 to S807 until it is able to identify a submission whose call frequency in profile information has changed by a reference value or more compared to that in C9 (S808). Fig. 6A In the example, S802 to S807 are performed until commit C2 is identified in which the calling frequency of function bsw_func2 is greatly changed compared with C9. When a commit is identified, the questionable commit information sending unit 260 notifies the analyzing device 100 of the message and the name of the function whose calling frequency has greatly changed (S809).

[0079] ( Figure 8 : Step S809: Supplement)

[0080] Implementation 1: As described in the supplement, there is also a case where the search on the ASW side is completed and no further search is required on the BSW side. In this case, it is determined in S803 that the search object is submitted: None, and the BSW side is notified of this message in S809, thereby completing the operation of the software analysis system 1.

[0081] Fig. 9 is a flowchart for explaining the operation of the analysis device 100. Figure 8 The corresponding BSW side action (action example using tracking information). In the second embodiment, this flowchart is at the time when the warehouse to be searched is switched from the ASW side to the BSW side (at Figure 6B The second performance test begins.

[0082] ( Fig. 9 :Steps S901~S902)

[0083] The search unit 130 determines the search target on the BSW side (the version to be investigated for the cause of performance degradation by testing) (S901). Specifically, when S901 is first implemented, the BSW version that is taken in for the version with a problem in the test result on the ASW side is set as the search start point. After the second time, the version with a problem in the test result is searched by the binary search method. If there is a search target submitted, proceed to S903, and if not, jump to S908 (S902).

[0084] ( Fig. 9 :Steps S903~S907)

[0085] The source code acquisition unit 120 acquires the BSW source code submitted by the search object (S903). The test execution unit 150 sets the trace information that should be set before the test if there is any (S904), and then executes the test submitted by the search object (S905). The trace information acquisition can also be implemented in S904. After the test execution unit 150 determines that the test result has a problem (in Figure 6B In the case of commit C5), S901 to S905 are executed until the calling source function in the tracking information in the previous commit is determined (in Figure 6B bsw_init) is changed (S906). The search unit 130 sets the next object to be searched as the time point at which the calling source function is changed (S907). Figure 6B , commit C4 is set as the next search object.

[0086] ( Fig. 9 : Step S908)

[0087] The search unit 130 identifies the problem site (the problematic version) that causes the performance degradation on the BSW side through the above-mentioned procedures. The search unit 130 outputs data, a screen interface, etc. describing the identified problem site.

[0088] Fig.10 This is an example of a screen interface that visualizes and prompts the result of identifying the problem location that is the cause of performance degradation. The search unit 130 or 230 can visualize the problem location (the version that is the cause of the problem on the BSW side) identified by the above sequence and its search process, for example, as a screen interface prompt. As specific content of the visualization, you can either Figure 4A to Figure 4B , Figure 6A to Figure 6B Alternatively, the data describing the visualization results may be output so that the data can be reproduced on other devices.

[0089] <Variations of the present invention>

[0090] The present invention is not limited to the above-mentioned embodiments, and includes various modified examples. For example, the above-mentioned embodiments are described in detail in order to explain the present invention in an easy-to-understand manner, and are not limited to necessarily having all the described structures. In addition, a part of the structure of a certain embodiment can be replaced with the structure of other embodiments, and the structure of other embodiments can also be added to the structure of a certain embodiment. In addition, with respect to a part of the structure of each embodiment, other structures can be added / deleted / replaced.

[0091] In the above embodiment, an example of searching for source code versions by binary search is described, but the present invention can also be applied when other search methods are used. That is, by using the method described in the above embodiment to narrow down the versions to be searched, the same effect as in the above embodiment can be achieved in search methods other than binary search.

[0092] In the above embodiments, each functional unit included in the analysis devices 100 and 200 may be constituted by hardware such as a circuit device implementing the function, or may be constituted by a computing device such as a CPU (Central Processing Unit) executing software implementing the function.

[0093] In the above embodiment, the ECU is described as an example of a system composed of a plurality of software, but the present invention is also applicable to other software systems.

[0094] Description of symbols

[0095] 1: Software Analysis System

[0096] 100: Analytical device

[0097] 110: BSW Warehouse

[0098] 130: Search Department

[0099] 200: Analytical device

[0100] 210: ASW Warehouse

[0101] 230: Search Department

Claims

1. A software analysis system comprising a first analysis device for executing a performance test of a first software and a second analysis device for executing a performance test of a second software, characterized in that: The first analysis device comprises: A first warehouse storing software information including update information of the first software; and The first search unit searches for the plurality of first software stored in the first warehouse based on the update information of the first software. The second analysis device comprises: a second warehouse storing software information including update information of the second software; and The second search unit searches for a plurality of second software stored in the second warehouse based on the update information of the second software. The second search unit mentioned above, When a performance degradation of the second software is detected by executing the performance test of the second software, Retrieving the update information in the second warehouse, determining one or more old versions of the second software that are updated by merging the update information of the first software, performing a test on the determined old versions of the second software, and determining the first software that contributes to the performance degradation of the second software, Alternatively, the update information in the second repository is retrieved to identify one or more old versions of the second software and the first software whose reference information of the first software is updated. sending the determined information of the first software to the first analyzing device, The first search section mentioned above, determining one or more old versions of the first software stored in the first repository based on the information of the first software determined from the second search unit, By executing the performance test of the old version of the first software, the first software with degraded performance is identified.

2. The software analysis system according to claim 1, characterized in that: The second search unit determines a version with performance degradation and a version without performance degradation in the old version of the second software by performing a test on only the version updated by merging the update information of the first software in the old version of the second software, respectively. The first search unit determines the versions of the first software that are merged with the version with performance degradation and the version without performance degradation in the second software determined by the second search unit as both ends of the version of the first software to be searched from the first warehouse, The first search unit determines a version of the first software having degraded performance by performing a performance test on versions of the first software between the determined two ends.

3. The software analysis system according to claim 2, characterized in that: The software analysis system estimates that the cause of the performance degradation of the second software lies in the second software when the version updated by incorporating the update information of the first software in the old version of the second software does not have performance degradation.

4. The software analysis system according to claim 2, characterized in that: The software analysis system infers that the cause of the performance degradation of the second software lies on the second software side when the versions of the first software at both ends determined by the first search unit do not have performance degradation.

5. The software analysis system according to claim 2, characterized in that: The second search unit obtains statistics on the calling frequency of the function called by the second software. The second search unit determines, among the functions included in the statistics, a function provided by the first software and having a call frequency difference of more than a predetermined value between a version with performance degradation and a version without performance degradation in the version of the second software as a candidate function for degradation cause, The first search unit determines, based on the update history of the candidate degradation cause function, a version of the first software between the two ends in which the candidate degradation cause function is changed from a previous version as a candidate cause of performance degradation of the second software.

6. The software analysis system according to claim 5, characterized in that: The first search unit obtains trace information describing the order of functions called by the first software. The first search unit determines a call source function that calls the degradation cause candidate function based on the trace information, The first search unit determines, based on the update history of the call source function, a version of the first software between the two ends in which the call source function is changed from a previous version as a candidate cause of performance degradation of the second software.

7. The software analysis system according to claim 6, characterized in that: The first search unit searches for a version of the first software in which the candidate function for the degradation cause has been changed from a previous version, instead of searching for a version of the first software in which the calling source function has been changed from a previous version, among versions of the first software between the two ends. The first search unit searches for a version of the first software in which the call source function is changed from the previous version among versions of the first software between the two ends when the deterioration candidate function is not changed from the previous version.

8. The software analysis system according to claim 1, characterized in that: The reference information is at least one of change information of a system call function of the first software, change information of an operating system referenced by the first software, and change information of a compiler of the first software.

9. The software analysis system according to claim 2, characterized in that: The first search unit obtains at least one of change information describing a setting change of an operating system referenced by the first software and change information describing a change of a compile option of a compiler of the first software as the reference information, The first search unit determines, as a candidate cause of performance degradation of the second software, a version of the first software between the two ends, in which the change information describes changes in the operating system settings or the compile options compared with the previous version.

10. The software analysis system according to claim 1, characterized in that: At least one of the first search unit and the second search unit outputs a result that visualizes at least one of the search process and the test result.

11. The software analysis system according to claim 1, characterized in that: The first search unit and the second search unit identify the first software and the second software through binary search.

12. A software analysis method, wherein a first analysis device for performing a performance test on a first software and a second analysis device for performing a performance test on a second software analyze the software, characterized in that: The first analyzing device implements a step of searching for a plurality of first software stored in a first warehouse based on the update information of the first software, wherein the first warehouse stores information of software including the update information of the first software, The second analyzing means implements a step of searching for a plurality of second software stored in a second warehouse based on the update information of the second software, wherein the second warehouse stores information of software including the update information of the second software, In the step of searching for the second software above, When a performance degradation of the second software is detected by executing the performance test of the second software, Retrieving the update information in the second warehouse, determining one or more old versions of the second software that are updated by merging the update information of the first software, performing a test on the determined old versions of the second software, and determining the first software that contributes to the performance degradation of the second software, Alternatively, the update information in the second repository is retrieved to identify one or more old versions of the second software and the first software whose reference information of the first software is updated. sending the determined information of the first software to the first analyzing device, In the step of searching for the first software above, determining one or more old versions of the first software stored in the first repository based on the information of the first software determined and transmitted from the second analyzing device; By executing the performance test of the old version of the first software, the first software with degraded performance is identified.

Images