Determine the impact of multiple applications on system performance

Abstract

A computer-implemented method for determining the impact of multiple applications on system performance includes collecting, by a processor, respective instrumentation data for multiple applications running on a computing system during multiple time segments. The method includes determining, for each of the multiple applications, a performance value and a robustness value for each of the multiple time segments based on the respective instrumentation data. Furthermore, multiple healthy waveforms are generated using the performance values ​​and the robustness values ​​for each time segment, wherein a healthy waveform is generated for each application. The method further includes determining, by the processor, an impact factor of a first application on a second application, wherein the first application and the second application run on the computing system. The method further includes adjusting, by the processor, an allocation of computer resources by freeing computer resources from the first application and allocating the computer resources to the second application based on the impact factor.

Description

[Technical field]

[0001] The present invention relates generally to computing systems, and more particularly to dynamically quantifying the impact of multiple applications on the performance of a computing system executing the applications. [Background technology]

[0002] Traditional monitoring systems that monitor captured parameters for real-time analysis of computing systems may evaluate real-time or near real-time data and compare them to thresholds, limits, or other indicators to determine whether an error or warning condition may exist. The captured parameters may include real-time analysis of machine-generated instrumentation data. When an alert is triggered, some communication is made to a human administrator who may take corrective action. Such systems tend to be reactive, capable of detecting when a problem has already occurred. Summary of the Invention [Means for solving the problem]

[0003] According to one aspect of the invention, a computer-implemented method for determining the impact of a plurality of applications on system performance includes collecting, by a processor, respective instrumentation data for a number of applications running on a computing system during a plurality of time segments. The method further includes determining, by the processor, for each of the plurality of applications, a performance value and a robustness value for each of the plurality of time segments based on the respective instrumentation data. The method further includes generating, by the processor, a healthy waveform for each running application using the performance value and the robustness value for each time segment. The method further includes determining, by the processor, an impact factor of a first application on a second application, where the first application and the second application are running applications. The method further includes adjusting, by the processor, an allocation of the computer resources by freeing up computer resources from the first application and allocating the computer resources to the second application based on the impact factor.

[0004] In one or more embodiments of the invention, generating the healthy waveform includes converting the performance values ​​and the robustness values ​​for each time segment into polar coordinates by the processor, and converting the polar coordinates for each of the plurality of applications into the respective healthy waveform by the processor.

[0005] In one or more embodiments of the invention, the influence factor is determined by calculating a Pearson correlation coefficient between a first healthy waveform of the first application and a second healthy waveform of the second application.

[0006] In one or more embodiments of the invention, determining the influence factor further comprises identifying an apex in each of the first healthy waveform and the second healthy waveform.

[0007] In one or more embodiments of the invention, the influence factor is calculated according to the following formula:

[0008]

number

[0009]

number

[0010] In one or more embodiments of the invention, the method further comprises, in response to detecting a system fault during execution of the second application: storing a performance record associated with the first application in response to the impact factor exceeding a threshold; and not storing an execution record of the first application in response to the impact factor being less than the threshold. Further includes.

[0011] In one or more embodiments of the invention, the method further includes, in response to detecting a system failure during execution of the second application, analyzing only execution records associated with the first application from the execution records of the plurality of applications based on the impact factor exceeding a threshold.

[0012] According to another aspect of the invention, a system includes a memory device and one or more processing units coupled to the memory device, the one or more processing units executing a method for determining an impact of a plurality of applications on system performance.

[0013] According to yet another aspect of the present invention, a computer program product comprising a storage device having one or more computer-executable instructions stored therein, which when executed by one or more processing units causes the one or more processing units to perform a method for determining an impact of multiple applications on system performance.

[0014] In accordance with one or more embodiments of the present invention, a computer-implemented method includes collecting, by a processor, respective instrumentation data for a plurality of applications executing on a computing system during a plurality of time segments. The method further includes determining, by the processor, for each of the plurality of applications, a performance value and a robustness value for each of the plurality of time segments based on the respective instrumentation data. The method further includes generating, by the processor, a plurality of healthy waveforms including a healthy waveform for each application from the plurality of applications using the performance value and the robustness value for each time segment. The method further includes determining, by the processor, an impact factor of a first application being executed on a second application. The method further includes, in response to detecting a system fault during execution of the second application, analyzing only the execution records associated with the first application from the execution records of the plurality of applications based on the impact factor exceeding a threshold.

[0015] According to another aspect of the invention, a computer-implemented method includes collecting, by a processor, respective instrumentation data for a plurality of applications executing on a computing system during a plurality of time segments. The method further includes determining, for each of the plurality of applications, a performance value and a robustness value for each of the plurality of time segments based on the respective instrumentation data. The method further includes generating, by the processor, a plurality of healthy waveforms including a healthy waveform for each application from the plurality of applications being executed using the performance values ​​and the robustness values ​​for each time segment. The method further includes determining, by the processor, an impact factor of a first application on a second application. The method further includes, in response to detecting a system fault during execution of the second application, storing, by the processor, a performance record associated with the first application in response to the impact factor exceeding a threshold; and not storing an execution record of the first application in response to the impact factor being less than the threshold. Further includes.

[0016] Additional features and advantages are realized through the techniques described herein. Other embodiments and aspects are described in detail herein. For a better understanding, reference is made to the specification and drawings.

[0017] The subject matter which is regarded as the invention is particularly pointed out and distinctly claimed in the claims appended hereto. The foregoing and other features and advantages will become apparent from the following detailed description taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 illustrates a block diagram of a system in accordance with one or more embodiments of the invention. [Diagram 2] FIG. 2 illustrates a flow chart diagram for dynamically determining the impact of multiple applications on system performance in accordance with one or more embodiments of the present invention. [Diagram 3] FIG. 3 illustrates an exemplary rendering of performance-robustness data for multiple applications in accordance with one or more embodiments of the present invention. [Figure 4] FIG. 4 illustrates a plot of a performance-robustness curve in polar coordinates (r, θ) in accordance with one or more embodiments of the present invention. [Diagram 5] FIG. 5 illustrates health-waveforms for multiple applications in accordance with one or more embodiments of the present invention. [Figure 6] FIG. 6 illustrates a flow chart diagram of a method for comparing two healthy waveforms in accordance with one or more embodiments of the present invention. [Figure 7] FIG. 7 illustrates a visual depiction of a comparison of healthy waveforms with reference to the plot in FIG. 5 in accordance with one or more embodiments of the present invention. [Figure 8] FIG. 8 illustrates influence-factors calculated between multiple pairs of applications in accordance with one or more embodiments of the present invention. [Figure 9] FIG. 9 illustrates a flowchart of a method for diagnosing a system fault in accordance with one or more embodiments of the present invention. [Figure 10] FIG. 10 illustrates a computer system according to one embodiment of the invention. [Figure 11] FIG. 11 illustrates a cloud computing environment in accordance with one or more embodiments of the invention. [Figure 12] FIG. 12 illustrates abstraction model layers in accordance with one or more embodiments of the invention.

[0019] It should be noted that the drawings herein may not be to scale. In the accompanying drawings and the following detailed description of the disclosed embodiments, various elements illustrated in the drawings are labeled with two, three, or four digit reference numbers. In most cases, the leftmost digit or digits of each reference number correspond to the drawing in which the element is first illustrated. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] An embodiment of the present invention facilitates quantitative analysis and measurement of interactive impact of applications executing on a computing system based on incremental performance changes of one or more performance parameters of the computing system. The performance parameters may include instrumentation data that quantifies the operation of the computing system. For example, the System Management Facility (SMF) is a component of IBM z / OS for mainframe computers that provides a standardized way to write records of activity to a file or data set. SMF provides instrumentation of baseline activity running on an IBM mainframe operating system, including such activities as input / output (I / O), network activity, software usage, error conditions, and processor utilization. SMF forms the basis for many monitoring and automation utilities. It is understood that embodiments of the present invention are not limited to SMF or IBM z / OS, but rather any technique can be used to capture execution records having performance parameters of a computing system, and those performance parameters can be analyzed as described herein to improve the performance of the computing system. The journal entries can be analyzed in real-time. Alternatively or additionally, one or more embodiments of the present invention facilitate determining which journal entries should be saved (or which journal entries should not be saved, or which journal entries should and should not be saved) to reduce the amount of data to be saved, and to reduce the time and other resources required to analyze that data. Further, in one or more embodiments of the present invention, after the journal entries are saved, the journal entries are analyzed offline.

[0021] Technical challenges associated with analyzing performance data of a computing system, especially in real-time, include analyzing large volumes of such performance data. For example, for each running application, the performance data may include parameters such as memory usage, storage usage, paging, processor utilization, processor usage time, input / output metrics, network usage, interrupt usage, and any other such instrumentation-related data indicative of the usage of computing resources by the application. Furthermore, the performance data also includes performance parameters of several applications (e.g., software programs, processes, daemons, etc.) running simultaneously on the computing system. Thus, determining the relationships between applications is also a technical challenge. In addition, the performance data is captured at several time segments and at a predefined frequency, such as 1 minute every 10 minutes, or 10 seconds every 15 minutes, or any other such frequency. Alternatively or additionally, the performance data may be captured based on an event, such as when an application is started, or when an application completes execution, or any other such event related to the execution of one or more applications.

[0022] Embodiments of the present invention address such technical challenges by quantifying the impact of an application on the performance of one or more other applications simultaneously executing in the computing system. Thus, embodiments of the present invention facilitate providing improved techniques for identifying the root cause of performance problems in the computing system. For example, when analyzing a performance problem, an analyst can use impact-related data to start analyzing the system performance from the applications (leaders) that are most influencing during the time segment in which the performance problem occurred / identified. Furthermore, embodiments of the present invention facilitate predicting future performance of influenced applications. For example, the performance trends of the influencing applications (leaders) can be used to predict the performance trends of the affected applications (followers). Additionally, embodiments of the present invention facilitate more agile and efficient monitoring and prediction of system health. For example, by facilitating the determination of the impact of applications in a time segment, system management rules do not need to be predefined by an analyst or system programmer. Rather, the system management can be performed dynamically, thus improving agility. Also, based on the impact-related data, Key Performance Indicators (KPIs) of the system can be dynamically measured during a time segment. To determine the health of the system, it is more efficient to monitor only the impactful applications than to monitor all applications.

[0023] Embodiments of the present invention facilitate determining the impact between applications running on a computer system, as well as analyzing the extent of such impact based on incremental system health changes during different time segments, and further facilitate quantifying the respective health curves of the system caused by each application.

[0024] 1 illustrates a block diagram of a system 100 in accordance with one or more embodiments of the present invention. The system 100 includes a computer system 101 that is monitored and analyzed for any performance related issues by a system analyzer 110. Although the system analyzer 110 is illustrated as a separate block from the computer system 101, it should be noted that in one or more embodiments of the present invention, the system analyzer 110 can be part of the computer system 101 itself. In other embodiments, the system analyzer 110 can be a separate device from the computer system 101.

[0025] The system analyzer 110 performs one or more methods described herein and facilitates determining the impact of one or more applications on system performance as well as the impact on other applications running on the computer system 101. In one or more embodiments of the present invention, the system analyzer 110 can adjust the allocation of one or more resources of the computer system 101 among the applications 106. The system analyzer 110 can also determine which applications running on the computer system 101 can store performance records. Furthermore, in one or more embodiments of the present invention, the system analyzer 110 determines performance records to analyze the performance of the computer system 101. In one or more embodiments of the present invention, the system analyzer 110 uses the performance records to diagnose one or more problems with the computer system 101.

[0026] The computer system 101 can be any type of computing device, such as a mainframe computer, a server computer, a desktop computer, a laptop computer, a tablet computer, a telephone, a modem, a router, or any other computing system comprising hardware and software. The computer system 101 comprises one or more hardware resources 102, such as a processor, a memory device, a storage disk, communication hardware, a graphics processor, a digital signal processor, an input / output device, a cache memory device. The computer system 101 includes one or more software resources 104, such as a hypervisor, a virtual machine, an operating system, a database system, an electronic commerce system, a communication system, an artificial intelligence resource, etc. In one or more embodiments, the hardware resources 102 and the software resources 104 are accessed by one or more applications 106 running on the computer system 101. The hardware resources 102 and the software resources 104 are accessed via respective application programming interfaces provided by the computer system 101.

[0027] The applications 106 may include, but are not limited to, web servers, database access servers, web browsers, games, media streaming applications, spreadsheet applications, word processing applications, mathematical applications, simulation applications, enterprise software applications, or any other type of software application. Such software applications may be used in any field, such as, but not limited to, mathematical simulation, healthcare, science / engineering research, retail, finance, business, etc. FIG. 1 illustrates four applications, namely, Application-A 106A, Application-B 106B, Application-C 106C, and Application-D 106D, running simultaneously on computer system 101. However, it is understood that in other embodiments, any other number of applications may run simultaneously on computer system 101. Although embodiments of the present invention are described herein using an example including the four applications described above, the present invention is not limited by such example.

[0028] 2 illustrates a flow chart diagram of a method 200 for dynamically determining the impact of an application on system performance in accordance with one or more embodiments of the present invention. The method 200 begins at block 202 by collecting instrumentation data X={x1, x2, x3, ..., x n}. The instrumentation data X includes one or more system parameter measurement data x i , for example, processor usage, memory usage, storage disk usage, network usage, operating system usage, virtual machine usage, hypervisor usage, and any other parameter indicative of the usage of hardware resources 102 or software resources 104 or a combination thereof.

[0029] The time segment can be a predefined duration or timepoint at which performance data is captured. For example, the performance data (X) collected can be data observed at a particular time, e.g., 10:00 a.m. or 11:30 a.m. or any other such time. This may be the predefined time at which the performance data is collected.

[0030] Alternatively, the collected instrumentation data (X) can be a collection of performance data observed over a particular duration, for example, from a start time (e.g., 10:00 AM) to an end time (e.g., 11:00 AM). The start and end times can be preset or can be dynamically determined based on the operation of the system 100. The collection can be a sum, an average, a geometric mean, a median, a maximum, a minimum, a weighted average, or any other such statistical function or combination thereof. In other words, for each of the multiple applications 106, instrumentation data is captured from a start time to an end time, and one or more statistical operations are performed on the captured data and the results are stored as performance data. In the example scenario of applications 106A-106D, X is respectively assigned to applications 106A-106D. A ,X B ,X C ,X D Four sets of instrumentation data are collected.

[0031] In block 204, performance and robustness (PR) values ​​are predicted for each of the collected instrumentation data X. In a multivariate system, the PR prediction quantifies the impact of uncertainty and thus facilitates making informed decisions. The PR prediction also reduces the gap between predicted performance and operational performance. The PR prediction identifies uncertain influential parameters that affect the performance of the computer system 101. The PR prediction can facilitate identifying the most influential uncertainty sources from the collected instrumentation data (X).

[0032] Various PR modeling and prediction techniques are known, and embodiments of the present invention can use any of the known PR modeling and prediction techniques. For example, in embodiments of the present invention, the PR value is determined using a neural network that receives instrumentation data X for each of a plurality of applications 106. In one embodiment, the neural network is trained using a radial basis function (RBF) to provide a corresponding PR value for each of the plurality of applications. Thus, a PR value is obtained for each of applications 106A, 106B, 106C, and 106D, respectively, and for each time segment during which instrumentation data X is acquired.

[0033] FIG. 3 illustrates an exemplary plot of PR data for multiple applications in accordance with one or more embodiments of the present invention. In the illustrated plot 300, performance is plotted along the Y-axis 302, and robustness is plotted along the X-axis 304. The plot 300 includes four curves 306A-306D corresponding to applications 106A-106D, respectively. Each curve 306A, 306B, 306C, and 306D illustrates PR values ​​for applications 106A, 106B, 106C, and 106D corresponding to various time segments. Each time segment is represented in FIG. 3 as an arrow (vector). Although four time segments per application are represented in FIG. 3, in other embodiments of the present invention, a different number of time segments can be analyzed.

[0034] Referring again to the flow chart diagram in Figure 2, at block 206, the method 200 includes mapping the PR values ​​to polar coordinates (r, θ). Figure 4 illustrates a corresponding plot 400 illustrating the curves PR-A 306A, PR-B 306B, PR-C 306C, and PR-D 306D in polar coordinates (r, θ) in accordance with one or more embodiments of the present invention. The polar coordinates can be calculated, for example, from the Cartesian (P,R) coordinates as follows:

[0035]

number

[0036] where r is the distance of the point from the origin, x is the performance value, y is the robustness value; and

[0037]

number

[0038] where θ is the angle (in degrees) relative to the zero axis.

[0039] Further, in step 208, the polar coordinates for each of the plurality of applications 106 are converted into a corresponding healthy waveform. The healthy waveform represents system performance during the execution of the application 106. The system performance is related to resource utilization. Thus, the healthy waveform represents the usage of the resources (102, 104) of the computing system 101 during execution by the application 106, where each waveform represents the usage by the corresponding application. FIG. 5 illustrates healthy waveforms of a plurality of applications in accordance with one or more embodiments of the present invention. The healthy waveforms 506A, 506B, 506C, and 506D are shown on a plot 500 in Cartesian coordinates having a Y-axis 502 and an X-axis 504. The polar coordinates of each of the PR curves 406A-406D are converted into the Cartesian coordinates of the corresponding healthy waveforms 506A-506D using a dimensionality reduction function.

[0040]

number

[0041] Here, t represents time, and (a0, θ0) is the starting point of each vector in PR curves 406A-406D.

[0042] 5, plot 500 is derived from plot 400, which in turn is derived from plot 300. Healthy waveforms 506A-506D correspond to PR curves 306A-306D, respectively.

[0043] Referring to the flow chart diagram in FIG. 2, the method 200 further includes, at block 210, comparing a pair of healthy waveforms to detect which of the healthy waveforms leads and which leads.

[0044] 6 illustrates a flow chart diagram of a method 600 for comparing two healthy waveforms in accordance with one or more embodiments of the present invention. FIG 7 illustrates a visual depiction of the comparison with reference to plot 500. The pair of healthy waveforms being compared are HW-A 506A and HW-B 506B.

[0045] At block 602, a first crest-point (t2) and a first trough-point (t1) are identified for a first healthy waveform (HW-A 506A). A trough point in a healthy waveform is determined by the value of f (value at t1 = f t1 (r t1 ,θ t1 )) and determining a point (t1 702) (X-axis) where at least a threshold number of consecutive previous points have successively decreasing values ​​of f and at least a threshold number of subsequent points have successively increasing values ​​of f. In Figure 7, t1 702 is a minimum or valley point of HW-A 506A.

[0046] The peak of the healthy waveform is the value of f (value at t2 = f t2 (r t2 ,θ t2 )) and determining a point (t2 704) (X-axis) where at least a threshold number of consecutive previous points have successively increasing values ​​of f and at least a threshold number of subsequent points have successively decreasing values ​​of f. In Figure 7, t2 704 is the maximum point or apex on HW-A 506A.

[0047] The threshold number of points used to determine valleys and peaks can be a predetermined value. Alternatively or additionally, the threshold number of points can be dynamically configured based on the number of points in the healthy waveform. In one or more embodiments of the invention, the threshold number of points for a first healthy waveform (e.g., HW-A 506A) can be different from the threshold number of points used for a second healthy waveform (e.g., HW-B 506B).

[0048] Additionally, valleys and peaks are detected for the second, compared healthy waveform in block 604. In Figure 7, valley t3 706 and peak t4 708 on HW-B 506B are shown.

[0049] At block 606, a correlation between the two compared healthy waveforms is determined based on the valleys and peaks. For example, if the difference between t1 and t2 is within a predetermined threshold of the difference between t3 and t4, the two healthy waveforms may be considered to have a leader-follower relationship. In other embodiments, the correlation may be determined based on other techniques.

[0050] 7, it can be seen that HW-A 306A leads HW-D 306D, and the two healthy waveforms have very similar peak and valley timings. Similarly, HW-C 306C closely follows the healthy waveform of HW-B 306B.

[0051] At block 608, the influence between the applications corresponding to the two compared healthy waveforms is determined based on the valleys and peaks, for example, by calculating a set of influence factors between the two corresponding applications by calculating the Pearson Correlation Coefficient (PCC).

number

number

[0052] For example, the influence factor between application 106A and application 106B is calculated using valley points t1 702, t3 706, and peak points t2 704, t4 708. It should be noted that the influence factor is directional, i.e., the influence factor of application 106A on application 106B is different from the influence factor of application 106B on application 106A.

[0053] 8 illustrates influence factors calculated between a set of applications in accordance with one or more embodiments of the present invention. Although influence factors between applications 106A, 106B, and 106C are illustrated, it is understood that influence factors between other possible combinations of applications 106 can be calculated. It should also be understood that the illustrated influence factors are example values, and that in other embodiments, the values ​​of the influence factors can vary.

[0054] Referring back to the flow chart diagram of FIG. 6, the calculated influence factors are used to determine whether a correlation exists between the applications at block 610. For example, if at least one of the influence factors between the two applications is below (or above) a predetermined threshold, it can be deemed that no leader-follower relationship exists between the two applications corresponding to the compared healthy waveforms. In such a case, the method further includes determining whether additional valleys and peaks exist in the two healthy waveforms being compared at block 612. If additional valleys and peaks exist, the operations described above can be repeated for the additional valleys and peaks. If additional valleys and peaks do not exist, then at block 614, the two applications are marked as not having any influence relationship, and other applications are analyzed using method 200.

[0055] Alternatively, if a relationship is identified at block 610, then at block 616, an influence factor between the two applications is stored.

[0056] In this manner, the impact of a first application (e.g., 106A) on other applications running simultaneously on computing system 101 is determined. The impact of each of the other applications on the first application is also determined. The impact of an application on a second application is represented by an impact factor of the first application on the second application. The relationship between the first application and the second application is represented by a set of impact factors between the two applications.

[0057] 2, in block 212, the influencing factors are used to adjust the allocation of hardware resources 102 and software resources 104 from the computing system 101. For example, as can be seen in FIG. 7, application 106A uses maximum resources at t2 (704), minimum resources at t1 (702), and application 106B uses maximum resources at t4 (708), t2 (704), t1 (702), and t4 (708), occurring in the following order: Thus, at least a portion of the resources used by application 106A can be released (de-allocated) after t2 (704) and allocated to application 106B in anticipation of t4 (708).

[0058] Alternatively or additionally, at substantially time t4 708, both application 106B and application 106C have their respective vertices. Thus, both applications are using maximum resources at substantially the same time. Thus, resource allocations can be adjusted such that at least a portion of the resources from applications 106A and 106C are released and allocated to applications 106B and 106D. Other adjustments to resource allocations can also be made.

[0059] The resources allocated and freed can include, but are not limited to, memory, cache, processors, software licenses (APIs), and any other hardware resources 102, or software resources 104. Such resource allocation adjustments can be performed dynamically, i.e., at run-time, and can improve the performance of the applications 106, and thus the computing system 101.

[0060] In addition, in case of a system failure, e.g., resource congestion, deadlock, or other type of failure, diagnosis can be performed using the impact factors. FIG. 9 illustrates a flowchart diagram of a method 900 for diagnosing a system failure, in accordance with one or more embodiments of the present invention. Consider that the computing system 101 experiences a failure during the execution of application 106D. As shown in FIG. 9, the method 900 includes, at block 902, determining a set of leading applications of the application that experienced the failure (in this case, application 106D). The set of leading applications includes only applications that have an impact factor of at least a predetermined threshold in application 106D. Thus, application 106A is in the set of leading applications of application 106D only if the impact factor of application 106A in application 106D is at least a predetermined threshold.

[0061] In one set of embodiments, only applications from the set of predecessor applications save their execution records for such analysis, at block 903. The execution records may include system dumps, execution logs, or any other type of execution record provided by application 106. Thus, by reducing the number of applications that store their execution records, the amount of time and storage space required by the execution records can be reduced and therefore improved.

[0062] Further, in block 904, only the execution records of the applications in the set of predecessor applications are analyzed. Instead of analyzing the execution of all applications 106 that are executed simultaneously, even if all applications 106 have saved their execution records, only the execution records of the set of applications of the predecessor applications are analyzed. Thus, the efficiency of diagnosis when a fault occurs can be improved. The analysis can be performed manually or automatically.

[0063] Embodiments of the present invention provide practical applications for improving the performance of a computing system. Embodiments of the present invention facilitate improved diagnosis of the root cause of a computing system's performance problems by reducing the amount of data that must be stored and analyzed or analyzed for such diagnosis. Embodiments of the present invention also facilitate predicting the performance of one or more applications running on the system and further adjusting the allocation of computing resources (hardware or software or a combination thereof) accordingly. Furthermore, embodiments of the present invention facilitate monitoring and predicting the health of a system in a more agile and efficient manner compared to existing techniques. Based on one or more impact factors, embodiments of the present invention can facilitate identifying which applications should be monitored instead of monitoring all applications. Furthermore, KPIs of the system can be dynamically measured for each time segment based on the impact factors, and only those associated with the preceding application can be monitored, thus reducing the amount of data to be monitored and improving efficiency.

[0064] Referring now to FIG. 10, a computer system 1000 according to one embodiment is shown in schematic form. The computer system 1000 can be the computing system 101 in one or more embodiments of the present invention. Alternatively or additionally, the computer system 1000 can be the system analyzer 110 in one or more embodiments of the present invention. The computer system 1000 can be an electronic, computer framework as described herein that comprises or uses or comprises and uses any number and combination of computing devices and networks utilizing various communication technologies. The computer system 1000 can be easily scalable, extensible, and modular with the ability to change to different services or reconfigure some functions independently of others. The computer system 1000 can be, for example, a server, a desktop computer, a laptop computer, a tablet computer, or a smartphone. In some examples, the computer system 1000 can be a cloud computing node. The computer system 1000 can be described in the general context of computer system executable instructions, such as program modules, being executed by the computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer system 1000 may be practiced in a distributed cloud computing environment where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media, including memory storage devices, as described above.

[0065] As shown in FIG. 10, computer system 1000 includes one or more central processing units (CPUs) 1001a, 1001b, 1001c, etc. (collectively or generally referred to as one or more processors 1001). Processor 1001 can be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. Processor 1001, also referred to as a processing circuit, is connected to system memory 1003 and various other components via a system bus 1002. System memory 1003 can include read only memory (ROM) 1004 and random-access memory (RAM) 1005. ROM 1004 is connected to system bus 1002 and can include a basic input / output system (BIOS) that controls certain basic functions of computer system 1000. The RAM is a readable and writable memory connected to system bus 1002 for use by processor 1001. The system memory 1003 provides temporary memory space for the execution of the instructions during operation. The system memory 1003 may include random access memory (RAM), read-only memory, flash memory, or any other suitable memory system.

[0066] Computer system 1000 includes an input / output (I / O) adapter 1006 and a communications adapter 1007 connected to a system bus 1002. I / O adapter 1006 may be a small computer system interface (SCSI) adapter that communicates with a hard disk 1008 or any other similar component or combination thereof. I / O adapter 1006 and hard disk 1008 are collectively referred to herein as mass storage 1010.

[0067] Software 1011 for executing on computer system 1000 may be stored on mass storage 1010. Mass storage 1010 is an example of a tangible storage medium readable by processor 1001, where software 1011 is stored as instructions for execution by processor 1001 to operate computer system 1000, such as those described below with respect to the various figures. Examples of computer program products and the execution of such instructions are discussed in more detail herein. Communications adapter 1007 interconnects system bus 1002 with network 1012, which may be an external network, enabling computer system 1000 to communicate with other such systems. In one embodiment, a portion of system memory 1003 and mass storage 1010 collectively stores an operating system, which may be any suitable operating system, for example, z / OS or AIX operating systems from IBM Corporation, for coordinating the functions of the various components shown in FIG. 10.

[0068] Additional input / output devices are shown connected to the system bus 1002 via a display adapter 1015 and an interface adapter 1016. In one embodiment, adapters 1006, 1007, 1015, and 1016 may be connected to one or more I / O buses that are connected to the system bus 1002 via an intermediate bus bridge (not shown). A display 1019 (e.g., a screen or display monitor) is connected to the system bus 1002 by a display adapter 1015, which may include a graphics controller and a video controller to improve the performance of graphics-intensive applications. A keyboard 1021, a mouse 1022, speakers 1023, etc. may be interconnected to the system bus 1002 via an interface adapter 1016, which may include, for example, a super I / O chip that integrates multiple device adapters into a single integrated circuit. Suitable I / O buses for connecting peripheral devices, such as hard disk controllers, network adapters, and graphics adapters, typically include a common protocol, PCI (Peripheral Component Interconnect). Thus, as configured in FIG. 10, computer system 1000 includes processing capability in the form of a processor 1001, as well as storage capabilities such as those described above including system memory 1003 and mass storage 1010, input means such as keyboard 1021 and mouse 1022, and output capabilities such as those described above including speakers 1023 and a display 1019.

[0069] In some embodiments, the communications adapter 1007 can transmit data using any suitable interface or protocol, such as an Internet Small Computer System Interface, among others. The network 1012 may be a cellular network, a wireless network, a wide area network (WAN), a local area network (LAN), or the Internet, among others. An external computing device may be connected to the computer system 1000 through the network 1012. In some examples, the external computing device may be an external web server or a cloud computing node.

[0070] It should be understood that the block diagram of Figure 10 is not intended to indicate that computer system 1000 includes all of the components shown in Figure 10. Rather, computer system 1000 can include any suitable fewer or additional components (e.g., additional memory components, embedded controllers, modules, additional network interfaces, etc.) not shown in Figure 10. Furthermore, the embodiments described herein with respect to computer system 1000 may be implemented with any suitable logic, and logic referred to herein may include, in various embodiments, any suitable hardware (e.g., a processor, an embedded controller, or an application specific integrated circuit, among others), software (e.g., an application, among others), firmware, or any suitable combination of hardware, software, and firmware.

[0071] Although this disclosure includes detailed descriptions of cloud computing, it should be understood that implementation of the teachings recited herein is not limited to a cloud computing environment. Rather, embodiments of the present invention can be implemented in conjunction with any other type of computing environment now known or later developed.

[0072] Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with the service provider. The cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0073] The features are as follows.

[0074] On-demand self-service: A cloud consumer can unilaterally provision computing capacity, such as server time and network storage, automatically as needed, without requiring human interaction with the provider of the service.

[0075] Broad Network Access: Capabilities are available over the network and accessed via standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., cell phones, laptops, and PDAs).

[0076] Resource Pooling: Provider computing resources are pooled to serve multiple consumers using a multi-tenant model, and various physical and virtual resources are dynamically allocated and reallocated according to demand. Consumers generally have no control or knowledge of the exact location of the provided resources, but are said to be location-independent in that they may be able to specify a location at a higher level of abstraction (e.g., country, state, or data center).

[0077] Rapid Elasticity: Capabilities can be provisioned quickly and elastically, sometimes automatically, quickly scaled out, quickly released and quickly scaled in. To the consumer, the capabilities available for provisioning are often unlimited and can be purchased in any quantity at any time.

[0078] Measured Services: Cloud systems automatically control and optimize resource usage by using metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the services being utilized.

[0079] The service model is as follows:

[0080] Software as a Service (SaaS): The capability offered to consumers to use a provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through a thin-client interface, e.g., a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even individual application functions, with the possible exception of limited user-specific application configuration settings.

[0081] Platform as a Service (PaaS): A capability offered to a consumer to deploy consumer-created or acquired applications, created using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, e.g., including networks, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, the application-hosting environment configuration.

[0082] Infrastructure as a Service (IaaS): The capability offered to consumers to provision processing, storage, network, and other basic computing resources on which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but does have control over the operating systems, storage, deployed applications, and in some cases, limited control over selecting network components (e.g., host firewalls).

[0083] The deployment models are as follows:

[0084] Private Cloud: Cloud infrastructure is operated exclusively for an organization. The cloud infrastructure may be managed by the organization or a third party, and may be on-premises or off-premises.

[0085] Community Cloud: Cloud infrastructure is shared by several organizations and supports a particular community with common concerns (e.g., mission, security requirements, policies, and compliance considerations). The cloud infrastructure may be managed by the organizations or a third party, and may reside on-premise or off-premise.

[0086] Public Cloud: The cloud infrastructure is available to the general public or large industry groups and is owned by an organization that sells cloud services.

[0087] Hybrid Cloud: The cloud infrastructure remains a unique entity, but is a hybrid of two or more clouds (private, community, or public) brought together by standardized or specialized technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0088] A cloud computing environment is service oriented with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0089] 11, an exemplary cloud computing environment 50 is illustrated. As illustrated, the cloud computing environment 50 includes one or more cloud computing nodes 10 with which local computing devices used by cloud consumers, such as a personal digital assistant (PDA) or mobile phone 54A, a desktop computer 54B, a laptop computer 54C, or an automobile computer system 54N, or combinations thereof, may communicate. The nodes 10 may communicate with each other. They may be physically or virtually grouped in one or more networks, such as a private cloud, a community cloud, a public cloud, or a hybrid cloud, or combinations thereof, as described herein (not shown). This allows the cloud computing environment 50 to provide infrastructure, platform, or software, or combinations thereof, as a service without the cloud consumer having to maintain resources on the local computing device. It will be understood that the types of computing devices 54A-54N shown in FIG. 11 are intended to be illustrative only, and that the computing node 10 and cloud computing environment 50 communicate with any type of computerized device (e.g., using a web browser) via any type of network or network-addressable connection or combination thereof.

[0090] Referring now to Figure 12, a set of functional abstraction layers provided by cloud computing environment 50 (Figure 11) is shown. It should be understood that the components, layers and functions shown in Figure 12 are intended to be merely exemplary, and that embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0091] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include mainframes 61, RISC (Reduced Instruction Set Computer) architecture based servers 62, servers 63, blade servers 64, storage devices 65, and networks and networking components 66. In some embodiments, the software components include network application server software 67 and database software 68.

[0092] The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities may be provided: virtual servers 71; virtual storage 72; virtual networks 73, including, for example, virtual private networks; virtual applications and operating systems 74; and virtual clients 75.

[0093] In one example, the management layer 80 may provide multiple functions described below. Resource provisioning 81 provides dynamic procurement of computing and other resources utilized to execute tasks within the cloud computing environment. Metering and pricing 82 provides cost tracking as resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks, and protection for data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides allocation and management of cloud computing resources such that required service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides pre-provisioning and procurement of cloud computing resources where future requirements are predicted according to SLAs.

[0094] Workload tier 90 provides examples of functions for which the cloud computing environment may be utilized. Examples of workloads and functions that may be provided from this tier include mapping and navigation 91; software development and lifecycle management 92; virtual classroom instructional delivery 93; data analytics processing 94; transaction processing 95; and system performance analysis 96.

[0095] In this description, the terms "computer program medium," "computer usable medium," "computer program product," and "computer readable medium" are used generally to refer to a medium, e.g., a memory. Computer programs (also referred to as computer control logic) are stored in the memory. Such computer programs, when executed, are capable of causing a computer system to perform the inventive features discussed herein. In particular, the computer programs, when executed, enable the controller to perform the features and operations described herein. Thus, such computer programs enable the controller of the computer system.

[0096] A computer readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer readable storage media includes the following: portable computer diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in grooves on which instructions are recorded, or any suitable combination thereof. As used herein, the computer-readable storage medium should not be construed as a transitory signal per se, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted over an electrical wire.

[0097] Many of the functional units described herein are labeled as modules. Embodiments of the present invention apply to a wide variety of module implementations. For example, modules can be implemented as hardware circuits including custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. Modules can also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, etc.

[0098] Modules may also be implemented in software for execution by various types of processors. An identified module of executable code may comprise one or more physical or logical blocks of computer instructions, which may be organized as, for example, an object, a procedure, or a function. Nevertheless, the executable code of an identified module need not be physically located together and may include heterogeneous instructions stored in different locations that, when logically connected, constitute a module and achieve the stated purpose for the module.

[0099] The following definitions and abbreviations shall be used for the interpretation of the claims and the specification. As used herein, the words "comprises," "comprising," "includes," "including," "has," "having," "contains," or other variations thereof, are intended to cover a non-exclusive inclusion. For example, a composition, mixture, process, method, article, or device that includes a list of elements is not necessarily limited to only those elements, but may include other elements not expressly listed or inherent to such composition, mixture, process, method, article, or device.

[0100] The terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that as used herein, the words "comprises" or "comprising" or combinations thereof specify the presence of stated features, integers, steps, operations, elements, or components or combinations thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or groups thereof, or combinations thereof.

[0101] Additionally, the word "exemplary" and variations thereof are used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. The words "at least one," "one or more," and variations thereof can include any integer number greater than or equal to one, i.e., 1, 2, 3, 4, etc. The word "multiple" and variations thereof can include any integer number greater than or equal to two, i.e., 2, 3, 4, 5, etc. The word "connected" and variations thereof can include both indirect and direct "connections."

[0102] The words "about," "substantially," "approximately," and variations thereof are intended to include the degree of error associated with the measurement of a particular quantity based on the equipment available at the time of filing. For example, "about" can include a range of ±8%, 5%, or 2% of a given value.

[0103] As used herein, "in signal communication," "in communication," "communicatively connected," and variations thereof may be used interchangeably and may refer to any coupling, connection, or interaction using electrical signals to exchange information or data, whether such exchange occurs wirelessly or via a wired connection, using any system, hardware, software, protocol, or format using electrical signals to exchange information or data.

[0104] Aspects of the present invention are described herein with reference to flowchart illustrations or block diagrams, or combinations thereof, of methods, apparatus (systems), and computer program products or computer programs according to embodiments of the present invention. It will be understood that each block of the flowchart illustrations or block diagrams, or combinations thereof, and combinations of blocks in the flowchart illustrations or block diagrams, or combinations thereof, can be implemented by computer readable program instructions.

[0105] The flowchart diagrams and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products or computer programs according to various embodiments of the present invention. In this regard, each block in the flowchart diagrams or block diagrams may represent a module, segment, or portion of an instruction, which includes one or more executable instructions for implementing one or more specified logical functions. In some alternative implementations, the functions shown in the blocks may occur out of the order shown in the drawings. For example, two blocks shown in succession may actually be accomplished as one step performed simultaneously, substantially simultaneously, partially or fully in a time-overlapping manner, depending on the functions involved, or the blocks may be performed in the reverse order. It should be noted that each block of the block diagrams or flowchart diagrams or combinations thereof, as well as combinations of multiple blocks of the block diagrams or flowchart diagrams or combinations thereof, may be implemented by a special purpose hardware-based system that performs the specified functions or operations, or may execute a combination of special purpose hardware and computer instructions.

[0106] In the following claims, the corresponding structures, materials, acts, and equivalents of all means or steps in addition to functional elements are intended to include any structure, material, or act for performing a function in combination with the elements of other claims that are specifically claimed. The detailed description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or to limit the invention in the disclosed form. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the invention. The embodiments have been chosen and described in order to best explain the principles and practical applications of the invention and to enable those skilled in the art to understand the invention in various embodiments with various modifications as suited to the particular uses contemplated.

[0107] It will be appreciated that those skilled in the art, both now and in the future, can make various improvements and enhancements that fall within the scope of the appended claims.

Claims

1. 1. A computer-implemented method for determining an impact of multiple applications on system performance, comprising: collecting, by a processor, respective instrumentation data for a plurality of applications executing on the computing system during a plurality of time segments; determining, by the processor, for each of the plurality of applications, a performance value and a robustness value for each of the plurality of time segments based on the respective instrumentation data; generating, by the processor, a plurality of healthy waveforms, including a healthy waveform for each application from the plurality of applications, using the performance value and the robustness value for each time segment; determining, by the processor, an impact factor of a first application on a second application, where the first application and the second application are from the plurality of applications; and adjusting, by the processor, the allocation of computer resources by releasing computer resources from the first application and allocating the computer resources to the second application based on the influence factor; The method comprising:

2. generating the healthy waveform, converting the performance and robustness values ​​for each time segment into polar coordinates; said processor converting said polar coordinates for each of said plurality of applications into said respective healthy waveforms.

2. The computer-implemented method of claim 1, comprising:

3. 3. The computer-implemented method of claim 1 or 2, wherein the influence factor is determined by calculating a Pearson correlation coefficient between a first healthy waveform of the first application and a second healthy waveform of the second application.

4. The computer-implemented method of claim 3 , wherein determining the influence factor further comprises identifying an apex in each of the first healthy waveform and the second healthy waveform.

5. 4. The computer-implemented method of claim 3, wherein the influence factor is calculated according to the following formula: [0010] [0025]

6. 6. The computer-implemented method of claim 1, further comprising, in response to detecting a system fault during execution of the second application, storing an execution record associated with the first application in response to the impact factor exceeding a threshold, and not storing an execution record of the first application in response to the impact factor being less than the threshold.

7. 6. The computer-implemented method of claim 1, further comprising: in response to detecting a system fault during execution of the second application, analyzing only execution records associated with the first application from among the execution records of the plurality of applications based on the impact factor exceeding a threshold.

8. 1. A system comprising: A memory device; and one or more processing units coupled to the memory device, the one or more processing units configured to execute a method for determining an impact of multiple applications on system performance; wherein the method comprises: collecting respective instrumentation data for a plurality of applications executing on the computing system during a plurality of time segments; determining, for each of the plurality of applications, a performance value and a robustness value for each of the plurality of time segments based on the respective instrumentation data; generating a plurality of healthy waveforms using the performance value and the robustness value for each time segment, the plurality of healthy waveforms including a healthy waveform for each application from the plurality of applications; determining an impact factor of a first application on a second application, where the first application and the second application are from the plurality of applications; and adjusting allocation of computer resources by releasing computer resources from the first application and allocating the computer resources to the second application based on the influencing factors; The system comprising:

9. generating the healthy waveform, converting the performance values ​​and the robustness values ​​for each time segment into polar coordinates; Converting the polar coordinates for each of the plurality of applications into the respective healthy waveform. The system of claim 8 , comprising:

10. The system of claim 8 or 9, wherein the influence factor is determined by calculating a Pearson correlation coefficient between a first healthy waveform of the first application and a second healthy waveform of the second application.

11. The system of claim 10 , wherein determining the influence factor comprises identifying an apex in each of the first healthy waveform and the second healthy waveform.

12. The system of claim 10, wherein the influence factor is calculated according to the following formula: [0030] [0045]

13. The method, in response to detecting a system failure during execution of the second application, comprises: storing a performance record associated with the first application in response to the impact factor exceeding a threshold; and not storing an execution record of the first application in response to the impact factor being less than the threshold value. The system according to any one of claims 8 to 12, further comprising:

14. The system of any one of claims 8 to 12, wherein the method further comprises, in response to detecting a system failure during execution of the second application, analyzing only execution records associated with the first application from among the execution records of the plurality of applications based on the impact factor exceeding a threshold.

15. 1. A computer program product comprising a storage device having one or more computer-executable instructions stored therein, collecting respective instrumentation data for a plurality of applications executing on the computing system during a plurality of time segments; determining, for each of the plurality of applications, a performance value and a robustness value for each of the plurality of time segments based on the respective instrumentation data; generating a plurality of healthy waveforms using the performance value and the robustness value for each time segment, the plurality of healthy waveforms including a healthy waveform for each application from the plurality of applications; determining an impact factor of a first application on a second application, where the first application and the second application are from the plurality of applications; and adjusting allocation of computer resources by releasing computer resources from the first application and allocating the computer resources to the second application based on the influencing factors; The computer program product causing one or more processors to execute a method comprising:

16. generating the healthy waveform, converting the performance values ​​and the robustness values ​​for each time segment into polar coordinates; Converting the polar coordinates for each of the plurality of applications into the respective healthy waveform.

16. The computer program of claim 15, comprising:

17. 17. The computer program of claim 15 or 16, wherein the influence factor is determined by calculating a Pearson correlation coefficient between a first healthy waveform of the first application and a second healthy waveform of the second application.

18. 20. The computer program product of claim 17, wherein determining the influence factor comprises identifying an apex in each of the first healthy waveform and the second healthy waveform.

19. 18. The computer program of claim 17, wherein the influence factor is calculated by the following formula: [0050] [006]

20. The method, in response to detecting a system failure during execution of the second application, comprises: storing a performance record associated with the first application in response to the impact factor exceeding a threshold; and not storing an execution record of the first application in response to the impact factor being less than the threshold value. The computer program of any one of claims 15 to 19, further comprising:

21. The computer program product of any one of claims 15 to 19, further comprising: in response to detecting a system failure during execution of the second application, analyzing only execution records associated with the first application from among the execution records of the plurality of applications based on the impact factor exceeding a threshold.

22. 1. A computer-implemented method comprising: collecting, by a processor, respective instrumentation data for a plurality of applications executing on the computing system during a plurality of time segments; determining, by the processor, for each of the plurality of applications, a performance value and a robustness value for each of the plurality of time segments based on the respective instrumentation data; generating, by the processor, a plurality of healthy waveforms, including a healthy waveform for each application from the plurality of applications, using the performance value and the robustness value for each time segment; determining, by the processor, an impact factor of a first application on a second application, where the first application and the second application are from the plurality of applications; and In response to detecting a system fault during execution of the second application, the processor analyzes only execution records associated with the first application from among the execution records of the plurality of applications based on the impact factor exceeding a threshold. The method comprising:

23. 23. The computer-implemented method of claim 22, wherein the influence factor is calculated according to the following formula: [0070] [0080]

24. 1. A computer-implemented method comprising: collecting, by a processor, respective instrumentation data for a plurality of applications executing on the computing system during a plurality of time segments; determining, by the processor, for each of the plurality of applications, a performance value and a robustness value for each of the plurality of time segments based on the respective instrumentation data; generating, by the processor, a plurality of healthy waveforms, including a healthy waveform for each application from the plurality of applications, using the performance value and the robustness value for each time segment; determining, by the processor, an impact factor of a first application on a second application, where the first application and the second application are from the plurality of applications; and in response to detecting a system fault during execution of the second application; storing, by the processor, a performance record associated with the first application in response to the impact factor exceeding a threshold; and not storing an execution record of the first application in response to the impact factor being less than the threshold value. The method comprising:

25. 25. The computer-implemented method of claim 24, wherein the influence factor is calculated according to the following formula: [0090] [0010]

Images