Attribution result determination method and device, electronic equipment, storage medium and product

Abstract

Embodiments of the present disclosure provide an attribution result determination method and apparatus, an electronic device, a storage medium and a product, the attribution result determination method comprising: determining a description difference attribution result of degradation metric data according to description change information of the degradation metric data, the description change information indicating whether description of a determination mode of the degradation metric data is changed; determining a degradation function associated with the degradation metric data under the condition that the description difference attribution result indicates that the degradation metric data is associated with the performance degradation of the application program; and determining an attribution result causing degradation of the degradation function. According to the scheme, the problem of low attribution speed caused by manual measurement data degradation attribution at present is solved, automatic attribution of the degradation measurement data is realized, and the attribution speed is increased. The attribution of the degradation of the degradation metric data is realized from two dimensions, namely whether the description of the determination mode of the degradation metric data is changed or not and whether the performance is degraded or not.

Description

Technical Field

[0001] This disclosure relates to computer technology, and more particularly to methods, apparatus, electronic devices, storage media, and products for determining attribution results. Background Technology

[0002] Metrics are used to evaluate and quantify software performance. Degraded online metrics can have a range of negative consequences. To address this issue, a series of measures are needed.

[0003] Currently, the approach to addressing degradation in metric data involves identifying the degradation and then manually attributing the causes. This process is labor-intensive and slow. Summary of the Invention

[0004] This disclosure provides methods, apparatus, electronic devices, storage media, and products for determining attribution results to automate the attribution of degradation of measurement data.

[0005] In a first aspect, embodiments of this disclosure provide a method for determining attribution results, including:

[0006] Based on the description change information of the degradation measurement data, the description difference attribution result of the degradation measurement data is determined. The degradation measurement data indicates the degradation measurement data that occurs in the application. The measurement data includes data that measures the performance of the application. The description change information indicates whether the description of the method for determining the degradation measurement data has changed.

[0007] If the description of the attribution results indicates that the degradation metric data is associated with the performance degradation of the application, determine the degradation function associated with the degradation metric data;

[0008] Determine the attribution results that led to the degradation of the degradation function.

[0009] Secondly, embodiments of this disclosure also provide an attribution result determination apparatus, comprising:

[0010] The first determining module is used to determine the attribution result of the description difference of the degradation measurement data based on the description change information of the degradation measurement data. The degradation measurement data indicates the degradation measurement data that occurs in the application. The measurement data includes data that measures the performance of the application. The description change information indicates whether the description of the determination method of the degradation measurement data has changed.

[0011] The second determining module is used to determine the degradation function associated with the degradation measurement data when the description difference attribution results indicate that the degradation measurement data is associated with the performance degradation of the application.

[0012] The third determining module is used to determine the attribution result that leads to the degradation of the degradation function.

[0013] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0014] One or more processing devices;

[0015] Storage device for storing one or more programs.

[0016] When the one or more programs are executed by the one or more processing devices, the one or more processing devices implement the attribution result determination method as provided in the embodiments of this disclosure.

[0017] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the attribution result determination method provided in embodiments of this disclosure.

[0018] Fifthly, embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the attribution result determination method provided in embodiments of this disclosure.

[0019] This embodiment of the disclosure determines the attribution result of description differences by using description change information of degradation measurement data. When the description difference attribution result indicates a correlation between degradation measurement data and application performance degradation, it further determines the degradation function associated with the degradation measurement data and the attribution result causing the degradation of the degradation function. This solves the problem of slow attribution speed caused by manual attribution of degradation measurement data, achieving automated attribution of degradation measurement data and improving the speed of attribution. Attribution of degradation measurement data degradation is achieved from two dimensions: whether the description of the degradation measurement data has changed, and whether performance degradation has occurred. Attached Figure Description

[0020] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0021] Figure 1 This is a flowchart illustrating a method for determining attribution results provided in an embodiment of this disclosure;

[0022] Figure 2 This is a flowchart illustrating another attribution result determination method provided in the embodiments of this disclosure;

[0023] Figure 3This is a schematic flowchart illustrating a method for determining the attribution results of discrepancies, provided in an embodiment of this disclosure.

[0024] Figure 4 This is a schematic diagram illustrating a scenario for filtering application change information provided in an embodiment of this disclosure;

[0025] Figure 5 This is a schematic diagram of the structure of an attribution determination device provided in an embodiment of this disclosure;

[0026] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0027] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0028] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0029] The term "comprising" and its variations as used herein are open-ended inclusion, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0030] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0031] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0032] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0033] Software metrics (such as performance indicators, i.e., numerical values ​​that measure application performance) are measures of the software's performance during runtime. Software metrics can involve two aspects:

[0034] 1. The objective performance characteristics of the software during runtime;

[0035] 2. How to measure the performance of software during runtime.

[0036] Considering the various factors that determine the objective performance of software during runtime, the factors that affect software metrics, also known as objective performance, include code, configuration, and runtime environment.

[0037] Code is the fundamental building block of software (such as applications), responsible for implementing specific functions and logic; it is a collection of defined functions and logic. Different versions of code may differ in functionality, code quality, and algorithm efficiency, leading to varying performance.

[0038] Configuration involves adjusting functionality and logic to achieve different performance levels in software. Configuration can include A / B testing and setting configurations (such as software settings options). A / B testing involves randomly dividing clients into two or more groups, displaying different software versions or feature settings to each group, and then comparing the performance of each group to determine which version or setting is superior.

[0039] The operating environment, also known as the software runtime environment, includes factors such as the version of the operating system on the device, the type of hardware, network conditions, and external dependencies. These factors can all have a significant impact on software performance.

[0040] From the perspective of how to measure software runtime performance, even if the objective performance of the software runtime remains unchanged, different methods of measuring performance will lead to different metric data. For example, whether time-consuming metrics are described using the mean or the median will result in significant differences in these metrics. Furthermore, the method for determining smoothness metrics is related to the duration of scrolling; changes in the scrolling duration will lead to changes in the determination method, thus affecting the performance of the metric data. Therefore, the method of determining the metric data itself is a very important factor affecting the metric data. The description of the method for determining the metric data can also be called the statistical caliber of the metric data. This refers to the standards and methods used when collecting metric data, determining how to obtain and determine various metric data during software runtime. The statistical caliber of metric data can include the statistical range, statistical quantity (such as mean, median, etc.), time period, data source and / or sampling rate, etc.

[0041] Therefore, in order to address the degradation of online measurement data, this disclosure explains the degradation of measurement data by attributing it to differences in the description of how the measurement data is determined or from dimensions such as code, configuration, and operating environment.

[0042] Online metrics can be considered as the overall metrics for all users of the application online. This metric data can be divided by time, such as by day, representing the current total metric data. Specifically, it includes the average cold start time for all users using the application today and the average cold start time for all users using the application yesterday.

[0043] The following describes how the attribution results of this disclosure are determined: Figure 1 This is a flowchart illustrating an attribution result determination method provided in an embodiment of this disclosure. This embodiment is applicable to situations where degradation of degradation measurement data is attributed (i.e., cause analysis of problems or defects in software). The method can be executed by an attribution result determination device, which can be implemented in the form of software and / or hardware, or optionally, by an electronic device, such as a mobile terminal, a PC, or a server.

[0044] like Figure 1 As shown, the method includes:

[0045] S110. Based on the description change information of the degradation measurement data, determine the attribution result of the description difference of the degradation measurement data.

[0046] Degradation metrics indicate metrics that indicate degradation within an application. Metrics include data that measures the application's performance. For example, when analyzing metrics over a time period, degradation may occur across different statistical periods. By analyzing metrics from different time periods, degradation metrics within the application can be identified. For instance, one could collect various metrics for the entire application today, such as cold start time (the time it takes for the application to go from a completely unresponsive state to being able to provide services). This data could then be compared to yesterday's metrics, such as cold start time, and the changes in these metrics could determine whether degradation has occurred.

[0047] There are no restrictions on how to determine the degradation metric data here, as long as the degradation metric data that shows degradation in the application can be obtained.

[0048] The description of the change indicates whether the description of how the degradation metric data was determined has changed. The description of how the metric data was determined refers to the standards and methods used to define and calculate the metric data. If the description of how the metric data was determined has changed, it may lead to a lack of comparability between data from different time periods, thus affecting the accurate judgment of application performance and status. For example, when calculating application smoothness metrics, if the definition of "smoothness" changes at different times, then the smoothness metrics from the two periods cannot be directly compared.

[0049] Therefore, in the process of analyzing degradation measurement data, this disclosure can first determine whether the description of the determination method of degradation measurement data has changed based on the description change information of the degradation measurement data. If the description of the determination method has changed, then the degradation cause of the degradation measurement data can be considered to be due to the change in the description of the determination method.

[0050] The description of the attribution results can be considered as an indication of whether a descriptive change in a specific manner caused the degradation of the degradation measurement data. The attribution results can be considered as conclusions drawn from the analysis of the causes of degradation of the degradation measurement data.

[0051] The description of the discrepancy attribution results can indicate a correlation between degradation metrics and application performance degradation; that is, application performance degradation led to the degradation of the degradation metrics. The description of the discrepancy attribution results can also indicate that the degradation metrics were caused by a change in the determination method.

[0052] After obtaining the degradation measurement data, this operation can determine the description change information of the degradation measurement data. For example, it can obtain the statistical results of measurement data that have undergone description changes in a specific manner within the application, and query whether the degradation measurement data is included in the statistical results. If it is included, it can be considered that the description of the determination method of the degradation measurement data has changed. If it is not included, it can be considered that the description of the determination method of the degradation measurement data has not changed. A change in the description of the determination method indicates that the determination method has changed.

[0053] After identifying the description change information, this operation can determine the attribution result of the description difference based on the information indicated in the description change information. For example, if the description of the method for determining the degradation metric data, as indicated by the description change information, remains unchanged (i.e., it is consistent), the attribution result of the description difference can be a correlation between the degradation of the degradation metric data and the performance degradation of the application. If the description change information indicates a change in the description of the degradation metric data, the difference can be determined after aligning the determination method to see if it is caused by the change in the description of the determination method. If the difference is not caused by the change in the description of the determination method, it can be attributed to performance degradation.

[0054] S120, if the description of the difference attribution results indicates that the degradation metric data is associated with the performance degradation of the application, determine the degradation function associated with the degradation metric data.

[0055] The description of the difference attribution results indicates that the correlation between degradation metrics and application performance degradation can be considered as the degradation of degradation metrics potentially being caused by application performance degradation. Therefore, this operation can analyze the application to determine the degradation function associated with the degradation metrics. The degradation function associated with the degradation metrics can be considered a function within the application that has caused degradation, such as a function that causes time-consuming degradation. Because this function causes time-consuming degradation, it leads to the degradation of the degradation metrics. For example, if function 'a' in the application is called and executed multiple times, it causes time consumption, which in turn leads to the degradation of cold start time.

[0056] This operation can analyze the application before and after degradation metrics data to determine the degradation function within the application. The degradation metrics data before and after degradation can be considered as two time points for determining degradation. For example, if degradation metrics data is determined based on today's and yesterday's metrics data, then the application before and after degradation metrics data can be today's application and yesterday's application. The factors affecting the software's metrics data may differ at different times.

[0057] S130. Determine the attribution results that led to the degradation of the degradation function.

[0058] After determining the degradation function, in order to improve the efficiency of solving degradation problems, this operation further locates the source of the degradation function and uses the source as the attribution result.

[0059] This operation can attribute the degradation of a degraded function from multiple dimensions. The performance of the online dashboard is jointly determined by the version composition, online configuration composition, and the overall runtime environment of the dashboard. The version composition and online configuration composition are affected by releasing new versions of the application and changing online configurations, respectively. Both of these are modifications or adjustments to the application and can be collectively referred to as application changes. Changes in the overall runtime environment of the dashboard are observed phenomena or results and can be attributed to runtime environment differences. Therefore, attributing the degradation of a function can involve identifying the application changes or runtime environment differences that lead to the function's degradation, such as runtime environment differences.

[0060] In one embodiment, determining the attribution result leading to the degradation of the degradation function includes:

[0061] The degradation function is attributed based on the impact dimension of the application associated with the degradation function, and the attribution results are obtained.

[0062] The impact dimension of an application can be the dimension of the application's application metric data performance, such as the application changes and runtime environment differences mentioned above.

[0063] This embodiment can attribute the degradation of a function to its deterioration from the perspective of application changes, and determine the attribution result. This embodiment can also attribute the degradation of a function to its deterioration from the perspective of runtime environment differences, and determine the attribution result.

[0064] Most degradation is caused by application changes, so we should start by attributing the degradation to these changes. To attribute degradation functions to specific application changes, follow these steps:

[0065] 1. Filter application changes based on their relevance to potentially degrade measurement data, thereby narrowing down the scope of the investigation;

[0066] 2. Within the scope determined in the previous step, locate the code or configuration changes that cause the degradation of the degradation function.

[0067] If application changes are excluded, the issue can be attributed to differences in the runtime environment. However, to ensure the interpretability of the attribution results, this embodiment further performs degradation attribution from the perspective of runtime environment differences to determine whether the degradation is caused by runtime environment differences. For example, when performing degradation attribution, the differences in the runtime environment before and after degradation measurement data can be statistically analyzed, and runtime environment difference items associated with the degradation function can be identified.

[0068] This embodiment can perform attribution in parallel from two dimensions: application changes and runtime environment differences. Alternatively, application change attribution can be performed first, followed by attribution from the runtime environment difference dimension.

[0069] The technical solution of this disclosure determines the description difference attribution result by using description change information of degradation measurement data. When the description difference attribution result indicates a correlation between degradation measurement data and application performance degradation, it further determines the degradation function associated with the degradation measurement data and the attribution result causing the degradation of the degradation function. This solves the problem of slow attribution speed caused by manual attribution of degradation measurement data, achieving automated attribution of degradation measurement data and improving the speed of attribution. Attribution of degradation measurement data degradation is achieved from two dimensions: whether the description of the method for determining degradation measurement data has changed, and whether performance degradation has occurred.

[0070] Based on the above embodiments, modified embodiments of the above embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in the modified embodiments.

[0071] In one embodiment, the impact dimension is an application change, and the attribution of the degradation function to the degradation function based on the application's impact dimension associated with the degradation function includes:

[0072] Obtain an application change information set for the application, the application change information set indicating changes within the application;

[0073] From the application change information set, filter the target change information associated with the degradation function;

[0074] Determine the attribution results in the target change information that led to the degradation of the degradation function.

[0075] An application change information set can indicate all application changes to an application. The application change information set can be recorded after a change occurs. It can include changes at the application's code level and changes at the application's configuration level.

[0076] This embodiment determines the cause of degradation function degradation by analyzing the application change set. The target change information can be considered as the information of in-application changes that are filtered out and associated with degradation function degradation.

[0077] This disclosure analyzes measurement data using days as the dimension. The application change information set can also be time-dimensional, such as daily. For example, the application change information set for today's application, or the application change information set for yesterday's application.

[0078] After obtaining the application change information set, this embodiment can filter target change information associated with the degradation function from the application change information, that is, target change information that may affect the degradation of the degradation function. The filtering method is not limited here; it can be based on time or on the correlation between the degradation function and the application change.

[0079] After identifying the target change information, this embodiment can further analyze the target change information to determine the attribution results leading to the degradation of the degradation function. During this further analysis, the analysis methods can be determined based on the type of target change information. The type of target change information can include configuration changes and version changes.

[0080] In one embodiment, filtering target change information associated with the degradation function from the application change information set includes:

[0081] From the set of application change information, filter application change information that meets the filtering criteria;

[0082] The filtered application change information is identified as the target change information associated with the degradation function.

[0083] The filtering criteria can be predefined conditions used to filter application change information. There are no restrictions on the criteria here, as long as filtering based on these criteria can identify application changes that may affect the degradation function. Application change information can be considered as information indicating changes within the application. Application change information can uniquely identify application changes. In the case where the application change is a version change, the application change information can be the version number.

[0084] This embodiment can traverse the content change information included in the application change information set to determine whether each application change information meets the filtering criteria. Application changes that meet the filtering criteria are identified as target change information associated with the degradation function.

[0085] In one embodiment, the filtering criteria include one or more of the following:

[0086] The indicated application changes affect the performance of the application;

[0087] The application change indicated was made earlier than the degradation time of the degradation metric data.

[0088] The difference in penetration rate before and after the indicated application change is greater than the set penetration rate threshold.

[0089] Application change information in the application change information set can be used as target change information if it meets one or more of the following conditions:

[0090] The indicated application changes affect the performance of the application;

[0091] The application change indicated was made earlier than the degradation time of the degradation metric data.

[0092] The difference in penetration rate before and after the indicated application change is greater than the set penetration rate threshold.

[0093] The change time can be the time when the application was modified. The degradation time can be the time when the degradation measurement data is considered to have degraded. The statistics of the measurement data can be based on time. In this embodiment, when determining whether the measurement data has degraded after statistics, the later of the two compared times can be determined as the time when the degradation measurement data degraded. For example, if the degradation measurement data is determined to be degraded by analyzing today's measurement data and yesterday's measurement data, then the degradation time can be today. The time example in this embodiment is not limited, and today and yesterday can be any two consecutive days.

[0094] The difference in penetration rate before and after the indicated application change can be considered as the difference between the penetration rate after the application change and the penetration rate before the application change. The two times before and after the application change can be the same as the two times at which degradation is determined by degradation metrics.

[0095] For example, let i be the metric data collected on day z of year y month. Let y be the metric data collected on day z+1 of year y month. If metric data y is worse than metric data x, then we need to obtain the application change information set. This application change information set can be a collection of all application change information from day z of year y month to day z+1 of year y month. The penetration rate of the application change before and after the change can be the penetration rate of the application change on day z+1 of year y month and the penetration rate of the application change on day z of year y month.

[0096] If the difference in penetration rate before and after an application change exceeds a set penetration rate threshold, the application change can be considered to have caused a significant difference in penetration rate. The set penetration rate threshold can be considered a threshold set for penetration rate; there is no limitation on the value of the set penetration rate threshold here, and it can be determined based on the actual scenario.

[0097] In one embodiment, determining the attribution result in the target change information that leads to the degradation function degradation includes:

[0098] In the case where the target change information is a configuration change of the application, the target change information is determined as the attribution result that caused the degradation function to deteriorate;

[0099] When the target change information is a version change of the application, a differential analysis is performed on the application before and after the version change to determine the changed function in the function call chain to which the degraded function belongs, and the merging information corresponding to the changed function is determined, and the merging information is determined as the attribution result.

[0100] When the target change information is an application configuration change, the target change information can be directly identified as the attribution result that caused the degradation function to deteriorate.

[0101] When the target change information is an application version change, differential analysis can be performed on the application before and after the version change. The object of differential analysis can be the function call chain to which the degraded function belongs, that is, the function call chain in which the degraded function resides. A function call chain can be considered as the calling relationship between functions in a program. When one function calls another function, a function call chain is formed. Each function has a caller and a callee; the caller calls the callee to perform a specific task.

[0102] During differential analysis, the function call chains of the degraded function in the application before and after the version change can be compared to identify the changed functions in the function call chain originating from the degraded function. A changed function can be considered as a function in the function call chain determined after comparison that has undergone changes. This changed function can be the function that influenced the degraded function. After identifying the changed function, the corresponding merge information can be used as the attribution result.

[0103] In this context, merge information can be considered the information associated with a Merge Request (MR). In software development, MR is used for collaborative development. When a developer completes a feature or fixes a problem, they initiate a merge request, requesting that their code be merged into the main branch or other relevant branches. Merge information may include the version of the code to be merged.

[0104] The merge request includes functions in the code it's merging. A mapping can be established between the merge information in the merge request and the functions included in the merge request. After determining the functions to be changed, the corresponding merge information can be retrieved based on this mapping.

[0105] In one embodiment, the impact dimension is the difference in the operating environment, and the attribution result obtained by attributing the degradation function to the degradation function from the impact dimension of the application associated with the degradation function includes one or more of the following:

[0106] The mapping relationship is used to find the difference in the operating environment corresponding to the degradation function, and the difference in the operating environment is determined as the attribution result. The mapping relationship includes the association between the degradation of the function and the difference in the operating environment.

[0107] Determine the equipment-related attributes corresponding to the degradation function, and select the attribute whose corresponding degradation function ratio is greater than a set ratio threshold from the equipment-related attributes to determine the attribution result;

[0108] The differences in the operating environment of the application before and after the degradation of the degradation measurement data are determined. The influence coefficients corresponding to the differences are determined from the correlation information, which indicates the influence coefficients of the differences in different operating environments on the measurement data. Based on the influence coefficients of the determined differences, the differences are selected from the determined differences, and the selected differences are determined as the attribution results.

[0109] In determining the attribution results of the degradation function based on differences in the operating environment, the following operations can be performed sequentially, or in parallel:

[0110] The mapping relationship is used to find the difference in the operating environment corresponding to the degradation function, and the difference in the operating environment is determined as the attribution result. The mapping relationship includes the association between the degradation of the function and the difference in the operating environment.

[0111] Determine the equipment-related attributes corresponding to the degradation function, and select the attribute whose corresponding degradation function ratio is greater than a set ratio threshold from the equipment-related attributes to determine the attribution result;

[0112] The differences in the operating environment of the application before and after the degradation of the degradation measurement data are determined. The influence coefficients corresponding to the differences are determined from the correlation information, which indicates the influence coefficients of the differences in different operating environments on the measurement data. Based on the influence coefficients of the determined differences, the differences are selected from the determined differences, and the selected differences are determined as the attribution results.

[0113] The mapping relationship can be a pre-built association between the degradation of different functions and differences in the runtime environment. This association represents the degradation of the function caused by the differences in the runtime environment. Therefore, after determining the degradation function, this embodiment can directly look up the runtime environment differences corresponding to the degradation function from the mapping relationship to locate the degradation of the function caused by the runtime environment differences associated with it. For example, a decrease in the cache ratio in the runtime environment leads to an increase in the time taken to refresh the interface for the first time.

[0114] Differences in operating environments can be considered as variations in the operating environment itself. Operating environments can include both software-level and hardware-level environments. Software-level operating environments may include cache usage, network conditions, etc. Hardware-level operating environments may include the operating system and model of the electronic device. Device-related attributes can be considered as attributes associated with the electronic device, such as its operating system and model. When analyzing data, de-identifying and analyzing device-related information used by the application can be helpful.

[0115] This embodiment can identify the device-related attributes of devices exhibiting degradation functions. From all device-related attributes of degradation functions, the proportion of each attribute is analyzed. Then, attributes with a proportion greater than a set threshold are selected and identified as the attribution result. Selecting attributes with a proportion greater than the set threshold can be considered an analysis of the clustering of degradation functions, examining whether clustering exists among the various device-related attributes. The set threshold is a threshold set for the proportion; its value is not limited here and can be set based on actual conditions.

[0116] The percentage can be considered as the proportion of the degradation function determined by the degradation function corresponding to the equipment-related attribute. When the degradation function corresponding to the equipment-related attribute has a high percentage in the determined degradation function, the equipment-related attribute can be identified as the attribution result.

[0117] The degradation metric data can be defined by two points in time before and after degradation, indicating the occurrence of degradation. For example, year x, month y, day z+1 and year x, month y, day z. The difference in the application's operating environment before and after degradation can be considered as the difference obtained by comparing the application's operating environment at these two points in time. Relevance information indicates the influence coefficient of different operating environment differences on the metric data. The influence coefficient reflects the degree of impact of the operating environment differences on the metric data. After identifying the difference, the corresponding influence coefficient can be retrieved from the relevance information.

[0118] In this embodiment, the degradation metric data before and after degradation can include multiple differences in the operating environment. For each difference, an impact coefficient can be assigned to a specific difference, which is then used as the attribution result.

[0119] In this embodiment, when selecting difference items based on the influence coefficient, the difference items corresponding to the influence coefficients greater than the threshold can be selected based on the threshold, or n influence coefficients with large influence coefficients can be selected to determine the difference items corresponding to the selected influence coefficients.

[0120] In one embodiment, determining the degradation function associated with the degradation metric data includes:

[0121] Obtain the running information during the operation of the application. The running information includes function call information and function execution time. The running information includes first running information and second running information. The first running information is the running information during the operation of the application at the time node before the degradation of the degradation measurement data. The second running information is the running information during the operation of the application at the degradation time when the degradation measurement data is degraded.

[0122] By comparing the first running information and the second running information, the degradation function is determined.

[0123] Operational information can be determined through tools in dynamic analysis applications, or it can be obtained by aggregating information determined by such tools. For example, operational information can be obtained by aggregating all operational information within a single day.

[0124] Function call information can reflect the function call relationships during application execution. Function execution time can characterize the execution time of functions during application execution.

[0125] The first and second operational information can be the application's operational information before and after degradation, respectively. For example, the operational information during application operation at a time point before degradation, such as day z of month y of year x, and the operational information during application operation at the degradation time, such as day z+1 of month y of year x.

[0126] After determining the first and second runtime information, this embodiment can compare the first and second runtime information, perform a difference analysis on the two, and identify the degraded functions. A degraded function can be considered a function whose execution time has increased. During the difference analysis, function call information and function execution time information can be analyzed to determine the changes between functions in the call relationship and the execution time of the functions, thereby identifying the degraded functions.

[0127] Figure 2 This is a flowchart illustrating another attribution result determination method provided in this disclosure embodiment. This embodiment refines the operations for determining and describing the differential attribution results, such as... Figure 2 As shown, this embodiment includes the following steps:

[0128] S210. Based on the description change information of the degradation measurement data, determine the attribution result of the description difference of the degradation measurement data.

[0129] S220. If the description of the difference attribution results indicates the correlation between degradation measurement data and application performance degradation, determine whether the description of the determination method of degradation measurement data indicated by change information is consistent, then execute S230; otherwise, execute S240.

[0130] If the description of the method for determining degradation measurement data is consistent, it can be assumed that the description of the method for determining degradation measurement data has not changed. Otherwise, it can be assumed that the description of the method for determining degradation measurement data has changed.

[0131] If the description of the determination method for degradation measurement data is consistent, it can be determined that the degradation of degradation measurement data is caused by inconsistent description of the non-deterministic method, which can be considered as application performance degradation, and S230 is executed.

[0132] If the descriptions of how degradation measurement data is determined are inconsistent, it can be assumed that the degradation in the degradation measurement data cannot be determined as being caused by application degradation, and the influence of inconsistent descriptions of the determination methods needs to be excluded. Execute S240.

[0133] S230, Determine that the description difference attribution result is a result indicating the correlation between the degradation metric data and the performance degradation of the application, and execute S260.

[0134] When the description of the method for determining degradation measurement data is uniform, the result of describing the difference attribution is determined as the result that indicates the performance degradation of the application, that is, the result caused by the performance degradation of the application.

[0135] S240: Align the determination method of the degradation measurement data, determine whether the degradation measurement data is degraded after the determination method of the degradation measurement data is aligned, if so, execute S250.

[0136] When there is a lack of consistency in the description of how degradation measurement data is determined, the methods for determining degradation measurement data can be aligned first. For example, the description of how degradation measurement data was determined before degradation can be changed to the description of how it was determined after degradation. After aligning the methods, the values ​​of degradation measurement data can be re-counted, and the results can be used to determine whether degradation has still occurred.

[0137] If the time taken for statistical analysis was calculated using the mean before degradation of the degradation metrics, and then the time taken for statistical analysis was calculated using the median after degradation, then the time taken using the median before degradation of the degradation metrics will be compared to check for degradation. If so, execute step 250.

[0138] If the degradation metric data deteriorates after alignment, it can be determined that the description difference attribution result is related to the degradation metric data and the description change of the determination method, that is, the degradation of the degradation metric data is caused by the change of the description of the determination method.

[0139] S250. Determine that the description difference attribution result is a result indicating the correlation between the degradation metric data and the performance degradation of the application.

[0140] S260. Determine the attribution results that led to the degradation of the degradation function.

[0141] In this embodiment, the attribution result for description differences is determined based on whether the description of the method for determining degradation metrics has changed. During the judgment process, if the description of the method for determining degradation metrics has changed, the method is first aligned before determining whether degradation has occurred. This eliminates the possibility that degradation was caused by a change in the description of the method before performance degradation analysis, thus improving the efficiency of attribution.

[0142] Based on the above embodiments, the attribution determination method provided in this disclosure is described below as an example. This attribution determination method can be considered a technical solution for attributing degradation in Android online measurement data. Attribution is performed from two dimensions: attribution based on metric description differences and attribution based on performance degradation. Performance degradation attribution is determined by aggregating and analyzing the overall system's operational information to identify degradation functions, and then attributing these functions. Attribution of degradation functions can be performed from two dimensions: application change attribution and runtime environment difference attribution. Application change attribution includes change records, change filtering, version change attribution, and online configuration change attribution. Runtime environment difference attribution is performed from two dimensions: degradation of specific functions and degradation of non-specific functions. The following is a detailed description:

[0143] When attributing discrepancies in metric descriptions, the first step is to determine if the statistical methodology has changed, specifically whether the description of how the degradation metric data was determined has changed. If so, the determination methods are aligned or converted to make them comparable, i.e., the method of determining the degradation metric data is aligned. If, after this step, the metric data is no longer degraded, then the attribution is complete; the degradation of the metric data is attributed to differences in statistical description. This situation typically occurs during a period of transition between old and new determination methods. If the determination method has not changed, or if the metric data still shows degradation even after aligning the determination methods or converting, then it can be determined that the objective performance of the metric data has indeed degraded, i.e., the attribution result of the description discrepancy indicates a correlation between the degradation metric data and the performance degradation of the application. The subsequent performance degradation attribution process then begins.

[0144] Figure 3 This is a flowchart illustrating a method for determining and describing the attribution results of differences, provided in an embodiment of this disclosure. See also... Figure 3 When attributing degradation metrics data, the process first determines whether the method for determining the degradation metrics data has changed. If so, the method for determining the degradation metrics data is aligned, and then it is determined whether the degradation metrics data deteriorates after alignment. If not, the description difference attribution result indicates that the degradation of the metrics data is caused by the difference in the determination method. If the determination method has not changed, or if degradation of the degradation metrics data occurs after alignment, the description difference attribution result indicates that the degradation metrics data is associated with the performance degradation of the application. This triggers performance degradation attribution, also known as performance degradation attribution.

[0145] Software performance is determined by its code, configuration, and runtime environment. This is reflected in the overall online performance, where these factors correspond to the application's version composition, online configuration, and the overall runtime environment.

[0146] When attributing performance degradation, the first step is to perform a comprehensive market trace, i.e., operational information analysis. To attribute performance degradation, a thorough analysis of the overall market's objective performance is required, which necessitates finding a way to describe the market's performance in detail.

[0147] Offline, traces are typically collected to obtain detailed descriptions of an application's performance during a single run. Similarly, online dashboard performance can be described using traces. However, unlike offline analysis, single-point traces—that is, the runtime information of a single user's use in a specific scenario (such as startup)—are insufficient for analyzing online dashboards. This is because the number of single-point traces collected online is enormous, making individual analysis impractical. Even if it were possible, single-point traces cannot reflect the overall dashboard situation, rendering the analysis largely meaningless. Therefore, to facilitate the analysis of overall dashboard performance, it is necessary to aggregate all collected representative traces (such as randomly sampled single-point traces) into a single dashboard aggregated trace. The dashboard aggregated trace provides a refined description of the application's overall performance in a specific scenario over a period of time online; any factors that cause degradation in the metric data will be reflected in the trace.

[0148] By comparing the aggregated traces of two large-scale data points—the comparison point and the baseline point—the causes of metric data degradation can be focused on a series of specific functions, including locks, garbage collection (GC), and other functions that reflect resource usage issues.

[0149] Specifically, using time as a dimension, such as a day, we obtain single-point traces of the application within that day and aggregate these single-point traces, such as by scenario. We aggregate single-point traces from all versions of the application within a day, obtaining aggregated data differentiated by scenario, namely, first runtime information and second runtime information. Then, we perform differential analysis on the first and second runtime information to obtain the degradation function.

[0150] After identifying the degraded function, its causes need to be attributed. That is, after finding the degraded function, to improve the efficiency of solving the degradation problem, it is necessary to further pinpoint the source of the function's degradation. Therefore, attributing the causes of a degraded function involves finding the application changes or runtime environment differences that lead to the function's degradation.

[0151] For application change attribution, a set of application change information is obtained. This set is then filtered to narrow down the application changes to be investigated. When attributing performance degradation, changes that may have contributed to the degradation are selected from the application change information set based on relevance.

[0152] Filtering can be based on filter criteria. For a change to cause a change in the metric data, the following three filter criteria must be met:

[0153] Condition 1. The change itself must affect performance, meaning the indicated application change impacts the performance of the application. This impact is independent of whether it is positive or negative; otherwise, it would not affect the metrics.

[0154] Condition 2. The change must occur before the degradation of the metric data, i.e., the time of the indicated application change is earlier than the time of degradation of the degradation metric data.

[0155] Condition 3. There is a significant difference between the impact surface of the change at the comparison point (e.g., date x, y, z) and the impact surface at the baseline point (e.g., date x, y, z+1). The impact surface can be measured by penetration rate, i.e., the difference in penetration rate before and after the indicated application change is greater than the set penetration rate threshold. For example, in an experiment, more devices participate; in a release, more devices upgrade to the new version.

[0156] Figure 4 This is a schematic diagram of a scenario for filtering application change information provided by an embodiment of this disclosure. Table 1 is a schematic table for filtering application change information provided by this disclosure.

[0157] Table 1. A schematic table of filtering application change information provided in this disclosure.

[0158] Application changes Does it affect application performance? Permeability at the benchmark point Permeability at comparison points Application Change A no 10％ 30％ Application Change B yes 0％ 30％ Application Changes C yes 30％ 30％ Application Change D yes 10％ 30％

[0159] See Figure 4 According to Table 1, only application change C is likely to cause changes in the measurement data of the comparison points: application change A does not meet condition 1; application change B does not meet condition 3; application change D does not meet condition 2.

[0160] The target change results are obtained by filtering the application change information set using the above filtering criteria. These target change results may lead to degradation of the degradation measurement data.

[0161] If the filtered target change information indicates changes including a release (version change), then the degradation point may be related to the newly released version. Based on the change logs—that is, records of application version and configuration changes, such as the application change information set—relevant information about the latest version can be obtained. Once the problematic version is identified, the next step is to locate the problematic download manager (MR).

[0162] The target change information filters out application change information related to version changes. Based on the release history, it determines the current version at the time of degradation metrics data degradation, and the version before the change when performing degradation metrics data degradation analysis. A difference analysis is performed on the application before and after the version change to identify the changed function in the function call chain containing the degradation function, i.e., to locate the change point. The MR indicated by the merge information associated with the changed function is identified as the problematic MR.

[0163] Online configuration changes in the target change information are filtered out for further attribution. Attribution can be done directly by using the configuration change as the attribution result, or the configuration change itself can be analyzed to determine the changes in metric data caused by the configuration change, and the impact of the configuration change on the metric data, thus obtaining the attribution result. Taking an A / B experiment as an example, the changes in metric data between the control group and the experimental group are analyzed to determine whether the metric data is caused by the A / B experiment. For example, the difference in degradation metric data between the control group and the experimental group is analyzed, and based on this difference, it is determined whether the configuration change should be identified as the attribution result.

[0164] When attributing runtime environment differences, the manifestation of degradation in the trace varies depending on the specific environment. Some environmental differences lead to degradation in specific functions, while others do not focus on any particular function. Therefore, different attribution methods are needed for different situations. The following examples of specific environmental differences illustrate the attribution approaches for two different types of environmental differences.

[0165] 1. Degradation of specific functions: Some environmental differences can manifest as degradation in certain functions. Take the difference in cache ratio during the first refresh (i.e., the first time the interface is refreshed) as an example: Without caching, recommendation data needs to be retrieved from the server, resulting in longer processing time compared to scenarios with caching. As the proportion of samples without caching increases, the metrics will show degradation. At this point, functions related to data retrieval will be reported as degradation functions (increased penetration). If functions related to data retrieval are reported as degradation points due to increased penetration, this can be attributed to the decrease in cache ratio within the environmental differences.

[0166] Similarly, network environment differences can lead to a deterioration in the initial refresh time when network requests slow down, especially in scenarios without caching. In such cases, the functions related to data retrieval will be reported as deterioration functions (average latency degradation). If the data retrieval functions are reported as degradation points due to an increase in average latency, this can be attributed to network degradation within the context of environmental differences.

[0167] The attribution approach for environmental differences involves first identifying one or more specific functions related to the degradation function. Then, a connection is established between the specific characteristics of these functions (such as time consumption, penetration rate, etc.) and the specific environmental differences. This mapping relationship includes the association between function degradation and operating environment differences, and is organized into a series of feature matching rules as preliminary attribution. When attributing degradation functions, these rules are used to check for matches, thereby associating the degradation function with the relevant environmental differences. In other words, the operating environment differences corresponding to the degradation function are found from the mapping relationship, and these operating environment differences are determined as the attribution result.

[0168] In addition, there are some device-related attribute issues that can manifest as degradation of specific functions, but targeted matching rules cannot be designed. For example, suppose a manufacturer releases a new system version, and applications have compatibility issues on this new system. When a large number of users upgrade to the new system in a short period of time, it may lead to degradation of metric data in a certain online scenario. This type of problem may also manifest as degradation of specific functions, but these problems are unpredictable, so it is impossible to match the corresponding environmental differences using pre-defined attribution methods. To address these problems, the device-related attributes of the degradation function should be analyzed to pinpoint the problematic manufacturer, operating system, or device model.

[0169] Regarding degradation without a specific function, some environmental differences will not manifest as degradation in a particular function. Take the difference in the proportion of high-end machines as an example: all online devices can be considered as a unified, abstract device. When the proportion of high-end machines decreases, the performance of this device degrades. Since performance degradation leads to degradation in almost all functions, the degradation points obtained from the overall trace comparison will be concentrated in higher-level functions, manifesting as widespread degradation across almost all stages. Higher-level functions can be considered as functions at or near the bottom of the stack. High-end machines can be considered as machines with performance or configuration exceeding a set level.

[0170] To assess the specific impact of these differences in operating environment on measurement data, the first step is to conduct a correlation analysis on various such environmental differences to obtain data on the influence coefficients of environmental factors on measurement data, i.e., correlation information, as a preliminary preparation for attribution.

[0171] During attribution, statistical data on this type of operating environment is first collected, i.e., the operating environment of the application before and after the degradation metric data degradation, and the differences between them before and after the degradation metric data are analyzed to obtain a series of difference points (i.e., to determine the difference items of the application's operating environment before and after the degradation metric data degradation). Then, the impact of each difference item is quantified according to the influence coefficient, and the quantification results are used to explain the changes in the metric data.

[0172] Figure 5 This is a schematic diagram of the structure of an attribution determination device provided in an embodiment of this disclosure, as shown below. Figure 5 As shown, the device includes:

[0173] The first determining module 510 is used to determine the description difference attribution result of the degradation measurement data based on the description change information of the degradation measurement data. The degradation measurement data indicates the degradation measurement data that occurs in the application. The measurement data includes data that measures the performance of the application. The description change information indicates whether the description of the determination method of the degradation measurement data has changed.

[0174] The second determining module 520 is used to determine the degradation function associated with the degradation measurement data when the description difference attribution result indicates that the degradation measurement data is associated with the performance degradation of the application.

[0175] The third determining module 530 is used to determine the attribution result that leads to the degradation of the degradation function.

[0176] The technical solution provided in this disclosure determines the attribution result of description difference based on the description change information of the degradation measurement data using a first determining module. When the description difference attribution result indicates a correlation between the degradation measurement data and application performance degradation, a second determining module further determines the degradation function associated with the degradation measurement data. A third determining module then determines the attribution result causing the degradation function's degradation. This solves the problem of slow attribution speed caused by manual attribution of degradation measurement data, achieving automated attribution of degradation measurement data and improving attribution speed. Attribution of degradation measurement data degradation is achieved from two dimensions: whether the description of the degradation measurement data has changed, and whether performance degradation has occurred.

[0177] In one embodiment, the third determining module 530 includes:

[0178] The attribution unit is used to perform degradation attribution on the degradation function from the dimension of the application's impact associated with the degradation of the degradation function to obtain attribution results.

[0179] In one embodiment, the impact dimension is application change, and the attribution unit includes:

[0180] A subunit is used to acquire an application change information set of the application, the application change information set indicating the information about changes within the application;

[0181] A filtering subunit is used to filter target change information associated with the degradation function from the application change information set;

[0182] A sub-unit is defined to determine the attribution results in the target change information that lead to the degradation of the degradation function.

[0183] In one embodiment, the filtering subunit is specifically used for:

[0184] From the set of application change information, filter application change information that meets the filtering criteria;

[0185] The filtered application change information is identified as the target change information associated with the degradation function.

[0186] In one embodiment, the filtering criteria include one or more of the following:

[0187] The indicated application changes affect the performance of the application;

[0188] The application change indicated was made earlier than the degradation time of the degradation metric data.

[0189] The difference in penetration rate before and after the indicated application change is greater than the set penetration rate threshold.

[0190] In one embodiment, the sub-unit is determined specifically for:

[0191] In the case where the target change information is a configuration change of the application, the target change information is determined as the attribution result that caused the degradation function to deteriorate;

[0192] When the target change information is a version change of the application, a differential analysis is performed on the application before and after the version change to determine the changed function in the function call chain to which the degraded function belongs, and the merging information corresponding to the changed function is determined, and the merging information is determined as the attribution result.

[0193] In one embodiment, the impact dimension is the difference in operating environment, and the attribution unit is specifically used for one or more of the following:

[0194] The mapping relationship is used to find the difference in the operating environment corresponding to the degradation function, and the difference in the operating environment is determined as the attribution result. The mapping relationship includes the association between the degradation of the function and the difference in the operating environment.

[0195] Determine the equipment-related attributes corresponding to the degradation function, and select the attribute whose corresponding degradation function ratio is greater than a set ratio threshold from the equipment-related attributes to determine the attribution result;

[0196] The differences in the operating environment of the application before and after the degradation of the degradation measurement data are determined. The influence coefficients corresponding to the differences are determined from the correlation information, which indicates the influence coefficients of the differences in different operating environments on the measurement data. Based on the influence coefficients of the determined differences, the differences are selected from the determined differences, and the selected differences are determined as the attribution results.

[0197] In one embodiment, the first determining module 510 is specifically used for:

[0198] When the description of the change information indicates a consistent method for determining the degradation measurement data, the description difference attribution result is determined to be a result indicating the correlation between the degradation measurement data and the performance degradation of the application.

[0199] In the case where the description change information indicates a change in the method of determining the degradation metric data, the method of determining the degradation metric data is aligned, and it is determined whether the degradation metric data is degraded after the method of determining the degradation metric data is aligned. If so, the description difference attribution result is determined to be a result indicating the association between the degradation metric data and the performance degradation of the application.

[0200] In one embodiment, the second determining module 520 is specifically used for:

[0201] Obtain the running information during the operation of the application. The running information includes function call information and function execution time. The running information includes first running information and second running information. The first running information is the running information during the operation of the application at the time node before the degradation of the degradation measurement data. The second running information is the running information during the operation of the application at the degradation time when the degradation measurement data is degraded.

[0202] By comparing the first running information and the second running information, the degradation function is determined.

[0203] The attribution result determination apparatus provided in this disclosure can execute the attribution result determination method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of executing the method.

[0204] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0205] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Refer to the following... Figure 6 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 6 A structural diagram of the terminal device or server in the 500.

[0206] Electronic equipment 500, including:

[0207] One or more processing devices 501;

[0208] Storage device 508, for storing one or more programs,

[0209] When the one or more programs are executed by the one or more processing devices 501, the one or more processing devices 501 implement any of the methods provided in this disclosure.

[0210] The terminal devices in this disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0211] like Figure 6 As shown, electronic device 500 may include a processing unit (e.g., central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. An edit / output (I / O) interface 505 is also connected to bus 504.

[0212] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0213] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0214] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0215] The electronic device provided in this embodiment and the attribution result determination method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0216] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the attribution result determination method provided in the above embodiments.

[0217] It should be noted that the computer-readable medium described above in this disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof.

[0218] The computer storage medium may be a storage medium for computer-executable instructions, which, when executed by a computer processor, are used to perform the methods provided in this disclosure.

[0219] Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0220] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0221] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0222] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: determine a description difference attribution result of the degradation measurement data based on description change information of the degradation measurement data, wherein the degradation measurement data indicates degradation measurement data occurring in the application, the measurement data includes data measuring the performance of the application, and the description change information indicates whether the description of the method for determining the degradation measurement data has changed;

[0223] If the description of the attribution results indicates that the degradation metric data is associated with the performance degradation of the application, determine the degradation function associated with the degradation metric data;

[0224] Determine the attribution results that led to the degradation of the degradation function.

[0225] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0226] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0227] The modules or units described in the embodiments of this disclosure can be implemented in software or hardware. The names of modules or units do not necessarily limit the specific unit; for example, the first determining module can also be described as a "module for determining the difference attribution results".

[0228] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0229] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0230] A computer program product includes a computer program that, when executed by a processor, implements the attribution result determination method provided in the embodiments of this disclosure. It possesses the corresponding functional modules and beneficial effects for implementing the method.

[0231] According to one or more embodiments of this disclosure, [Example 1] provides a method for determining attribution results, including:

[0232] Based on the description change information of the degradation measurement data, the description difference attribution result of the degradation measurement data is determined. The degradation measurement data indicates the degradation measurement data that occurs in the application. The measurement data includes data that measures the performance of the application. The description change information indicates whether the description of the method for determining the degradation measurement data has changed.

[0233] If the description of the attribution results indicates that the degradation metric data is associated with the performance degradation of the application, determine the degradation function associated with the degradation metric data;

[0234] Determine the attribution results that led to the degradation of the degradation function.

[0235] According to one or more embodiments of this disclosure, [Example 2] provides the method of Example 1, wherein determining the attribution result leading to the degradation of the degradation function includes:

[0236] The degradation function is attributed based on the impact dimension of the application associated with the degradation function, and the attribution results are obtained.

[0237] According to one or more embodiments of this disclosure, [Example 3] provides the method of Example 2, wherein the impact dimension is an application change, and the attribution of the degradation function to the degradation function from the impact dimension of the application associated with the degradation of the degradation function to obtain the attribution result includes:

[0238] Obtain an application change information set for the application, the application change information set indicating changes within the application;

[0239] From the application change information set, filter the target change information associated with the degradation function;

[0240] Determine the attribution results in the target change information that led to the degradation of the degradation function.

[0241] According to one or more embodiments of this disclosure, [Example 4] provides the method described in Example 3, wherein filtering target change information associated with the degradation function from the application change information set includes:

[0242] From the set of application change information, filter application change information that meets the filtering criteria;

[0243] The filtered application change information is identified as the target change information associated with the degradation function.

[0244] According to one or more embodiments of this disclosure, Example 5 provides the method described in Example 4, wherein the filtering conditions include one or more of the following:

[0245] The indicated application changes affect the performance of the application;

[0246] The application change indicated was made earlier than the degradation time of the degradation metric data.

[0247] The difference in penetration rate before and after the indicated application change is greater than the set penetration rate threshold.

[0248] According to one or more embodiments of this disclosure, [Example 6] provides the method of Example 3, wherein determining the attribution result in the target change information that leads to the degradation of the degradation function includes:

[0249] In the case where the target change information is a configuration change of the application, the target change information is determined as the attribution result that caused the degradation function to deteriorate;

[0250] When the target change information is a version change of the application, a differential analysis is performed on the application before and after the version change to determine the changed function in the function call chain to which the degraded function belongs, and the merging information corresponding to the changed function is determined, and the merging information is determined as the attribution result.

[0251] According to one or more embodiments of this disclosure, [Example 7] provides the method of Example 2, wherein the influence dimension is the difference in the operating environment, and the attribution of the degradation function to the degradation function from the influence dimension of the application associated with the degradation of the degradation function to obtain the attribution result includes one or more of the following:

[0252] The mapping relationship is used to find the difference in the operating environment corresponding to the degradation function, and the difference in the operating environment is determined as the attribution result. The mapping relationship includes the association between the degradation of the function and the difference in the operating environment.

[0253] Determine the equipment-related attributes corresponding to the degradation function, and select the attribute whose corresponding degradation function ratio is greater than a set ratio threshold from the equipment-related attributes to determine the attribution result;

[0254] The differences in the operating environment of the application before and after the degradation of the degradation measurement data are determined. The influence coefficients corresponding to the differences are determined from the correlation information, which indicates the influence coefficients of the differences in different operating environments on the measurement data. Based on the influence coefficients of the determined differences, the differences are selected from the determined differences, and the selected differences are determined as the attribution results.

[0255] According to one or more embodiments of this disclosure, [Example 8] provides the method of Example 1, wherein determining the attribution result of the descriptive difference of the degradation measurement data based on the descriptive change information of the degradation measurement data includes:

[0256] When the description of the change information indicates a consistent method for determining the degradation measurement data, the description difference attribution result is determined to be a result indicating the correlation between the degradation measurement data and the performance degradation of the application.

[0257] In the case where the description change information indicates a change in the method of determining the degradation metric data, the method of determining the degradation metric data is aligned, and it is determined whether the degradation metric data is degraded after the method of determining the degradation metric data is aligned. If so, the description difference attribution result is determined to be a result indicating the association between the degradation metric data and the performance degradation of the application.

[0258] According to one or more embodiments of this disclosure, [Example 9] provides the method of Example 1, wherein determining the degradation function associated with the degradation metric data includes:

[0259] Obtain the running information during the operation of the application. The running information includes function call information and function execution time. The running information includes first running information and second running information. The first running information is the running information during the operation of the application at the time node before the degradation of the degradation measurement data. The second running information is the running information during the operation of the application at the degradation time when the degradation measurement data is degraded.

[0260] By comparing the first running information and the second running information, the degradation function is determined.

[0261] According to one or more embodiments of this disclosure, [Example 10] provides an attribution result determination apparatus, comprising:

[0262] The first determining module is used to determine the attribution result of the description difference of the degradation measurement data based on the description change information of the degradation measurement data, wherein the description change information indicates whether the description of the determination method of the degradation measurement data has changed;

[0263] The second determining module is used to determine the degradation function associated with the degradation measurement data when the description difference attribution results indicate that the degradation measurement data is associated with the performance degradation of the application.

[0264] The third determining module is used to determine the attribution result that leads to the degradation of the degradation function.

[0265] According to one or more embodiments of this disclosure, [Example 11] an electronic device is provided, the electronic device comprising:

[0266] One or more processing devices;

[0267] Storage device for storing one or more programs.

[0268] When the one or more programs are executed by the one or more processing devices, the one or more processing devices perform the method as described in any of Examples 1-9.

[0269] According to one or more embodiments of this disclosure, [Example 12] provides a storage medium containing computer-executable instructions that, when executed by a computer processor, are used to perform the method described in any of Examples 1-9.

[0270] According to one or more embodiments of this disclosure, [Example 13] provides a computer program product including a computer program that, when executed by a processor, implements the method according to any one of Examples 1-9.

[0271] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0272] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0273] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for determining attribution results, characterized in that, include: Based on the description change information of the degradation measurement data, the description difference attribution result of the degradation measurement data is determined. The degradation measurement data indicates the degradation measurement data that occurs in the application. The measurement data includes data that measures the performance of the application. The description change information indicates whether the description of the method for determining the degradation measurement data has changed. If the description of the attribution results indicates that the degradation metric data is associated with the performance degradation of the application, determine the degradation function associated with the degradation metric data; Determine the attribution results that led to the degradation of the degradation function.

2. The method according to claim 1, characterized in that, The determination of the attribution results leading to the degradation of the degradation function includes: The degradation function is attributed based on the impact dimension of the application associated with the degradation function, and the attribution results are obtained.

3. The method according to claim 2, characterized in that, The impact dimension is application change. The attribution of the degradation function to obtain the attribution result from the impact dimension of the application associated with the degradation function includes: Obtain an application change information set for the application, the application change information set indicating changes within the application; From the application change information set, filter the target change information associated with the degradation function; Determine the attribution results in the target change information that led to the degradation of the degradation function.

4. The method according to claim 3, characterized in that, The step of filtering target change information associated with the degradation function from the application change information set includes: From the set of application change information, filter application change information that meets the filtering criteria; The filtered application change information is identified as the target change information associated with the degradation function.

5. The method according to claim 4, characterized in that, The filtering criteria include one or more of the following: The indicated application changes affect the performance of the application; The application change indicated was made earlier than the degradation time of the degradation metric data. The difference in penetration rate before and after the indicated application change is greater than the set penetration rate threshold.

6. The method according to claim 3, characterized in that, The determination of the attribution results in the target change information that led to the degradation of the degradation function includes: In the case where the target change information is a configuration change of the application, the target change information is determined as the attribution result that caused the degradation function to deteriorate; When the target change information is a version change of the application, a differential analysis is performed on the application before and after the version change to determine the changed function in the function call chain to which the degraded function belongs, and the merging information corresponding to the changed function is determined, and the merging information is determined as the attribution result.

7. The method according to claim 2, characterized in that, The impact dimension refers to differences in the operating environment. The attribution result obtained by attributing the degradation function to the application based on the impact dimension associated with the degradation function includes one or more of the following: The mapping relationship is used to find the difference in the operating environment corresponding to the degradation function, and the difference in the operating environment is determined as the attribution result. The mapping relationship includes the association between the degradation of the function and the difference in the operating environment. Determine the equipment-related attributes corresponding to the degradation function, and select the attribute whose corresponding degradation function ratio is greater than a set ratio threshold from the equipment-related attributes to determine the attribution result; The differences in the operating environment of the application before and after the degradation of the degradation measurement data are determined. The influence coefficients corresponding to the differences are determined from the correlation information, which indicates the influence coefficients of the differences in different operating environments on the measurement data. Based on the influence coefficients of the determined differences, the differences are selected from the determined differences, and the selected differences are determined as the attribution results.

8. The method according to claim 1, characterized in that, The step of determining the attribution result of the descriptive difference in the degradation measurement data based on the descriptive change information of the degradation measurement data includes: When the description of the change information indicates a consistent method for determining the degradation measurement data, the description difference attribution result is determined to be a result indicating the correlation between the degradation measurement data and the performance degradation of the application. In the case where the description change information indicates a change in the method of determining the degradation metric data, the method of determining the degradation metric data is aligned, and it is determined whether the degradation metric data is degraded after the method of determining the degradation metric data is aligned. If so, the description difference attribution result is determined to be a result indicating the association between the degradation metric data and the performance degradation of the application.

9. The method according to claim 1, characterized in that, The degradation function associated with the degradation metric data includes: Obtain the running information during the operation of the application. The running information includes function call information and function execution time. The running information includes first running information and second running information. The first running information is the running information during the operation of the application at the time node before the degradation of the degradation measurement data. The second running information is the running information during the operation of the application at the degradation time when the degradation measurement data is degraded. By comparing the first running information and the second running information, the degradation function is determined.

10. An attribution result determination device, characterized in that, include: The first determining module is used to determine the attribution result of the description difference of the degradation measurement data based on the description change information of the degradation measurement data. The degradation measurement data indicates the degradation measurement data that occurs in the application. The measurement data includes data that measures the performance of the application. The description change information indicates whether the description of the determination method of the degradation measurement data has changed. The second determining module is used to determine the degradation function associated with the degradation measurement data when the description difference attribution results indicate that the degradation measurement data is associated with the performance degradation of the application. The third determining module is used to determine the attribution result that leads to the degradation of the degradation function.

11. An electronic device, characterized in that, The electronic device includes: One or more processing devices; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processing devices, the one or more processing devices perform the method as described in any one of claims 1-9.

12. A storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to perform the method as described in any one of claims 1-9.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-9.

Images