Performance Index Processing Method, Apparatus, Device, and Storage Medium

Abstract

The present disclosure provides a method, apparatus, device, and storage medium for processing performance indicators. The method for processing performance indicators may include: when the target performance indicator of an object is abnormal at the current moment, determining candidate performance indicators that are abnormal at the current moment among other performance indicators of the object; extracting key change points from the historical time series of the target performance indicator and the historical time series of the candidate performance indicators; determining the correlation between the target performance indicator and each candidate performance indicator of the candidate performance indicators based on the extracted key change points; and determining a relevant performance indicator associated with the target performance indicator from the candidate performance indicators according to the determined correlation.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to a performance index processing method, apparatus, device, and storage medium for automatically matching performance index time series data associated with a time series of a target performance index. Background Art

[0002] Audio and video are an indispensable part of modern people's lives. In order to manage audio and video product services, engineers monitor and collect a large amount of performance index data, such as release success rate, user first-screen duration, and live broadcast failure rate. These indexes reflect the usage of audio and video products. The data form of the performance index is a univariate / single time series, that is, only a single variable changes with time at each moment. The index data of the performance index can be called a time series.

[0003] In the fault diagnosis work of audio and video products, in order to deeply understand the causes and impacts of abnormal occurrences of target performance indexes, engineers usually manually query a large number of time series of other performance indexes and find time series associated with the time series of the target performance index, so as to determine other performance indexes associated with the target performance index. However, it is time-consuming and laborious to determine the performance indexes associated with the target performance index by manually checking the associated time series. Summary of the Invention

[0004] The present disclosure provides a performance index processing method, a performance index processing apparatus, an electronic device, and a storage medium for automatically matching performance index time series data associated with a time series of a target performance index, so as to solve at least the above-mentioned problems.

[0005] According to a first aspect of an embodiment of the present disclosure, a performance index processing method is provided, which may include: when an abnormality occurs in a target performance index of an object at a current moment, determining candidate performance indexes that have an abnormality at the current moment among other performance indexes of the object; extracting key change points from a historical time series of the target performance index and a historical time series of the candidate performance indexes; determining a correlation between the target performance index and each candidate performance index of the candidate performance indexes based on the extracted key change points; and determining a relevant performance index associated with the target performance index from the candidate performance indexes according to the determined correlation.

[0006] Optionally, determining the correlation between the target performance metric and each candidate performance metric based on the extracted key change points may include: determining the corresponding time in the historical time series for the key change points; according to the corresponding time, determining the number of key change points of the target performance metric and each candidate performance metric that have the same time in the historical time series; and determining the correlation between the target performance metric and each candidate performance metric based on the number of key change points having the same time.

[0007] Optionally, determining the correlation between the target performance metric and each candidate performance metric based on the number of key change points having the same time may include: calculating a first correlation result for the target performance metric according to the number of key change points having the same time and the number of key change points of the target performance metric; respectively calculating a second correlation result for each candidate performance metric according to the number of key change points having the same time and the number of key change points of each candidate performance metric; and determining the correlation between the target performance metric and each candidate performance metric by respectively comparing the first correlation result and the second correlation result with a predetermined value.

[0008] Optionally, it may be determined that a performance metric is abnormal at the current moment based on the following: determining a reference range based on the time series of the performance metric within a predetermined time period at the current moment; and if the value of the performance metric at the current moment exceeds the reference range, determining that the performance metric is abnormal at the current moment.

[0009] Optionally, determining the reference range based on the metric data of the performance metric within a predetermined time period at the current moment may include: obtaining the mean and standard deviation of the time series based on the time series of the performance metric within a predetermined time period at the current moment; and determining the reference range according to the mean and the standard deviation.

[0010] Optionally, extracting key change points from the historical time series of the target performance metric and the historical time series of the candidate performance metric may include: respectively calculating the mean and standard deviation corresponding to the time series of the target performance metric and the candidate performance metric based on the historical time series; and determining the data points in the historical time series that exceed the reference range as key change points, where the reference range is formed by the mean and the standard deviation.

[0011] Optionally, extracting key change points from the historical time series of the target performance metric and the historical time series of the candidate performance metric may include: calculating the difference sequences of the target performance metric and the candidate performance metric respectively based on the historical time series; obtaining the trough points and / or peak points in the difference sequences, and using the trough points and the peak points as the key change points.

[0012] Optionally, extracting key change points from the historical time series of the target performance metric and the historical time series of the candidate performance metric may include: calculating the difference sequences of the target performance metric and the candidate performance metric respectively based on the historical time series; obtaining the data points in the difference sequences where the difference in the front and back trend changes exceeds a threshold; using the data points where the difference in the front and back trend changes exceeds the threshold as the key change points.

[0013] According to a second aspect of the embodiments of the present disclosure, there is provided a performance metric processing device, which may include: a determination module configured to determine, when an abnormality occurs in the target performance metric of an object at the current moment, the candidate performance metrics that have an abnormality at the current moment among the other performance metrics of the object; an extraction module configured to extract key change points from the historical time series of the target performance metric and the historical time series of the candidate performance metric; an analysis module configured to determine the correlation between the target performance metric and each candidate performance metric based on the extracted key change points; and determine the relevant performance metric associated with the target performance metric from the candidate performance metrics according to the determined correlation.

[0014] Optionally, the analysis module may be configured to: determine the corresponding moments of the key change points in the historical time series; according to the corresponding moments, determine the number of key change points of the target performance metric and each candidate performance metric that have the same moment in the historical time series; and determine the correlation between the target performance metric and each candidate performance metric based on the number of key change points that have the same moment.

[0015] Optionally, the analysis module may be configured to: calculate a first correlation result for the target performance metric according to the number of key change points that have the same moment and the number of key change points of the target performance metric; calculate second correlation results for each candidate performance metric respectively according to the number of key change points that have the same moment and the number of key change points of each candidate performance metric; and determine the correlation between the target performance metric and each candidate performance metric by comparing the first correlation result and the second correlation results with a predetermined value respectively.

[0016] Optionally, the determination module may be configured to perform: determining a reference range based on a time series of a performance metric within a predetermined time period at the current moment; and determining that the performance metric is abnormal at the current moment if the value of the performance metric at the current moment exceeds the reference range.

[0017] Optionally, the determination module may be configured to perform: obtaining a mean value and a standard deviation of the time series based on the time series of the performance metric within a predetermined time period at the current moment; and determining the reference range according to the mean value and the standard deviation.

[0018] Optionally, the analysis module may be configured to perform: respectively calculating a mean value and a standard deviation corresponding to the time series of the target performance metric and the candidate performance metric based on the historical time series; and determining data points in the historical time series that exceed the reference range as key change points, where the reference range is formed by the mean value and the standard deviation.

[0019] Optionally, the analysis module may be configured to perform: respectively calculating a difference sequence between the target performance metric and the candidate performance metric based on the historical time series; obtaining trough points and / or peak points in the difference sequence, and using the trough points and the peak points as the key change points.

[0020] Optionally, the analysis module may be configured to perform: respectively calculating a difference sequence between the target performance metric and the candidate performance metric based on the historical time series; obtaining data points in the difference sequence where the difference in the front and back trend changes exceeds a threshold; and using the data points where the difference in the front and back trend changes exceeds the threshold as the key change points.

[0021] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, which may include: at least one processor; and at least one memory storing computer-executable instructions, wherein when the computer-executable instructions are run by the at least one processor, the at least one processor is caused to execute the performance metric processing method as described above.

[0022] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium storing instructions, which when run by at least one processor, cause the at least one processor to execute the performance metric processing method as described above.

[0023] According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product, wherein instructions in the computer program product are run by at least one processor in an electronic device to execute the performance metric processing method as described above.

[0024] The technical solutions provided by the embodiments of the present disclosure at least bring the following beneficial effects:

[0025] By extracting the key change points of the time series and performing correlation calculations, relevant performance indicators of different categories are unsupervised found from a large number of performance indicators for the target performance indicator, thus providing more analysis clues for fault diagnosis.

[0026] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present disclosure. Brief Description of the Drawings

[0027] The drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0028] Figures 1a to 1d is a schematic diagram of the performance indicators of an audio-visual product;

[0029] Figure 2 is a flowchart of a performance indicator processing method according to an embodiment of the present disclosure;

[0030] Figure 3 is a schematic flow diagram of a performance indicator processing method according to an embodiment of the present disclosure;

[0031] Figure 4 is a block diagram of a performance indicator processing device according to an embodiment of the present disclosure;

[0032] Figure 5 is a schematic structural diagram of a performance indicator processing device according to an embodiment of the present disclosure;

[0033] Figure 6 is a block diagram of an electronic device according to an embodiment of the present disclosure.

[0034] Throughout the drawings, it should be noted that the same reference numerals are used to represent the same or similar elements, features, and structures. Detailed Description of the Embodiments

[0035] In order to enable those of ordinary skill in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings.

[0036] The following description with reference to the drawings is provided to assist in a comprehensive understanding of the embodiments of the present disclosure defined by the claims and their equivalents. Various specific details are included to assist in the understanding, but these details are only regarded as exemplary. Thus, those of ordinary skill in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and structures are omitted for clarity and conciseness.

[0037] The terms and words used in the following description and claims are not limited to their written meanings, but are used by the inventors only to achieve a clear and consistent understanding of the present disclosure. Therefore, it should be clear to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustrative purposes only and not for the purpose of limiting the present disclosure as defined by the claims and their equivalents.

[0038] It should be noted that the terms "first", "second", etc. in the specification, claims and above-mentioned drawings of the present disclosure are used to distinguish similar objects and do not necessarily have to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0039] Figures 1a to 1d Data showing the stuttering rate performance index, download failure rate performance index, hd19 playback ratio performance index, and upload success rate performance index of a certain audio-visual product are respectively presented. When problems and failures occur in the audio-visual product (such as network failures, data center failures, software vulnerabilities, etc.), corresponding abnormal changes (such as sudden increases, sudden decreases, etc.) will also be shown in the time series data of the performance indicators. In daily operation and maintenance management work, it is difficult to avoid failures of product services, and these failures will also spread to other indicators with business associations or logical call relationships, resulting in corresponding abnormalities in the data of these indicators. For example, for the release success rate indicator, when there are many indicators such as the number of upload error codes and TCP connection failure rate in the relevant time series, the cause of this failure may be the poor network status of the user.

[0040] In related technologies, for example, the Pearson correlation coefficient and the Spearman rank correlation coefficient can be used to find time series with similar change trends to the time series data of abnormal WiFi hotspots. The correlation coefficients of time series with similar change trends have higher values, so that all similar abnormal pattern data can be found. Or, the Granger causality analysis algorithm can be used to analyze the predictive correlation relationship between time series, and the time series that helps linearly predict a certain time series is the cause, so as to find the relevant time series that causes the abnormality. These classical performance index processing methods focus on the overall correlation of time series globally, including linear correlation and predictive correlation. However, these methods lack the capture and analysis of key data points in the time series, and are not very helpful for fault diagnosis work.

[0041] For another example, the CoFlux algorithm is designed to study the abnormal fluctuation relationship between time series, thereby assisting in fault diagnosis. This method first uses feature engineering to extract all abnormal fluctuation features of the time series, and then calculates the correlation degree between time series using the cross-correlation algorithm based on these abnormal fluctuation features. The abnormal fluctuation correlation results of the time series include whether there is fluctuation correlation, whether the positive and negative of the fluctuation are consistent, and the sequence of the fluctuation. This method focuses on the global change situation and historical rules of the time series and cannot determine whether the time series is abnormal at the latest moment. Although this algorithm calculates the time sequence of abnormal fluctuations to assist in causal judgment, for audio and video data, the time granularity of the time series is usually in seconds and minutes, which makes it difficult to distinguish the sequence. This algorithm also cannot calculate the unidirectionality and bidirectionality of the correlation results. In addition, the time complexity of this algorithm is high, and it is mainly applied to offline analysis and difficult to be applied to online fault diagnosis.

[0042] Based on this, the present disclosure designs a performance index processing method based on key change points, which can automatically find the time series related to the target time series. Compared with the classical global linear correlation and prediction correlation, the present disclosure focuses on the key data points in the time series; compared with the abnormal fluctuation correlation analysis of the time series, the present disclosure can first screen the time series that is abnormal at the latest moment and further analyze the unidirectionality and bidirectionality of the related time series, which is more conducive to fault analysis work.

[0043] Hereinafter, according to various embodiments of the present disclosure, the methods and apparatuses of the present disclosure will be described in detail with reference to the drawings.

[0044] Figure 2 is a flowchart of the performance index processing method according to an embodiment of the present disclosure. The performance index processing method according to the present disclosure can be applied to the performance indexes of various objects, such as the performance indexes of audio and video products, applications, etc., to more quickly and better troubleshoot service faults.

[0045] The performance index processing method according to the present disclosure can be executed by any electronic device. The electronic device can be the terminal where the user is located. The electronic device can be at least one of a smart phone, a tablet computer, a portable computer, a desktop computer, etc. For example, the electronic device can be installed with a target application to find other performance index data associated with it when a fault occurs in the target performance index, so as to better and more quickly analyze the cause of the fault.

[0046] Refer to Figure 2, in step S201, when the target performance metric of an object is abnormal at the current moment, determine the candidate performance metrics among the other performance metrics of the object that are abnormal at the current moment. The object may include application software, etc. For example, the performance metrics of an application may include release success rate, user first-screen time, live broadcast failure rate, stuttering rate, download failure rate, hd19 playback ratio, upload success rate, etc., as Figures 1a to 1d shown. In the present disclosure, the target performance metric may represent an index that a user expects to monitor in real time, and the other performance metrics may represent performance metrics other than the target performance metric. The present disclosure places no restrictions on the types and quantities of the target performance metric and the other performance metrics.

[0047] For each data point in the time series of the target performance metric, an anomaly detection method based on year-on-year and month-on-month comparisons can be used to determine whether the target performance metric is abnormal. Specifically, a reference range can be determined based on the time series of the performance metric within a predetermined time period at the current moment. If the value of the performance metric at the current moment exceeds the determined reference range, it is determined that the performance metric is abnormal at the current moment.

[0048] As an example, the mean and standard deviation of the time series can be calculated based on the time series of the performance metric within a predetermined time period at the current moment. If the value of the performance metric at the current moment is within the numerical range determined by the mean and the standard deviation, it is determined that the performance metric is not abnormal at the current moment; if the value of the performance metric at the current moment exceeds the determined reference range, it can be determined that the performance metric is abnormal at the current moment. In addition, the above detection method is only exemplary, and other detection methods can also be used to determine whether the value at the current moment is abnormal.

[0049] Nearby data of the target performance metric at the current moment (such as the time series including the current moment within a period of time before the current moment) (i.e., month-on-month) and nearby data of the target performance metric at the same moment in the historical data as the current moment (such as the time series including the same moment within a period of time before the same moment) (i.e., year-on-year) can be selected, and then the mean and standard deviation of the selected data are calculated. If the value of the target performance metric at the current moment < mean + 3 times the standard deviation and the value at the current moment > mean - 3 times the standard deviation, the data at the current moment is normal; otherwise, the data at the current moment is abnormal. The above example is only exemplary, and the present disclosure is not limited thereto.

[0050] After the target performance metric is abnormal at the current moment, the index performance processing algorithm of the present disclosure can be triggered. First, it can be determined whether the other performance metrics of the application are also abnormal at the current moment according to the above-described anomaly detection method based on year-on-year and month-on-month comparisons, and the performance metrics that are also abnormal at the current moment can be used as candidate performance metrics for subsequent index correlation analysis.

[0051] In step S202, key change points are extracted from the historical time series of the target performance metric and the historical time series of the candidate performance metric. For example, the key change points may include at least one of the abnormal points, peak and valley points, and data points with a difference in the front and back trend changes exceeding a threshold in the metric time series.

[0052] For the extraction of abnormal points, the mean and standard deviation of the time series of the target performance metric and the candidate performance metric can be calculated respectively based on the historical time series of the target performance metric and the candidate performance metric within a historical time (such as three days, one week, etc.). The data points in the historical time series of the target performance metric and the candidate performance metric that exceed the reference range formed by the corresponding mean and standard deviation are determined as abnormal points and used as key change points.

[0053] As an example, for each performance metric in the target performance metric and the candidate performance metric, historical data such as the time series for a historical period is obtained. The abnormal points that occur for each performance metric during this period are found from the obtained historical data according to the anomaly detection method based on year-on-year and month-on-month described above.

[0054] For the extraction of peak and valley points, based on the historical time series of the target performance metric and the candidate performance metric, the difference sequences of the target performance metric and the candidate performance metric are calculated respectively. The valley points and / or peak points in the difference sequences are obtained, and the valley points and the peak points are used as key change points.

[0055] Specifically, the difference sequences of the target performance metric and the candidate performance metric can be calculated respectively based on the historical time series of the target performance metric and the candidate performance metric within a historical time. For each data point in the difference sequence, if the difference data within a specific time period before this data point is continuously negative and the difference data within a specific time period after this data point is continuously positive, then this data point is determined as a valley point; otherwise, this data point is determined as a peak point. The valley points and the peak points are used as key change points.

[0056] As an example, for each performance metric in the target performance metric and the candidate performance metric, historical data such as the time series for a historical period is obtained. Then the difference sequence of each performance metric time series is calculated. For each data point in each difference sequence, if the difference data for a period of time (the time length is a hyperparameter, such as half an hour, etc.) before a certain time point is continuously negative and the difference data for a period of time after that is continuously positive, then this time point is a valley point; if the difference data for a period of time before a certain time point is continuously positive and the difference data for a period of time after that is continuously negative, then this time point is a peak point.

[0057] For data points with a large difference in the trend change before and after, based on the historical time series of the target performance metric and the candidate performance metric, calculate the difference sequences of the target performance metric and the candidate performance metric respectively, obtain the data points in the difference sequences where the difference in the trend change before and after exceeds the threshold, and use the data points where the difference in the trend change before and after exceeds the threshold as key change points.

[0058] Specifically, the difference sequences of the target performance metric and the candidate performance metric can be calculated respectively based on the historical time series of the target performance metric and the candidate performance metric within the historical time. For each data point in the difference sequence, calculate the first average difference within a specific time period before this data point and the second average difference within a specific time period after this data point based on the difference sequence. If the ratio between the first average difference and the second average difference is greater than the threshold, then determine this data point as a data point with a large difference in the trend change before and after, and use the data point with a large difference in the trend change before and after as a key change point.

[0059] As an example, for each performance metric among the target performance metric and the candidate performance metric, obtain the historical data of each performance metric, such as the time series for a certain period of history. Then calculate the difference sequence of each performance metric time series. For each data point in each difference sequence, calculate the average difference for a period of time (the time length is a hyperparameter, such as half an hour) before and after a certain time point, that is, the average difference before and the average difference after. If max( average difference 前 / average difference 后 , average difference 后 / average difference 前 ) > a certain threshold (the threshold is a hyperparameter, such as 5), then determine that the trend change difference at this time point is large, and determine this time point as a key change point.

[0060] The above exemplary method for extracting key change points is only exemplary, and the present disclosure is not limited thereto.

[0061] In step S203, based on the extracted key change points, determine the correlation between the target performance metric and each candidate performance metric of the candidate performance metrics. After obtaining the key change points of the target performance metric and the candidate performance metrics, the correlation calculation can be performed using the moments of the key change points. That is, whether the critical moments of the target time series (the metric data of the target performance metric) and the candidate time series (the metric data of the candidate performance metric) are correlated, and the correlation result reflects the degree of consistency of the time positions of the key change points.

[0062] Specifically, the corresponding moment of the key change point in the historical time series can be determined. Based on the corresponding moment, the number of key change points of the target performance metric and each candidate performance metric at the same moment in the historical time series can be determined. The correlation between the target performance metric and each candidate performance metric can be determined based on the number of key change points at the same moment. The present disclosure can accurately quantify the correlation between different metrics based on the number of key change points at the same moment, thereby improving the accuracy and reliability of the determined relevant performance metrics.

[0063] The first correlation result for the target performance metric can be calculated based on the number of key change points at the same moment and the number of key change points of the target performance metric. The second correlation result for each candidate performance metric can be calculated respectively based on the number of key change points at the same moment and the number of key change points of each candidate performance metric. The correlation between the target performance metric and each candidate performance metric can be determined by comparing the first correlation result and the second correlation result with a predetermined value respectively.

[0064] As an example, the corresponding moment of the key change points extracted in step S202 in the historical time series can be determined. Based on these corresponding moments, the number of key change points of the target performance metric and each candidate performance metric at the same moment in the historical time series can be determined. The correlation between the target performance metric and each candidate performance metric can be determined based on the number of key change points at the same moment. For example, the first correlation result for the target performance metric can be calculated based on the number of key change points at the same moment and the number of key change points of the target performance metric. The second correlation result for each candidate performance metric can be calculated respectively based on the number of key change points at the same moment and the number of key change points of each candidate performance metric. Based on the comparison of the first correlation result and the second correlation result with the predetermined value, it can be determined whether the target performance metric and each candidate performance metric have a two-way correlation or a one-way correlation.

[0065] For example, the correlation result 目标 (i.e., the first correlation result)= #common(target time series key change point moment, candidate relevant time series key change point moment) / # target time series key change points.

[0066] The correlation result 候选 (the second correlation result)= #common(target time series key change point moment, candidate relevant time series key change point moment) / # candidate relevant time series key change point number.

[0067] Wherein, #common() represents the number of key change points with the same time in two time series, #key change points of the target time series represents the number of key change points of the target time series, and #key change points of the candidate related time series represents the number of key change points of the candidate related time series.

[0068] Correlation result 目标 and the correlation result 候选 range from 0 to 1. The higher the value, the higher the degree of correlation; the lower the value, the lower the degree of correlation. The thresholds for high and low degrees of correlation are hyperparameters. For example, the high correlation threshold can be 0.8, and the low threshold can be 0.2. That is, the correlation results greater than or equal to 0.8 are determined as high correlation results, and the correlation results less than or equal to 0.2 are determined as low correlation results.

[0069] If the correlation result 目标 is high and the correlation result 候选 is high, then it is determined that the target time series and the candidate related time series are bidirectionally correlated.

[0070] If the correlation result 目标 is low and the correlation result 候选 is low, then it is determined that the target time series and the candidate related time series are bidirectionally unrelated.

[0071] If the correlation result 目标 is high and the correlation result 候选 is low, then it is determined that the target time series and the candidate related time series are unidirectionally correlated, and the candidate related time series causes the target time series to change / abnormal.

[0072] If the correlation result 目为 is low and the correlation result 候选 is high, then it is determined that the target time series and the candidate related time series are unidirectionally correlated, and the target time series causes the candidate related time series to change / abnormal.

[0073] According to the embodiments of the present disclosure, by comprehensively considering the correlation results of the target performance index and the correlation results of the candidate performance index, the mutual influence relationship between the candidate performance index and the target performance index can be determined, so as to more accurately locate the cause of the abnormal situation of the target performance index.

[0074] In step S204, the related performance index associated with the target performance index is determined from the candidate performance indexes according to the determined correlation.

[0075] Based on the above correlation results, relevant performance indicators of different categories can be found from a large number of performance indicators for the target performance indicator without supervision, thereby providing more analysis clues for fault diagnosis and enabling better and faster analysis of the causes of faults.

[0076] Figure 3 It is a schematic flowchart of a performance indicator processing method according to an embodiment of the present disclosure.

[0077] Referring to Figure 3 , the target time series and other time series (i.e., the candidate time series set) can be obtained and monitored in real time. Here, the target time series can represent the indicator data of the target performance indicator, and the candidate time series can represent the indicator data of the candidate performance indicators.

[0078] For each data point of the target time series at each moment, an anomaly detection method based on year-on-year and month-on-month ratios can be used to determine whether the target time series has an anomaly at the current moment.

[0079] As an example, the nearby data of the target time series at the current moment (i.e., the latest moment) and the nearby data of the same moment in the historical indicator data of the target time series can be selected, and the mean and standard deviation of the selected indicator data are calculated. If the indicator value at the current moment < mean + 3 times the standard deviation and the indicator value at the current moment > mean - 3 times the standard deviation, then the data point at the current moment can be determined to be normal; otherwise, the data point at the current moment can be determined to be abnormal.

[0080] When the target time series has an anomaly at the current moment, since the performance of each candidate time series in the candidate time series set at the current moment is very important, that is, only the time series that is also abnormal at the current moment may cause the current anomaly of the target time series, the indicator performance processing algorithm for the target time series can be triggered. At this time, it is necessary to determine whether each candidate time series in the candidate time series set also has an anomaly at the current moment.

[0081] Due to the large number of candidate time series, in order to quickly screen out the possible relevant time series, the above-mentioned anomaly detection method based on year-on-year and month-on-month ratios can be used to determine whether the candidate time series has an anomaly at the current moment. In the candidate time series set, the candidate time series that is abnormal at the current moment can be screened out.

[0082] Next, the key change points of the target time series and the screened candidate time series can be extracted. The key change points can include at least one of anomaly points, peak and valley points, and data points with large differences in the front and back trend changes.

[0083] When extracting key change points, historical metric data for a period before the current moment when an anomaly occurs can be obtained separately for the target time series and the selected candidate time series, and then key change points can be extracted based on the obtained historical metric data.

[0084] As an example, when extracting anomaly points, the anomaly detection method based on year-on-year and month-on-month described above can be used to extract anomaly points from the obtained historical metric data.

[0085] When extracting peak and valley points, the difference sequence of each time series can be calculated first. If the difference data for a period (the length of the period is a hyperparameter, such as half an hour) before a certain time point is continuously negative and the difference data for a period after this time point is continuously positive, then this time point is a valley point; if the difference data for a period before a certain time point is continuously positive and the difference data for a period after this time point is continuously negative, then this time point is a peak point.

[0086] When extracting data points with a large difference in trend changes before and after, since the difference sequence of each time series can well reflect the trend changes of the data, the difference sequence of each time series can be used to determine the data points with a large difference in trend changes.

[0087] For data points with a large difference in trend changes, the degree of change in the difference between the data before and after is large. Therefore, using the difference sequence of each time series, the average difference for a period (the length of the period is a hyperparameter, such as half an hour) before and after a certain data point can be calculated to obtain the average difference for the period before this data point (average difference 前 ) and the average difference for the period after this data point (average difference 后 ).

[0088] If max(average difference 前 / average difference 后 , average difference 后 / average difference 前 ) > threshold (the threshold is a hyperparameter, such as 5), then it can be determined that the trend change of this data point is large, and this data point is used as a data point with a large difference in trend changes.

[0089] By using representative data points in the time series of performance metrics as key change points, the target performance metric can be more accurately matched to other associated performance metrics, while reducing the computational amount for matching related performance metrics.

[0090] After obtaining the key change points of the target time series and the selected candidate time series (which may be referred to as candidate related time series hereinafter), the moments corresponding to these key change points can be used for correlation calculation, that is, to determine whether the critical moments (i.e., the moments corresponding to the key change points) of the target time series and the candidate related time series are relevant. The correlation result can reflect the degree of consistency of the time positions of the key change points.

[0091] As an example, the following equation can be used to calculate the correlation result for the target time series (correlation result 目标 ) and the correlation result for each candidate related time series (correlation result 候选 ).

[0092] Correlation result 目标 =#common(moments of key change points of the target time series, moments of key change points of the candidate related time series) / #key change points of the target time series.

[0093] Correlation result 候选 =#common(moments of key change points of the target time series, moments of key change points of the candidate related time series) / #number of key change points of the candidate related time series.

[0094] Where #common() represents the number of key change points with the same moment in two time series, #key change points of the target time series represents the number of key change points of the target time series, and #number of key change points of the candidate related time series represents the number of key change points of the candidate related time series.

[0095] Correlation result 目标 and correlation result 候选 range from 0 to 1. The higher the value, the higher the degree of correlation; the lower the value, the lower the degree of correlation. The thresholds for high and low degrees of correlation are hyperparameters. For example, the high degree of correlation threshold can be 0.8, and the low threshold can be 0.2.

[0096] If the correlation result 目标 is high and the correlation result 候选 is high, then it is determined that the target time series and the candidate related time series are bidirectionally correlated.

[0097] If the correlation result 目标 is low and the correlation result 候选 is low, then it is determined that the target time series and the candidate related time series are bidirectionally irrelevant.

[0098] If the correlation result 目标 is high and the correlation result 候选If it is low, it is determined that the target time series and the candidate relevant time series are unidirectionally correlated, and the candidate relevant time series causes the target time series to change.

[0099] If the correlation result 目为 is low and the correlation result 候选 is high, it is determined that the target time series and the candidate relevant time series are unidirectionally correlated, and the target time series causes the candidate relevant time series to change.

[0100] Based on the above correlation results, relevant time series of different categories can be found for the target time series, thereby providing more analysis clues for fault diagnosis.

[0101] Figure 4 It is a block diagram of a performance index processing device according to an embodiment of the present disclosure.

[0102] Referring to Figure 4 , the performance index processing device 400 may include a determination module 401, an extraction module 402, and an analysis module 403. Each module in the performance index processing device 400 may be implemented by one or more modules, and the name of the corresponding module may vary according to the type of the module. In various embodiments, some modules in the performance index processing device 400 may be omitted, or additional modules may also be included. In addition, the modules / components according to various embodiments of the present disclosure may be combined to form a single entity, and thus may equivalently execute the functions of the corresponding modules / components before combination.

[0103] When the target performance index of an object is abnormal at the current moment, the determination module 401 may determine candidate performance indexes that are abnormal at the current moment among other performance indexes of the object. The object may include an application, etc.

[0104] As an example, the determination module 401 may calculate the mean and standard deviation of the time series of a performance index within a predetermined time period at the current moment. If the value of the performance index at the current moment is included in the reference range formed by the mean and the standard deviation, it is determined that the performance index is not abnormal at the current moment; if the value of the performance index at the current moment exceeds the reference range formed by the mean and the standard deviation, it is determined that the performance index is abnormal at the current moment.

[0105] The determination module 401 may obtain the mean and standard deviation of the time series of a performance index within a predetermined time period at the current moment, and determine the reference range according to the mean and the standard deviation.

[0106] The extraction module 402 can extract key change points from the historical time series of the target performance metric and the candidate performance metric. The key change points can include at least one of the abnormal points of the metric data, the peak and valley points, and the data points where the difference in the front and back trend changes exceeds the threshold.

[0107] The extraction module 402 can respectively calculate the mean and standard deviation corresponding to the time series of the target performance metric and the candidate performance metric based on the historical time series of the performance metric, and determine the data points in the historical time series that exceed the reference range as key change points, where the reference range is formed by the mean and the standard deviation.

[0108] As an example, the extraction module 402 can respectively calculate the mean and standard deviation of the metric data of the target performance metric and the candidate performance metric based on the historical time series of the target performance metric and the candidate performance metric; respectively determine the data points in the time series of the target performance metric and the candidate performance metric that exceed the reference range formed by the corresponding mean and standard deviation as abnormal points, and then use the abnormal points as key change points.

[0109] In addition, the extraction module 402 can respectively calculate the difference sequences of the target performance metric and the candidate performance metric based on the historical time series of the performance metric, obtain the valley points and / or peak points in the difference sequences, and use the valley points and the peak points as the key change points.

[0110] As an example, calculate the difference sequences of the target performance metric and the candidate performance metric respectively based on the historical time series of the target performance metric and the candidate performance metric; for each data point in the difference sequence, if the difference data within a specific time period before this data point is continuously negative and the difference data within a specific time period after this data point is continuously positive, then determine this data point as a valley point, otherwise determine this data point as a peak point; use the valley points and the peak points as key change points.

[0111] In addition, the extraction module 402 can respectively calculate the difference sequences of the target performance metric and the candidate performance metric based on the historical time series of the performance metric, obtain the data points in the difference sequences where the difference in the front and back trend changes exceeds the threshold, and use the data points where the difference in the front and back trend changes exceeds the threshold as key change points.

[0112] As an example, the extraction module 402 may calculate the difference sequences of the target performance metric and the candidate performance metrics respectively based on the historical time series of the target performance metric and the candidate performance metrics; for each data point in the difference sequences, calculate the first average difference within a specific time period before the data point and the second average difference within a specific time period after the data point based on the difference sequences; if the ratio between the first average difference and the second average difference is greater than a threshold, determine the data point as a data point with a difference in the front and back trend changes exceeding the threshold; and use the data points with the difference in the front and back trend changes exceeding the threshold as key change points.

[0113] The analysis module 403 may determine the correlation between the target performance metric and each candidate performance metric based on the extracted key change points; and determine the relevant performance metric associated with the target performance metric from the candidate performance metrics according to the correlation result. In the present disclosure, based on the correlation result, a relevant performance metric that is unidirectionally correlated with the target performance metric, a relevant performance metric that is bidirectionally correlated with the target performance metric, and a performance metric that is bidirectionally unrelated to the target performance metric can be determined.

[0114] As an example, the analysis module 403 may determine the corresponding moment of the key change point in the historical time series; according to the corresponding moment, determine the number of key change points of the target performance metric and each candidate performance metric having the same moment in the historical time series; and determine the correlation between the target performance metric and each candidate performance metric based on the number of key change points having the same moment.

[0115] For example, the analysis module 403 may calculate a first correlation result for the target performance metric according to the number of key change points having the same moment and the number of key change points of the target performance metric; calculate second correlation results for each candidate performance metric respectively according to the number of key change points having the same moment and the number of key change points of each candidate performance metric; and determine the correlation between the target performance metric and each candidate performance metric by comparing the first correlation result and the second correlation results with a predetermined value respectively. Here, the predetermined value can be set differently according to actual needs.

[0116] The above has Figure 2 and Figure 3 detailedly described the association analysis operation for the target performance metric, and will not be described here again.

[0117] Figure 5 is a schematic structural diagram of a performance metric processing device in the hardware operating environment of an embodiment of the present disclosure.

[0118] As Figure 5As shown, the performance metric processing device 500 may include: a processing component 501, a communication bus 502, a network interface 503, an input / output interface 504, a memory 505, and a power supply component 506. Among them, the communication bus 502 is used to enable connection communication between these components. The input / output interface 504 may include a video display (such as a liquid crystal display), a microphone and a speaker, and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.). Optionally, the input / output interface 504 may further include a standard wired interface and a wireless interface. The network interface 503 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 505 may be a high-speed random access memory or a stable non-volatile memory. Optionally, the memory 505 may also be a storage device independent of the aforementioned processing component 501.

[0119] Those skilled in the art can understand that Figure 5 the structure shown in does not constitute a limitation on the performance metric processing device 500, and it may include more or fewer components than shown, or combine certain components, or have a different component arrangement.

[0120] As Figure 5 shown, the memory 505, as a storage medium, may include an operating system (such as a MAC operating system), a data storage module, a network communication module, a user interface module, a performance metric processing program, and a database.

[0121] In Figure 5 the performance metric processing device 500 shown, the network interface 503 is mainly used for data communication with external electronic devices / terminals; the input / output interface 504 is mainly used for data interaction with users; the processing component 501 and the memory 505 in the performance metric processing device 500 may be arranged in the performance metric processing device 500. The performance metric processing device 500 calls the performance metric processing program stored in the memory 505 and various APIs provided by the operating system through the processing component 501 to execute the performance metric processing method provided by the embodiments of the present disclosure.

[0122] The processing component 501 may include at least one processor. A set of computer-executable instructions is stored in the memory 505. When the set of computer-executable instructions is executed by at least one processor, the performance metric processing method according to the embodiments of the present disclosure is executed. However, the above examples are only exemplary, and the present disclosure is not limited thereto.

[0123] The processing component 501 can control the components included in the performance metric processing device 500 by executing a program.

[0124] The performance metric processing device 500 can receive or output a picture or audio via the input / output interface 504.

[0125] As an example, the performance metric processing device 500 can be a PC computer, a tablet device, a personal digital assistant, a smart phone, or other devices capable of executing the above instruction set. Here, the performance metric processing device 500 does not have to be a single electronic device, and can also be a collection of devices or circuits capable of executing the above instructions (or instruction sets) individually or jointly. The performance metric processing device 500 can also be a part of an integrated control system or a system manager, or can be configured to execute a portable electronic device that is interconnected locally or remotely (e.g., via wireless transmission).

[0126] In the performance metric processing device 500, the processing component 501 can include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. As an example and not a limitation, the processing component 501 can also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0127] The processing component 501 can run instructions or code stored in the memory, where the memory 505 can also store data. The instructions and data can also be sent and received via the network interface 503 over the network, where the network interface 503 can use any known transmission protocol.

[0128] The memory 505 can be integrated with the processing component 501. For example, RAM or flash memory can be arranged within an integrated circuit microprocessor, etc. In addition, the memory 505 can include a separate device, such as an external disk drive, a storage array, or other storage devices that can be used by any database system. The memory and the processing component 501 can be operatively coupled, or can communicate with each other, for example, through an I / O port, a network connection, etc., such that the processing component 501 can read the data stored in the memory 505.

[0129] According to an embodiment of the present disclosure, an electronic device can be provided. Figure 6 is a block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device 600 can include at least one memory 602 and at least one processor 601. The at least one memory 602 stores a set of computer-executable instructions. When the set of computer-executable instructions is executed by the at least one processor 601, a performance metric processing method according to an embodiment of the present disclosure is executed.

[0130] The processor 601 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, the processor 601 may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, and the like.

[0131] The memory 602, as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, a performance metric handler, and a database.

[0132] The memory 602 may be integrated with the processor 601. For example, RAM or flash memory may be disposed within an integrated circuit microprocessor and the like. In addition, the memory 602 may include a stand-alone device, such as an external disk drive, a storage array, or other storage devices that may be used by any database system. The memory 602 and the processor 601 may be operatively coupled or may communicate with each other, for example, via an I / O port, a network connection, etc., such that the processor 601 can read files stored in the memory 602.

[0133] In addition, the electronic device 600 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.). All components of the electronic device 600 may be connected to each other via a bus and / or a network.

[0134] Those skilled in the art will understand that Figure 6 the structure shown in does not constitute a limitation on, and may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0135] According to an embodiment of the present disclosure, a computer-readable storage medium storing instructions may also be provided, wherein when the instructions are run by at least one processor, the at least one processor is caused to execute the performance index processing method according to the present disclosure. Examples of such computer-readable storage media include: read-only memory (ROM), programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc memory, hard disk drive (HDD), solid state drive (SSD), cartridge memory (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer such that the processor or computer can execute the computer program. The computer program in the above computer-readable storage medium may run in an environment deployed in computer devices such as clients, hosts, proxy devices, servers, etc. Additionally, in one example, the computer program and any associated data, data files, and data structures are distributed on a networked computer system such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner by one or more processors or computers.

[0136] In an embodiment according to the present disclosure, a computer program product may also be provided, and the instructions in the computer program product may be executed by a processor of a computer device to complete the above performance index processing method.

[0137] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0138] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A method for processing performance indicators, characterized in that, it includes: When the target performance indicator of an object is abnormal at the current moment, determining candidate performance indicators among other performance indicators of the object that are abnormal at the current moment; Extracting key change points from the historical time series of the target performance indicator and the historical time series of the candidate performance indicators; Determining the corresponding moments of the key change points in the historical time series; According to the corresponding moments, determining the number of key change points of the target performance indicator and each candidate performance indicator that have the same moment in the historical time series; According to the number of key change points with the same moment and the number of key change points of the target performance indicator, calculating a first correlation result for the target performance indicator; According to the number of key change points with the same moment and the number of key change points of each candidate performance indicator, respectively calculating a second correlation result for each candidate performance indicator; By comparing the first correlation result and the second correlation results with a predetermined value respectively, determining the correlation between the target performance indicator and each candidate performance indicator; Determining the relevant performance indicators associated with the target performance indicator from the candidate performance indicators according to the determined correlation.

2. The performance indicator processing method according to claim 1, characterized in that, determining that a performance indicator is abnormal at the current moment based on the following method: Based on the time series of a performance indicator within a predetermined time period at the current moment, determining a reference range; If the value of the performance indicator at the current moment exceeds the reference range, determining that the performance indicator is abnormal at the current moment.

3. The performance indicator processing method according to claim 2, characterized in that, the determining the reference range based on the indicator data of a performance indicator within a predetermined time period at the current moment includes: Based on the time series of the performance indicator within a predetermined time period at the current moment, obtaining the mean and standard deviation of the time series; According to the mean and the standard deviation, determining the reference range.

4. The performance indicator processing method according to claim 1, characterized in that, the extracting key change points from the historical time series of the target performance indicator and the historical time series of the candidate performance indicators includes: Based on the historical time series, respectively calculating the mean and standard deviation corresponding to the time series of the target performance indicator and the candidate performance indicators; Determining the data points in the historical time series that exceed the reference range as key change points, where the reference range is formed by the mean and the standard deviation.

5. The performance indicator processing method according to claim 1, characterized in that, the extracting key change points from the historical time series of the target performance indicator and the historical time series of the candidate performance indicators includes: Based on the historical time series, respectively calculating the difference sequences of the target performance indicator and the candidate performance indicators; Obtaining the trough points and / or peak points in the difference sequences, and taking the trough points and the peak points as the key change points.

6. The performance index processing method according to claim 1, wherein, extracting key change points from the historical time series of the target performance index and the historical time series of the candidate performance index includes: Based on the historical time series, calculating the difference sequences of the target performance index and the candidate performance index respectively; Obtaining the data points in the difference sequence whose front-back trend change difference exceeds a threshold; taking the data points whose front-back trend change difference exceeds the threshold as the key change points.

7. A performance index processing device, wherein, including: A determination module configured to execute determining candidate performance indexes that have anomalies at the current moment among other performance indexes of the object when the target performance index of the object has an anomaly at the current moment; An extraction module configured to execute extracting key change points from the historical time series of the target performance index and the historical time series of the candidate performance index; An analysis module configured to execute: Determining the corresponding moments of the key change points in the historical time series; According to the corresponding moments, determining the number of key change points of the target performance index and each candidate performance index that have the same moment in the historical time series; According to the number of key change points having the same moment and the number of key change points of the target performance index, calculating a first correlation result for the target performance index; According to the number of key change points having the same moment and the number of key change points of each candidate performance index, calculating a second correlation result for each candidate performance index respectively; By respectively comparing the first correlation result and the second correlation result with a predetermined value, determining the correlation between the target performance index and each candidate performance index; and Determining associated performance indexes associated with the target performance index from the candidate performance indexes according to the determined correlation.

8. The performance index processing device according to claim 7, wherein, The determination module is configured to execute: Based on the time series within a predetermined time period of the performance index at the current moment, determining a reference range; If the value of the performance index at the current moment exceeds the reference range, determining that the performance index has an anomaly at the current moment.

9. The performance index processing device according to claim 8, wherein, The determination module is configured to execute: Based on the time series of the performance index within a predetermined time period at the current moment, obtaining the mean and standard deviation of the time series; According to the mean and the standard deviation, determining the reference range.

10. The performance index processing device according to claim 7, wherein, The analysis module is configured to execute: Based on the historical time series, calculating the mean and standard deviation corresponding to the time series of the target performance index and the candidate performance index respectively; Determining the data points in the historical time series that exceed the reference range as key change points, wherein the reference range is formed by the mean and the standard deviation.

11. The performance index processing device according to claim 7, It is characterized in that the analysis module is configured to perform: based on the historical time series, respectively calculate the difference sequences of the target performance metric and the candidate performance metric; obtain the trough points and / or peak points in the difference sequences, and use the trough points and the peak points as the key change points.

12. The performance metric processing device according to claim 7, it is characterized in that the analysis module is configured to perform: based on the historical time series, respectively calculate the difference sequences of the target performance metric and the candidate performance metric; obtain the data points in the difference sequence whose difference in the front and back trend changes exceeds a threshold; use the data points whose difference in the front and back trend changes exceeds the threshold as the key change points.

13. An electronic device, it is characterized in that comprising: at least one processor; at least one memory storing computer-executable instructions, wherein, when the computer-executable instructions are run by the at least one processor, the at least one processor is caused to execute the performance metric processing method according to any one of claims 1 to 6.

14. A computer-readable storage medium storing instructions, it is characterized in that when the instructions are run by at least one processor, the at least one processor is caused to execute the performance metric processing method according to any one of claims 1 to 6.

15. A computer program product, wherein the instructions in the computer program product are run by at least one processor in an electronic device to execute the performance metric processing method according to any one of claims 1 to 6.

Images