Index abnormal interval determination method and device and computer equipment

Abstract

The invention relates to an index exception interval determination method and device and computer equipment. The method comprises the following steps: determining a to-be-predicted index, and predicting to obtain a data baseline of the to-be-predicted index in a to-be-predicted time period; under the condition that the quantity of the historical data is smaller than a preset quantity threshold value, determining a reference index from the indexes, and determining a data distribution form of each reference index according to reference historical data of the to-be-predicted time period corresponding to each reference index; determining an abnormal data interval corresponding to abnormal historical data in the historical data, respectively determining a first distribution probability of each data distribution form corresponding to the abnormal data interval, and determining a target data distribution form from each data distribution form according to the first distribution probability; and based on the data baseline, the target data distribution pattern and a preset abnormal data probability, determining an index abnormal interval corresponding to the to-be-predicted index in the to-be-predicted time period. By adopting the method, the determination precision of the index abnormal interval can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, and computer device for determining an abnormal interval of an indicator. Background Art

[0002] Threshold setting is crucial in fields like data analysis and operations monitoring. A threshold can be defined as the boundary of a measurement or calculation result. When the actual value exceeds or falls below this threshold, it is considered to fall into the abnormal range of the indicator. The system or application will trigger a response or alert, notifying operations personnel of the situation that requires attention.

[0003] Related technologies often use dynamic threshold algorithms to adjust the abnormality interval for an indicator based on its fluctuations when setting the threshold for a specific indicator. However, these algorithms typically assume that the data underlying the indicator follows a normal distribution. This can lead to inaccurate thresholds and abnormality intervals determined by dynamic threshold algorithms when the actual distribution of the data under the indicator is not normally distributed, resulting in high rates of missed alerts and false positives. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device and computer equipment for determining the abnormal interval of indicators to address the above technical problems.

[0005] In a first aspect, the present application provides a method for determining an abnormal interval of an indicator. The method comprises:

[0006] Determine an indicator to be predicted from each indicator, and predict a data baseline of the indicator to be predicted in the period to be predicted based on historical data of the indicator to be predicted corresponding to the period to be predicted;

[0007] When the amount of the historical data is less than a preset threshold, a reference indicator is determined from the indicators, and a data distribution form of each reference indicator is determined based on the reference historical data of each reference indicator corresponding to the time period to be predicted; the reference indicator is an indicator that has a positive correlation with the indicator to be predicted and the amount of the reference historical data is greater than or equal to the preset threshold;

[0008] Determine an abnormal data interval corresponding to abnormal historical data in the historical data, respectively determine a first distribution probability of each of the data distribution forms corresponding to the abnormal data interval, and determine a target data distribution form from each of the data distribution forms according to the first distribution probability;

[0009] Based on the data baseline, the target data distribution form and the preset abnormal data probability, the indicator abnormal interval corresponding to the indicator to be predicted within the time period to be predicted is determined.

[0010] In one embodiment, the method further comprises:

[0011] When the amount of the historical data is greater than or equal to a preset amount threshold, the target data distribution form is determined based on the historical data.

[0012] In one embodiment, determining the abnormal data interval corresponding to the abnormal historical data in the historical data includes:

[0013] determining normal historical data and abnormal historical data from the historical data;

[0014] determining a data interval length based on a difference between a maximum value and a minimum value in the normal historical data and the amount of the normal historical data, and dividing the normal historical data into a plurality of normal data intervals based on the data interval length;

[0015] According to the data interval length, data intervals are further divided on both sides of each normal data interval until all the historical data have corresponding data intervals, and the data intervals with abnormal historical data are regarded as abnormal data intervals.

[0016] In one embodiment, determining the first distribution probability of each data distribution form corresponding to the abnormal data interval includes:

[0017] Constructing probability density functions corresponding to the data distribution forms according to normal historical data in the historical data;

[0018] Integrating each of the probability density functions over the abnormal data intervals respectively, to obtain a first distribution probability of each of the data distribution forms corresponding to the abnormal data intervals.

[0019] In one embodiment, determining a target data distribution form from each of the data distribution forms according to the first distribution probability includes:

[0020] The data distribution form with the minimum first distribution probability is used as the target data distribution form.

[0021] In one embodiment, determining a target data distribution form from each of the data distribution forms according to the first distribution probability includes:

[0022] Determining a normal data interval corresponding to normal historical data in the historical data;

[0023] respectively determining a second distribution probability of each of the data distribution forms corresponding to the normal data interval;

[0024] Determining a detection accuracy score corresponding to each data distribution form based on the first distribution probability and the second distribution probability of each data distribution form, wherein the detection accuracy score is negatively correlated with the first distribution probability and positively correlated with the second distribution probability;

[0025] The data distribution form with the highest detection accuracy score is used as the target data distribution form.

[0026] In one embodiment, determining the abnormal index interval corresponding to the indicator to be predicted within the period to be predicted based on the data baseline, the target data distribution pattern, and the preset abnormal data probability includes:

[0027] Taking the data baseline as the data mean or data median of the indicator to be predicted within the time period to be predicted, and constructing a target probability density function of the target data distribution corresponding to the data baseline based on the data mean or the data median;

[0028] An indicator abnormality interval is determined based on the target probability density function, and an integral value of the target probability density function in the indicator abnormality interval is equal to the preset abnormal data probability.

[0029] In one embodiment, determining the indicator abnormality interval based on the target probability density function includes:

[0030] Determine the estimated distribution range of abnormal data corresponding to the target probability density function, and determine the target abnormal data probability corresponding to each of the estimated distribution ranges of abnormal data according to the preset abnormal data probability, wherein the sum of the target abnormal data probabilities is the preset abnormal data probability;

[0031] Based on the probability of each target abnormal data, determining the cumulative probability lower boundary value and the cumulative probability upper boundary value corresponding to the estimated distribution range of each abnormal data;

[0032] For any of the abnormal data estimated distribution ranges, determining the indicator abnormal interval in the abnormal data estimated distribution range according to the cumulative probability lower boundary value and the cumulative probability upper boundary value;

[0033] Among them, the integral result of the target probability density function from the lower boundary value to the lower boundary value of the indicator abnormal interval is the cumulative probability lower boundary value, and the integral result of the target probability density function from the lower boundary value to the upper boundary value of the indicator abnormal interval is the cumulative probability upper boundary value.

[0034] In a second aspect, the present application also provides a device for determining an abnormal interval of an indicator. The device comprises:

[0035] A prediction module is used to determine an indicator to be predicted from various indicators, and predict a data baseline of the indicator to be predicted in the period to be predicted based on historical data of the indicator to be predicted corresponding to the period to be predicted;

[0036] a first determining module configured to, when the amount of the historical data is less than a preset threshold, determine a reference indicator from the indicators, and determine a data distribution form of each reference indicator based on the reference historical data corresponding to the time period to be predicted for each reference indicator; the reference indicator is an indicator that has a positive correlation with the indicator to be predicted and for which the amount of the reference historical data is greater than or equal to the preset threshold;

[0037] a second determining module, configured to determine an abnormal data interval corresponding to abnormal historical data in the historical data, respectively determine a first distribution probability of each of the data distribution forms corresponding to the abnormal data interval, and determine a target data distribution form from each of the data distribution forms according to the first distribution probability;

[0038] The third determination module is used to determine the indicator abnormality interval corresponding to the indicator to be predicted within the time period to be predicted based on the data baseline, the target data distribution form and the preset abnormal data probability.

[0039] In one embodiment, the first determining module is further configured to:

[0040] When the amount of the historical data is greater than or equal to a preset amount threshold, the target data distribution form is determined based on the historical data.

[0041] In one embodiment, the second determining module is further configured to:

[0042] determining normal historical data and abnormal historical data from the historical data;

[0043] determining a data interval length based on a difference between a maximum value and a minimum value in the normal historical data and the amount of the normal historical data, and dividing the normal historical data into a plurality of normal data intervals based on the data interval length;

[0044] According to the data interval length, data intervals are further divided on both sides of each normal data interval until all the historical data have corresponding data intervals, and the data intervals with abnormal historical data are regarded as abnormal data intervals.

[0045] In one embodiment, the second determining module is further configured to:

[0046] Constructing probability density functions corresponding to the data distribution forms according to normal historical data in the historical data;

[0047] Integrating each of the probability density functions over the abnormal data intervals respectively, to obtain a first distribution probability of each of the data distribution forms corresponding to the abnormal data intervals.

[0048] In one embodiment, the second determining module is further configured to:

[0049] The data distribution form with the minimum first distribution probability is used as the target data distribution form.

[0050] In one embodiment, the second determining module is further configured to:

[0051] Determining a normal data interval corresponding to normal historical data in the historical data;

[0052] respectively determining a second distribution probability of each of the data distribution forms corresponding to the normal data interval;

[0053] Determining a detection accuracy score corresponding to each data distribution form based on the first distribution probability and the second distribution probability of each data distribution form, wherein the detection accuracy score is negatively correlated with the first distribution probability and positively correlated with the second distribution probability;

[0054] The data distribution form with the highest detection accuracy score is used as the target data distribution form.

[0055] In one embodiment, the third determining module is further configured to:

[0056] Taking the data baseline as the data mean or data median of the indicator to be predicted within the time period to be predicted, and constructing a target probability density function of the target data distribution corresponding to the data baseline based on the data mean or the data median;

[0057] An indicator abnormality interval is determined based on the target probability density function, and an integral value of the target probability density function in the indicator abnormality interval is equal to the preset abnormal data probability.

[0058] In one embodiment, the third determining module is further configured to:

[0059] Determine the estimated distribution range of abnormal data corresponding to the target probability density function, and determine the target abnormal data probability corresponding to each of the estimated distribution ranges of abnormal data according to the preset abnormal data probability, wherein the sum of the target abnormal data probabilities is the preset abnormal data probability;

[0060] Based on the probability of each target abnormal data, determining the cumulative probability lower boundary value and the cumulative probability upper boundary value corresponding to the estimated distribution range of each abnormal data;

[0061] For any of the abnormal data estimated distribution ranges, determining the indicator abnormal interval in the abnormal data estimated distribution range according to the cumulative probability lower boundary value and the cumulative probability upper boundary value;

[0062] Among them, the integral result of the target probability density function from the lower boundary value to the lower boundary value of the indicator abnormal interval is the cumulative probability lower boundary value, and the integral result of the target probability density function from the lower boundary value to the upper boundary value of the indicator abnormal interval is the cumulative probability upper boundary value.

[0063] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements any of the above methods when executing the computer program.

[0064] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements any of the above methods when executed by a processor.

[0065] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which implements any of the above methods when executed by a processor.

[0066] The above-mentioned method, device and computer equipment for determining abnormal intervals of indicators first predict the data baseline based on historical data, and then, for the indicator to be predicted that has less historical data and cannot accurately determine the data distribution form under the indicator, determine the data distribution form that the data under the indicator may obey based on a reference indicator that may have a positive correlation with the indicator and has more historical data, and then determine the target data distribution form based on the distribution probability of the abnormal data interval where the abnormal historical data in the historical data corresponding to these data distribution forms are located, so that the indicator abnormal interval constructed according to the target data distribution form can detect abnormal values. Then, based on the data baseline, the target data distribution form and the preset abnormal data probability, the indicator abnormal interval corresponding to the indicator to be predicted in the period to be predicted can be constructed. The embodiment of the present application can dynamically adjust the data baseline based on the historical data of the indicator to be predicted, and then dynamically adjust the indicator abnormal interval based on the data baseline. And when the historical data of the indicator to be predicted is insufficient, the historical data of other indicators can also be used to determine the possible location of the abnormal value under the indicator to be predicted, thereby improving the determination accuracy of the indicator abnormal interval. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1Schematic diagram of a flow chart of a method for determining an abnormal indicator interval in one embodiment;

[0068] Figure 2 A schematic diagram of a process for dividing abnormal data intervals in one embodiment;

[0069] Figure 3 Schematic diagram of a process for determining a first distribution probability in one embodiment;

[0070] Figure 4 A schematic diagram of a process for determining target data distribution in one embodiment;

[0071] Figure 5 A schematic diagram of a process for determining an abnormal interval of an indicator in one embodiment;

[0072] Figure 6 A schematic diagram of a flow chart for determining an abnormal interval of an indicator in another embodiment;

[0073] Figure 7 This is a structural block diagram of a device for determining an abnormal indicator interval in one embodiment;

[0074] Figure 8 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0076] In one embodiment, Figure 1 As shown, a method for determining an abnormal interval of an indicator is provided. This embodiment uses the method applied to a server as an example for illustration. It is understandable that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0077] Step 102 : determining an indicator to be predicted from various indicators, and predicting a data baseline of the indicator to be predicted in the period to be predicted based on historical data of the indicator to be predicted corresponding to the period to be predicted.

[0078] In the embodiment of the present application, the indicator is a data category that needs to be monitored, such as website visits, average response time, number of orders, total transaction amount, etc. The period to be predicted is the period during which the abnormal interval of the indicator needs to be predicted. The specific length of the period to be predicted can be determined according to the fluctuation pattern of the indicator to be predicted. For example, for an indicator such as the number of orders that fluctuates more violently within a day and is easily affected by emergencies, the period to be predicted can be set to be shorter, such as 1 hour, 3 hours, etc., so that the abnormal interval of the indicator can be predicted again based on the latest data at short intervals, thereby improving the prediction accuracy of the abnormal interval of the indicator. For an indicator such as the average daily customer unit price that generally does not change violently in a short period of time, the period to be predicted can be set to be longer, such as 1 week, 1 month, etc., to save the server computing power overhead caused by the prediction.

[0079] The historical data corresponding to the time period to be predicted refers to the data generated before the indicator, which has a certain correlation and similarity with the data in the time period to be predicted, such as the data one hour before the time period to be predicted, the data corresponding to the same time period of the time period to be predicted in the previous week, and so on. The data baseline is a value that changes with time. The value corresponding to the data baseline at a certain moment represents the standard value of the data at that moment under ideal conditions. Since the data under the indicator can generally be regarded as a kind of data with time series characteristics, after obtaining the historical data, the data baseline of the indicator to be predicted in the time period to be predicted can be obtained based on any time series model. The embodiment of the present application does not specifically limit how to predict the data baseline.

[0080] Step 104, when the amount of historical data is less than the preset threshold, determine the reference indicator from each indicator, and determine the data distribution form of each reference indicator based on the reference historical data of the time period to be predicted corresponding to each reference indicator; the reference indicator is an indicator that has a positive correlation with the indicator to be predicted and the amount of reference historical data is greater than or equal to the preset threshold.

[0081] In the embodiment of the present application, the preset quantity threshold is the minimum number of historical data required when predicting abnormal intervals of indicators based on historical data. Those skilled in the art can determine the preset quantity threshold according to actual needs, for example, it can be set to 1,000, 10,000, etc.

[0082] When the amount of historical data is greater than or equal to a preset threshold, a target data distribution pattern can be determined based on the historical data. The target data distribution pattern refers to the distribution pattern that the data within the time period to be predicted may follow. Because the variation patterns of the historical data corresponding to the time period to be predicted are generally similar to the variation patterns of the data within the time period to be predicted, the data distribution pattern corresponding to the historical data can be determined based on the historical data, and then the data distribution pattern corresponding to the historical data can be used as the target data distribution pattern for the time period to be predicted.

[0083] When the amount of historical data is small, the data distribution corresponding to the historical data calculated from the historical data may not be accurate enough, resulting in an inaccurate target data distribution, which further leads to insufficient accuracy in determining the abnormal interval of the indicator. In this case, a data distribution can be determined as the target data distribution, even if it does not completely match the data distribution of the indicator to be predicted, but can best describe the distribution pattern of normal and abnormal data under the indicator to be predicted. To save time in finding such a data distribution, the target data distribution can be selected from the data distribution patterns corresponding to reference indicators that have a certain similarity to the data distribution pattern of the indicator to be predicted.

[0084] A reference indicator is an indicator that has a positive correlation with the indicator to be predicted and for which the amount of historical data corresponding to the forecast period (hereinafter referred to as reference historical data) is greater than or equal to a preset threshold. Whether an indicator has a positive correlation with the indicator to be predicted can be determined by those skilled in the art based on the historical data of both indicators. After initially identifying reference indicators that have a positive correlation with the indicator to be predicted, if there are a large number of reference indicators, further screening of the reference indicators can be performed based on the strength of the positive correlation and parameters that describe the data distribution, such as the variance and skewness of the historical data. For example, indicators with a linear positive correlation and a variance similar to the variance of the historical data of the indicator to be predicted can be preferentially selected as reference indicators.

[0085] After determining the reference indicators, you can determine the corresponding data distribution patterns for each reference indicator based on their historical data. These data distribution patterns can be considered as the likely data distribution patterns for the indicator to be predicted. If fewer reference indicators are identified, you can also add some common data distribution patterns (such as Gaussian, Gamma, and Exponential) in addition to the data distribution patterns for each reference indicator.

[0086] Step 106 , determining the abnormal data interval corresponding to the abnormal historical data in the historical data, determining the first distribution probability of each data distribution form corresponding to the abnormal data interval, and determining the target data distribution form from each data distribution form according to the first distribution probability.

[0087] In the embodiments of this application, historical data is divided into normal historical data and abnormal historical data. Abnormal historical data refers to historical data that is determined to be an outlier, and normal historical data refers to historical data other than abnormal historical data. To improve the accuracy of predicting abnormal intervals of indicators, only historical data manually confirmed as an outlier by operations and maintenance personnel can be used as abnormal historical data.

[0088] Abnormal data interval refers to the data interval where abnormal historical data is located in the historical data. The embodiment of the present application does not limit how to divide the abnormal data interval. Multiple consecutive abnormal historical data without normal historical data in the middle can be divided into an abnormal data interval. For example, when the historical data is the average response time, and the historical data includes 0ms, 5ms, 300ms, 1000ms, and 20000ms, assuming that 0ms, 5ms, and 20000ms are abnormal historical data, then 0ms and 5ms belong to "multiple consecutive abnormal historical data without normal historical data in the middle". An abnormal data interval can be divided based on these two abnormal historical data: 0ms to 5ms. Another abnormal data interval can be divided for 20000ms: 20000ms. Or, considering that there is less historical data and even less abnormal historical data, there may be a smaller risk of determining the abnormal data interval based on this method of dividing the abnormal data interval, or it can be divided as follows: Figure 2 The method shown divides the abnormal data interval:

[0089] Step 202, determining normal historical data and abnormal historical data from the historical data;

[0090] Step 204: determining a data interval length based on a difference between a maximum value and a minimum value in the normal historical data and the amount of normal historical data, and dividing the normal historical data into a plurality of normal data intervals based on the data interval length;

[0091] Step 206 : Continue dividing data intervals on both sides of each normal data interval according to the length of the data interval until all historical data have corresponding data intervals, and treat the data intervals with abnormal historical data as abnormal data intervals.

[0092] In the embodiment of the present application, a reasonable data interval length that can reflect the distribution pattern of historical data is first determined based on the amount and distribution of normal historical data. Then, after each normal historical data is divided into data intervals according to the data interval length, data intervals are further divided on both sides of the data interval of each normal historical data until all historical data have a corresponding data interval. The data interval in which abnormal historical data exists can then be used as an abnormal data interval. This method is equivalent to treating all values ​​contained in the abnormal data interval as abnormal historical data, thereby expanding the amount of abnormal historical data.

[0093] First, the number of normal data intervals that need to be divided for the normal historical data can be determined based on the amount of normal historical data. The number of normal data intervals can be determined to the extent possible so that, after the normal data intervals are divided according to the number of normal data intervals, the amount of data in each normal data interval can still reflect the data distribution pattern of the normal historical data. The present application does not limit how to determine this number; for example, the square root of the amount of normal historical data can be taken and then rounded up, with the rounded value being used as the number of normal data intervals.

[0094] The data interval length can be calculated based on the ratio of the difference between the maximum and minimum values ​​in the normal historical data to the number of normal data intervals. After dividing the normal historical data into multiple normal data intervals based on the data interval length, the data intervals are further divided on both sides based on the data interval length to obtain data intervals that cover the entire data range of the historical data. Data intervals containing abnormal historical data within these data intervals are abnormal data intervals. This division process can avoid the situation where both normal historical data and abnormal historical data exist in the same data interval, making it difficult to determine whether the data interval is a normal data interval or an abnormal data interval.

[0095] Taking the example in which the above historical data includes 0ms, 5ms, 300ms, 1000ms, and 20000ms, the normal historical data in this example is 300ms and 1000ms. Assuming that the number of normal data intervals is 2, the data interval length is 350ms. The normal data intervals divided for normal historical data are 300ms-650ms and 650ms-1000ms respectively.

[0096] The data intervals are further divided into two parts. The left-hand data interval is -50ms-300ms. (Since negative average response times can occur in the event of system errors, and these erroneous average response times should also be considered outliers, the -50ms boundary value can be retained. If the boundary value exceeds the minimum or maximum possible value of the data under the indicator, the boundary value of the data interval can be modified to the minimum or maximum possible value of the data.) The right-hand data intervals are 1000ms-1350ms, 1350ms-1700ms, and so on, to 19900ms-20250ms. All historical data now has corresponding data intervals. 0ms and 5ms fall within the -50ms-300ms range, and 20000ms falls within the 19900ms-20250ms range. Therefore, -50ms-300ms and 19900ms-20250ms are outlier data intervals.

[0097] After determining the abnormal data interval, we further determine the first distribution probability of each data distribution form corresponding to the abnormal data interval. The first distribution probability is the probability that the data falls into the abnormal data interval under each data distribution form. This probability can be obtained by constructing the probability density function of the data distribution form corresponding to the historical data and integrating the probability density function over the abnormal data interval. For details, see Figure 3 As shown:

[0098] Step 302: constructing probability density functions corresponding to the distribution forms of each data based on the normal historical data in the historical data;

[0099] In step 304 , each probability density function is integrated over the abnormal data interval to obtain a first distribution probability of each data distribution form corresponding to the abnormal data interval.

[0100] In the embodiments of the present application, in order to prevent abnormal historical data from affecting the form of the probability density function, a probability density function can be constructed based only on normal historical data. Based on a certain data distribution form and multiple data that conform to the data distribution form, the method for calculating the probability density function corresponding to these data can use any existing method, which will not be repeated in the embodiments of the present application.

[0101] After obtaining the probability density function corresponding to each data distribution form, for each probability density function, the step of integrating the probability density function over the abnormal data interval is performed to obtain the probability that the data falls into the abnormal data interval under the data distribution form, that is, the first distribution probability. It should be noted that if there are multiple abnormal data intervals, it is necessary to integrate the probability density function over each abnormal data interval separately, and then add the integration results of different abnormal data intervals to obtain the first distribution probability.

[0102] After obtaining the first distribution probability, we can determine the target data distribution form that is most likely to match the data distribution form of the indicator to be predicted from the various data distribution forms. Two different strategies can be adopted based on actual needs: using the data distribution form with the lowest first distribution probability as the target data distribution form, or further calculating the second distribution probability corresponding to the normal data interval and combining the first and second distribution probabilities to determine the target data distribution form.

[0103] When the data distribution form with the lowest first distribution probability is used as the target data distribution form, it indicates that the true outliers are located at a relatively extreme position within the target data distribution form. Therefore, using the indicator anomaly interval calculated based on this target data distribution form can maximize the probability of true outliers being judged as outliers and minimize the false alarm rate. Determining the target data distribution form based on the second distribution probability and the first distribution probability can balance the false alarm rate and false alarm rate.

[0104] The process of determining the target data distribution form based on the second distribution probability and the first distribution probability can be found in Figure 4 Shown, including:

[0105] Step 402, determining a normal data interval corresponding to normal historical data in the historical data;

[0106] Step 404, determining the second distribution probability of each data distribution form corresponding to the normal data interval;

[0107] Step 406: Determine a detection accuracy score corresponding to each data distribution form based on the first distribution probability and the second distribution probability of each data distribution form, wherein the detection accuracy score is negatively correlated with the first distribution probability and positively correlated with the second distribution probability;

[0108] Step 408: The data distribution form with the highest detection accuracy score is used as the target data distribution form.

[0109] In an embodiment of the present application, the probability density function of each data distribution form is integrated over the normal data interval to obtain a second distribution probability corresponding to each data distribution form. The detection accuracy score corresponding to the data distribution form can be calculated based on the second distribution probability and the first distribution probability. The detection accuracy score is positively correlated with the second distribution probability and negatively correlated with the first distribution probability. The higher the second distribution probability, the higher the probability that the true normal value in the data distribution form is judged as a normal value. The lower the first distribution probability, the higher the probability that the true abnormal value in the data distribution form is judged as an abnormal value. Therefore, the higher the detection accuracy score, the higher the accuracy of outlier detection based on the indicator abnormal interval constructed according to the data distribution form.

[0110] A function between the first distribution probability, the second distribution probability, and the detection accuracy score can be constructed, and the detection accuracy score can be calculated based on the function. The parameters of the function can be set by those skilled in the art according to actual needs, and the embodiments of the present application do not specifically limit this.

[0111] After obtaining the detection accuracy scores corresponding to the various data distribution forms, the data distribution form with the highest detection accuracy score may be used as the target data distribution form.

[0112] Step 108 : Based on the data baseline, the target data distribution pattern, and the preset abnormal data probability, determine the indicator abnormality interval corresponding to the indicator to be predicted within the period to be predicted.

[0113] In the present embodiment, the preset abnormal data probability is the estimated proportion of abnormal values ​​in all data under the target data distribution. This value can be set based on experience. After obtaining the target data distribution, the indicator abnormal interval is constructed according to the data baseline so that the probability of data falling within the indicator abnormal interval within the target data distribution is equal to the preset abnormal data probability.

[0114] The target probability density function of the indicator to be predicted can be constructed based on the data baseline, and then the abnormal interval of the indicator can be determined according to the calculus method, such as Figure 5 As shown:

[0115] Step 502: Taking the data baseline as the data mean or data median of the indicator to be predicted within the time period to be predicted, and constructing a target probability density function of the target data distribution corresponding to the data baseline based on the data mean or data median;

[0116] Step 504 : determining an indicator abnormality interval based on the target probability density function, wherein the integral value of the target probability density function in the indicator abnormality interval is equal to a preset abnormal data probability.

[0117] In the embodiment of the present application, the data baseline is regarded as the data mean or data median of the indicator to be predicted in the time period to be predicted, and then the target probability density function is constructed according to the relationship between the data mean or data median and the probability density function in the target data distribution form. It should be noted that if there are some parameters in the target data distribution form that cannot be derived from the data mean or data median (such as the standard deviation in the normal distribution), or there are parameters that can only be derived from the relationship between parameters based on the data mean or data median, but cannot be derived from specific numerical values ​​(such as the shape parameter and scale parameter in the gamma distribution), the values ​​of these parameters can be determined based on the historical data of the indicator to be predicted.

[0118] Based on the target probability density function, an indicator abnormal interval can be determined on which the integral value of the target probability density function is equal to the preset abnormal data probability. For some common data distribution forms, the indicator abnormal interval can be determined by looking up the table, for example, for normal distribution, when the preset abnormal data probability is 5%, the indicator abnormal interval can be obtained by the mean ± 1.96 times the variance. For data distribution forms that cannot determine the indicator abnormal interval by looking up the table, Figure 6 The method shown determines the indicator abnormality interval:

[0119] Step 602: Determine the estimated distribution range of abnormal data corresponding to the target probability density function, and determine the target abnormal data probability corresponding to each abnormal data estimated distribution range based on the preset abnormal data probability. The sum of the target abnormal data probabilities is the preset abnormal data probability.

[0120] Step 604: Determine the cumulative probability lower boundary value and the cumulative probability upper boundary value corresponding to the estimated distribution range of each abnormal data based on the probability of each target abnormal data;

[0121] Step 606: for any abnormal data estimated distribution range, determine the indicator abnormal interval in the abnormal data estimated distribution range according to the cumulative probability lower boundary value and the cumulative probability upper boundary value;

[0122] Among them, the integral result of the target probability density function from the lower boundary value to the lower boundary value of the indicator abnormal interval is the cumulative probability lower boundary value, and the integral result of the target probability density function from the lower boundary value to the upper boundary value of the indicator abnormal interval is the cumulative probability upper boundary value.

[0123] In an embodiment of the present application, the estimated distribution range of abnormal data corresponding to the target data distribution form can be determined, a probability density threshold can be preset, and the part of the probability density function whose corresponding value is less than the probability density threshold can be used as the estimated distribution range of abnormal data.

[0124] By allocating the preset abnormal data probability to each abnormal data estimated distribution range, the target abnormal data probability corresponding to each abnormal data estimated distribution range can be obtained. The ratio of the preset abnormal data probability and the abnormal data estimated distribution range can be used as the target abnormal data probability, or the target abnormal data probability can be determined based on the width of each abnormal data estimated distribution range. This embodiment of the present application does not specifically limit this.

[0125] Then the cumulative probability lower boundary value and the cumulative probability upper boundary value corresponding to the estimated distribution range of each abnormal data can be determined. Generally speaking, the abnormal value under an indicator is the data that is too small or too large under the indicator, that is, the estimated distribution range of abnormal data usually includes the range from the lower boundary of the probability density function domain to a certain point, and the range from a certain point to the upper boundary of the probability density function domain. For the range from the lower boundary of the probability density function domain to a certain point, the cumulative probability lower boundary value of the range can be set to 0, and the cumulative probability upper boundary value can be set to the sum of 0 and the target abnormal data probability of the range. For the range from a certain point to the upper boundary of the probability density function domain, the cumulative probability lower boundary value of the range can be set to the difference between 100 and the target abnormal data probability of the range, and the cumulative probability upper boundary value of the range can be set to 100.

[0126] Then, according to the integration method, we can find the indicator abnormality interval that satisfies the requirement that the integral result of the target probability density function from the lower boundary value of its definition domain to the lower boundary value of the indicator abnormality interval is the cumulative probability lower boundary value, and the integral result from the lower boundary value of its definition domain to the upper boundary value of the indicator abnormality interval is the cumulative probability upper boundary value.

[0127] The method for determining an abnormal interval of an indicator provided in an embodiment of the present application first predicts a data baseline based on historical data, and then, for an indicator to be predicted that has less historical data and cannot accurately determine the data distribution form under the indicator, determines the data distribution form that the data under the indicator may obey based on a reference indicator that may have a positive correlation with the indicator and has more historical data, and then determines the target data distribution form based on the distribution probability of the abnormal data interval where the abnormal historical data in the historical data corresponding to these data distribution forms are located, so that the indicator abnormal interval constructed according to the target data distribution form can detect abnormal values. Then, based on the data baseline, the target data distribution form and the preset abnormal data probability, the indicator abnormal interval corresponding to the indicator to be predicted in the period to be predicted can be constructed. The embodiment of the present application can dynamically adjust the data baseline based on the historical data of the indicator to be predicted, and then dynamically adjust the indicator abnormal interval based on the data baseline. And when the historical data of the indicator to be predicted is insufficient, the historical data of other indicators can also be used to determine the possible location of the abnormal value under the indicator to be predicted, thereby improving the determination accuracy of the indicator abnormal interval.

[0128] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0129] Based on the same inventive concept, embodiments of the present application also provide an apparatus for determining an abnormal indicator interval for implementing the aforementioned method for determining an abnormal indicator interval. The solution provided by this apparatus is similar to the solution described in the aforementioned method. Therefore, the specific limitations in the embodiments of one or more apparatuses for determining an abnormal indicator interval provided below can be found in the limitations of the method for determining an abnormal indicator interval above and will not be further elaborated here.

[0130] In one embodiment, Figure 7 As shown, a device 700 for determining an abnormal indicator interval is provided, comprising: a prediction module 702, a first determination module 704, a second determination module 706, and a third determination module 708, wherein:

[0131] A prediction module 702 is configured to determine an indicator to be predicted from various indicators, and to predict a data baseline of the indicator to be predicted within the period to be predicted based on historical data of the indicator to be predicted corresponding to the period to be predicted;

[0132] A first determining module 704 is configured to, when the amount of the historical data is less than a preset threshold, determine a reference indicator from the indicators, and determine a data distribution form of each reference indicator based on the reference historical data corresponding to the time period to be predicted for each reference indicator; the reference indicator is an indicator that has a positive correlation with the indicator to be predicted and for which the amount of the reference historical data is greater than or equal to the preset threshold;

[0133] A second determining module 706 is configured to determine an abnormal data interval corresponding to abnormal historical data in the historical data, determine a first distribution probability of each data distribution form corresponding to the abnormal data interval, and determine a target data distribution form from each data distribution form based on the first distribution probability;

[0134] The third determining module 708 is configured to determine an abnormal indicator interval corresponding to the indicator to be predicted within the period to be predicted based on the data baseline, the target data distribution pattern, and a preset abnormal data probability.

[0135] The indicator abnormal interval determination device provided in the embodiment of the present application first predicts a data baseline based on historical data, and then, for the indicator to be predicted with less historical data and unable to accurately determine the data distribution form under the indicator, determines the data distribution form that the data under the indicator may obey based on a reference indicator that may have a positive correlation with the indicator and has more historical data, and then determines the target data distribution form based on the distribution probability of the abnormal data interval where the abnormal historical data in the historical data corresponding to these data distribution forms are located, so that the indicator abnormal interval constructed according to the target data distribution form can detect abnormal values. Then, based on the data baseline, the target data distribution form and the preset abnormal data probability, the indicator abnormal interval corresponding to the indicator to be predicted in the period to be predicted can be constructed. The embodiment of the present application can dynamically adjust the data baseline based on the historical data of the indicator to be predicted, and then dynamically adjust the indicator abnormal interval based on the data baseline. And when the historical data of the indicator to be predicted is insufficient, the historical data of other indicators can also be used to determine the possible location of the abnormal value under the indicator to be predicted, thereby improving the determination accuracy of the indicator abnormal interval.

[0136] In one embodiment, the first determining module 704 is further configured to:

[0137] When the amount of the historical data is greater than or equal to a preset amount threshold, the target data distribution form is determined based on the historical data.

[0138] In one embodiment, the second determining module 706 is further configured to:

[0139] determining normal historical data and abnormal historical data from the historical data;

[0140] determining a data interval length based on a difference between a maximum value and a minimum value in the normal historical data and the amount of the normal historical data, and dividing the normal historical data into a plurality of normal data intervals based on the data interval length;

[0141] According to the data interval length, data intervals are further divided on both sides of each normal data interval until all the historical data have corresponding data intervals, and the data intervals with abnormal historical data are regarded as abnormal data intervals.

[0142] In one embodiment, the second determining module 706 is further configured to:

[0143] Constructing probability density functions corresponding to the data distribution forms according to normal historical data in the historical data;

[0144] Integrating each of the probability density functions over the abnormal data intervals respectively, to obtain a first distribution probability of each of the data distribution forms corresponding to the abnormal data intervals.

[0145] In one embodiment, the second determining module 706 is further configured to:

[0146] The data distribution form with the minimum first distribution probability is used as the target data distribution form.

[0147] In one embodiment, the second determining module 706 is further configured to:

[0148] Determining a normal data interval corresponding to normal historical data in the historical data;

[0149] respectively determining a second distribution probability of each of the data distribution forms corresponding to the normal data interval;

[0150] Determining a detection accuracy score corresponding to each data distribution form based on the first distribution probability and the second distribution probability of each data distribution form, wherein the detection accuracy score is negatively correlated with the first distribution probability and positively correlated with the second distribution probability;

[0151] The data distribution form with the highest detection accuracy score is used as the target data distribution form.

[0152] In one embodiment, the third determining module 708 is further configured to:

[0153] Taking the data baseline as the data mean or data median of the indicator to be predicted within the time period to be predicted, and constructing a target probability density function of the target data distribution corresponding to the data baseline based on the data mean or the data median;

[0154] An indicator abnormality interval is determined based on the target probability density function, and an integral value of the target probability density function in the indicator abnormality interval is equal to the preset abnormal data probability.

[0155] In one embodiment, the third determining module 708 is further configured to:

[0156] Determine the estimated distribution range of abnormal data corresponding to the target probability density function, and determine the target abnormal data probability corresponding to each of the estimated distribution ranges of abnormal data according to the preset abnormal data probability, wherein the sum of the target abnormal data probabilities is the preset abnormal data probability;

[0157] Based on the probability of each target abnormal data, determining the cumulative probability lower boundary value and the cumulative probability upper boundary value corresponding to the estimated distribution range of each abnormal data;

[0158] For any of the abnormal data estimated distribution ranges, determining the indicator abnormal interval in the abnormal data estimated distribution range according to the cumulative probability lower boundary value and the cumulative probability upper boundary value;

[0159] Among them, the integral result of the target probability density function from the lower boundary value to the lower boundary value of the indicator abnormal interval is the cumulative probability lower boundary value, and the integral result of the target probability density function from the lower boundary value to the upper boundary value of the indicator abnormal interval is the cumulative probability upper boundary value.

[0160] Each module in the above-mentioned apparatus may be implemented in whole or in part by software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0161] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When executed by the processor, the computer program implements a method for determining an abnormal indicator interval.

[0162] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0163] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0164] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0165] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0166] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0167] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0168] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for determining an abnormal interval of an indicator, characterized in that: The method comprises: Determine an indicator to be predicted from each indicator, and predict a data baseline of the indicator to be predicted in the period to be predicted based on historical data of the indicator to be predicted corresponding to the period to be predicted; When the amount of the historical data is less than a preset threshold, a reference indicator is determined from the indicators, and a data distribution form of each reference indicator is determined based on the reference historical data of each reference indicator corresponding to the time period to be predicted; the reference indicator is an indicator that has a positive correlation with the indicator to be predicted and the amount of the reference historical data is greater than or equal to the preset threshold; Determine an abnormal data interval corresponding to abnormal historical data in the historical data, respectively determine a first distribution probability of each of the data distribution forms corresponding to the abnormal data interval, and determine a target data distribution form from each of the data distribution forms according to the first distribution probability; Based on the data baseline, the target data distribution form and the preset abnormal data probability, the indicator abnormal interval corresponding to the indicator to be predicted within the time period to be predicted is determined.

2. The method according to claim 1, characterized in that The method further comprises: When the amount of the historical data is greater than or equal to a preset amount threshold, the target data distribution form is determined based on the historical data.

3. The method according to claim 1, characterized in that The determining of the abnormal data interval corresponding to the abnormal historical data in the historical data includes: determining normal historical data and abnormal historical data from the historical data; determining a data interval length based on a difference between a maximum value and a minimum value in the normal historical data and the amount of the normal historical data, and dividing the normal historical data into a plurality of normal data intervals based on the data interval length; According to the data interval length, data intervals are further divided on both sides of each normal data interval until all the historical data have corresponding data intervals, and the data intervals with abnormal historical data are regarded as abnormal data intervals.

4. The method according to claim 1, wherein The determining of the first distribution probability of each of the data distribution forms corresponding to the abnormal data interval includes: Constructing probability density functions corresponding to the data distribution forms according to normal historical data in the historical data; Integrating each of the probability density functions over the abnormal data intervals respectively, to obtain a first distribution probability of each of the data distribution forms corresponding to the abnormal data intervals.

5. The method according to any one of claims 1 to 4, characterized in that The determining of a target data distribution form from each of the data distribution forms according to the first distribution probability includes: The data distribution form with the minimum first distribution probability is used as the target data distribution form.

6. The method according to any one of claims 1 to 4, characterized in that The determining of a target data distribution form from each of the data distribution forms according to the first distribution probability includes: Determining a normal data interval corresponding to normal historical data in the historical data; respectively determining a second distribution probability of each of the data distribution forms corresponding to the normal data interval; Determining a detection accuracy score corresponding to each data distribution form based on the first distribution probability and the second distribution probability of each data distribution form, wherein the detection accuracy score is negatively correlated with the first distribution probability and positively correlated with the second distribution probability; The data distribution form with the highest detection accuracy score is used as the target data distribution form.

7. The method according to claim 1, characterized in that The determining, based on the data baseline, the target data distribution form, and the preset abnormal data probability, of an indicator abnormal interval corresponding to the indicator to be predicted within the period to be predicted includes: Taking the data baseline as the data mean or data median of the indicator to be predicted within the time period to be predicted, and constructing a target probability density function of the target data distribution corresponding to the data baseline based on the data mean or the data median; An indicator abnormality interval is determined based on the target probability density function, and an integral value of the target probability density function in the indicator abnormality interval is equal to the preset abnormal data probability.

8. The method according to claim 7, characterized in that Determining the indicator abnormality interval based on the target probability density function includes: Determine the estimated distribution range of abnormal data corresponding to the target probability density function, and determine the target abnormal data probability corresponding to each of the estimated distribution ranges of abnormal data according to the preset abnormal data probability, wherein the sum of the target abnormal data probabilities is the preset abnormal data probability; Based on the probability of each target abnormal data, determining the cumulative probability lower boundary value and the cumulative probability upper boundary value corresponding to the estimated distribution range of each abnormal data; For any of the abnormal data estimated distribution ranges, determining the indicator abnormal interval in the abnormal data estimated distribution range according to the cumulative probability lower boundary value and the cumulative probability upper boundary value; Among them, the integral result of the target probability density function from the lower boundary value to the lower boundary value of the indicator abnormal interval is the cumulative probability lower boundary value, and the integral result of the target probability density function from the lower boundary value to the upper boundary value of the indicator abnormal interval is the cumulative probability upper boundary value.

9. A device for determining abnormal interval of an indicator, characterized in that: The device comprises: A prediction module is used to determine an indicator to be predicted from various indicators, and predict a data baseline of the indicator to be predicted in the period to be predicted based on historical data of the indicator to be predicted corresponding to the period to be predicted; a first determining module configured to, when the amount of the historical data is less than a preset threshold, determine a reference indicator from the indicators, and determine a data distribution form of each reference indicator based on the reference historical data corresponding to the time period to be predicted for each reference indicator; the reference indicator is an indicator that has a positive correlation with the indicator to be predicted and for which the amount of the reference historical data is greater than or equal to the preset threshold; a second determining module, configured to determine an abnormal data interval corresponding to abnormal historical data in the historical data, respectively determine a first distribution probability of each of the data distribution forms corresponding to the abnormal data interval, and determine a target data distribution form from each of the data distribution forms according to the first distribution probability; The third determination module is used to determine the indicator abnormality interval corresponding to the indicator to be predicted within the time period to be predicted based on the data baseline, the target data distribution form and the preset abnormal data probability.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images