Outlier detection method, device, equipment and storage medium

Abstract

The embodiment of the present invention provides an abnormal point detection method, device, equipment and storage medium, the method comprising: first, obtaining the measurement data of the business indicator time series of the monitored system; second, determining the mean baseline for the business indicator time series, and calculating the measurement residual data of the measurement data and the mean baseline; then, performing residual prediction on the measurement residual data to generate predicted residual data, and finally, determining the abnormal point of the monitored system on the time series according to the predicted residual data and the measurement residual data. In this way, the false alarm rate is reduced when detecting abnormal situations, and the system stability is maintained.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to an abnormal point detection method, device, equipment and storage medium. Background Art

[0002] As business operations become more dependent on data, the need to keep abreast of data dynamics and quickly and accurately interpret data is growing. Correct data indicators can help us accurately grasp user trends and website business conditions. Therefore, timely discovery and resolution of data anomalies have become a concern in business operations.

[0003] At present, the main way to detect abnormalities in data is to compare the measured value of the data point with the artificially pre-set indicator abnormality threshold to determine whether the data point is abnormal. However, this method will have a high false alarm rate when the system has regular data jitter, or when there is a regular increase or decrease due to business characteristics, because the artificially set abnormality threshold is not flexible enough and only the abnormality threshold is used to determine whether the data point is abnormal. Summary of the invention

[0004] One or more embodiments of the present invention describe an outlier detection method, apparatus, device, and storage medium to solve the problem of high false alarm rate when detecting anomalies.

[0005] According to a first aspect, a method for detecting an outlier is provided, which may include:

[0006] Obtain the measurement data of the business indicator time series of the monitored system;

[0007] Determine the mean baseline for the business indicator time series and calculate the measurement residual data between the measured data and the mean baseline;

[0008] Perform residual prediction on the measurement residual data to generate predicted residual data;

[0009] Based on the prediction residual data and the measurement residual data, the abnormal points of the monitored system in the time series are determined.

[0010] According to a second aspect, an outlier detection device is provided, which may include:

[0011] The acquisition module is used to obtain the measurement data of the business indicator time series of the monitored system;

[0012] A processing module, used for determining a mean baseline for a business indicator time series, and calculating measurement residual data between the measurement data and the mean baseline;

[0013] A prediction module, used for performing residual prediction on the measurement residual data to generate predicted residual data;

[0014] The generation module is used to determine the abnormal points of the monitored system in the time series based on the prediction residual data and the measurement residual data.

[0015] According to the third aspect, a computing device is provided, which includes a transceiver, at least one processor and a memory, the transceiver is used to send and receive data, the memory is used to store computer program instructions, and the processor is used to execute the program in the memory to control the server to implement any one of the methods of the first aspect or the second aspect.

[0016] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. If the computer program is executed in a computer, the computer is caused to execute any one of the methods of the first aspect or the second aspect.

[0017] By using the solution of the embodiment of the present invention, the mean baseline determined by historical data is subtracted from the measured value, so as to eliminate the abnormal false alarms caused by the periodic increase and decrease of real-time indicators and the regular data jitter in the system. Then, the abnormal points of the monitored system in the time series are determined by the measurement residual data determined by the measurement data and the mean baseline, and the prediction residual data, so as to reduce or eliminate the possibility of false alarms or missed alarms in the abnormal point detection process, so as to maintain the stability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The present invention may be better understood from the following description of specific embodiments of the present invention in conjunction with the accompanying drawings, wherein the same or similar reference numerals represent the same or similar features.

[0019] Figure 1 A schematic diagram showing the structure of a data center according to an embodiment is shown;

[0020] Figure 2 A schematic diagram showing an application scenario of an outlier detection method according to an embodiment is shown;

[0021] Figure 3 A schematic diagram showing another application scenario of an outlier detection method according to an embodiment;

[0022] Figure 4 A flow chart showing an outlier detection method according to an embodiment;

[0023] Figure 5 A flow chart showing an outlier detection method according to another embodiment;

[0024] Figure 6 A flowchart showing an outlier detection system implementing outlier detection for a data center according to an embodiment is shown;

[0025] Figure 7 A block diagram of an outlier detection device according to an embodiment is shown;

[0026] Figure 8 A schematic diagram of the structure of a computing device according to an embodiment is shown. DETAILED DESCRIPTION

[0027] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention.

[0028] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such measured relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0029] In order to solve the problems in the prior art, the embodiments of the present invention provide an outlier detection method, apparatus, device and storage medium, as shown below.

[0030] Figure 1 A schematic diagram of the structure of a data center according to an embodiment is shown.

[0031] like Figure 1 As shown, the data center includes several monitored systems, including monitored system 1, monitored system 2, monitored system 3, and an abnormal point detection system 4.

[0032] The abnormal point detection system 4 determines the operation status of the data center by detecting the business indicators of the monitored system. The business indicators may come from the monitored system or other applications, or from the hardware equipment of the data center or other sources.

[0033] For example, in a scenario where a data center manages multiple customer service robots, Figure 2 As shown, the monitored system can be a customer service robot under different business lines, and the business indicators of the monitored system can be an indicator system for measuring the robot's operating performance and question-answering performance. Furthermore, monitored system 1 can be a customer service robot serving consumers, monitored system 2 can be a customer service robot serving merchants, and monitored system 3 can be a customer service robot serving enterprises. Business indicators can be visit volume, human-machine dialogue rounds, human-machine dialogue volume, transfer rate to manual, single-round dialogue resolution rate, etc.

[0034] The following uses the transfer rate of a customer service robot serving consumers as an example to explain in detail through two application scenarios:

[0035] In a possible application scenario, Figure 2 As shown, the abnormal point detection system 4 obtains the measurement data of the business indicator time series of the transfer rate of the customer service robot serving consumers; then, the mean baseline for the business indicator time series is determined, and the measurement residual data of the measurement data and the mean baseline is calculated; secondly, the residual prediction of the measurement residual data is performed to generate the predicted residual data; then, based on the predicted residual data and the measured residual data, it is determined that the transfer rate of the customer service robot serving consumers at 23:00 pm Beijing time is 89%, and the data of the transfer rate corresponding to this time point is determined as an abnormal point. When the abnormal point information "the transfer rate of the customer service robot serving consumers is 89% (23:00 pm Beijing time)" is detected, it is actively pushed to the user through the data robot device, such as: "My dear, hello, the transfer rate of the customer service robot serving consumers at 23:00 pm Beijing time is 89%, which is different from usual, please pay attention!"

[0036] In another possible application scenario, Figure 3 As shown, the anomaly detection system 4 obtains the measurement data of the business indicator time series of the transfer rate of the customer service robot serving consumers; then, determines the mean baseline for the business indicator time series, and calculates the measurement residual data between the measurement data and the mean baseline; secondly, performs residual prediction on the measurement residual data to generate predicted residual data; then, based on the predicted residual data and the measured residual data, determines that the transfer rate of the customer service robot serving consumers at 23:00 pm Beijing time is 89%, and the data of the transfer rate corresponding to this time point is taken as an anomaly.

[0037] When the data robot device receives query information in which a natural voice asks whether the indicator data is abnormal during a certain period of time or at a certain moment, it obtains the abnormal point information corresponding to the query information and pushes the abnormal point to the user.

[0038] For example, when the data robot receives a user voice query message such as "Hello, data robot, please help me check if there was any abnormality last night?" or "Hello, data robot, please help me check if there was any abnormality in the data center at 23:00 last night", it obtains and queries whether there is any abnormality in the indicator data at a certain period of time or at a certain moment in the information. If the abnormality information is obtained, it pushes the abnormality to the user, such as: "Hello, the customer service robot serving consumers had a transfer rate of 89% at 23:00 Beijing time, which is different from usual, please pay attention!"

[0039] Therefore, in the embodiment of the present invention, by improving and optimizing the abnormal point detection method executed on the abnormal point detection system 4, the monitoring operation can be made more accurate and sensitive, and the problem of high false alarm rate caused by the inflexibility of artificially setting abnormal thresholds can be avoided. Among them, the data center is only an application object of the abnormal point detection method provided in the embodiment of the present invention; in fact, in addition to the data center, the abnormal point detection method provided in the embodiment of the present invention can also be applied to other electronic devices, structures or systems, and the embodiment of the present invention does not limit this.

[0040] In addition, the method provided by the embodiment of the present invention can also push anomaly detection services related to the monitored system according to different attributes of the monitored system. For example, if the monitored system is a customer service robot of a merchant, anomaly detection services corresponding to the number of product visits and / or anomaly detection services corresponding to the number of product inquiries can be pushed to the merchant, so that the merchant can select different categories of anomaly detection services according to the pushed content.

[0041] Based on the above scenario, the embodiment of the present invention provides an outlier detection method. Figure 4 -Attached Figure 5 Two outlier detection methods provided in the embodiments of the present invention are described in detail.

[0042] Embodiment 1:

[0043] Figure 4 A flow chart of an outlier detection method according to an embodiment is shown.

[0044] like Figure 4 As shown, the method flow includes steps 410 to 440: first, in step 410, the measurement data of the business indicator time series of the monitored system is obtained; second, in step 420, the mean baseline for the business indicator time series is determined, and the measurement residual data between the measurement data and the mean baseline is calculated; then, in step 430, residual prediction is performed on the measurement residual data to generate predicted residual data; then, in step 440, the abnormal points of the monitored system in the time series are determined based on the predicted residual data and the measured residual data.

[0045] The following are detailed descriptions of the above steps:

[0046] First, step 410 is involved: obtaining measurement data of the business indicator time series of the monitored system.

[0047] In one example, the business indicator time series includes multiple discrete points; the multiple discrete points include a first discrete point, and measurement data of the first discrete point is used to represent statistical data within a first time window to which the first discrete point belongs.

[0048] For example, the first discrete point corresponds to the data at 10:00 am on May 4. In fact, the measurement data of the first discrete point is the average data of the time period from 9:55 to 10:00 in the morning, where the first time window is the 5-minute difference between 9:55 and 10:00.

[0049] Secondly, it involves step 420: determining a mean baseline for the business indicator time series, and calculating measurement residual data between the measurement data and the mean baseline.

[0050] This step includes two processes, the first is a process of determining a mean baseline, and the second is a process of calculating measurement residual data.

[0051] For the first process, in one example, historical sub-data of multiple discrete points are obtained from the historical data of the business indicator according to the time period; wherein, the measurement data of a preset number of discrete points before each discrete point are respectively selected and the mean is calculated to determine the mean baseline for the first discrete point.

[0052] For example: Assuming that the first discrete point is the measurement data at 10:00 a.m. on May 4, the historical data of the business indicator is divided according to a 24-hour cycle, and the historical sub-data of the three days before May 4 is determined, that is, the historical sub-data from May 1 to May 3. Each of the three historical sub-data includes the historical data of multiple discrete points. The measurement data at the same time as 10:00 a.m. on May 4 in the three historical sub-data are selected respectively, that is, the data at 10:00 a.m. on May 3, the data at 10:00 a.m. on May 2, and the data at 10:00 a.m. on May 1. The measurement data of 75 discrete points before 10:00 a.m. on May 3, 10:00 a.m. on May 2, and 10:00 a.m. on May 1 are selected respectively, so as to determine the average data of each day in the three days, and then, according to the average data of each day, the average baseline for 10:00 a.m. on May 3 is determined. It should be noted that the example part only takes the measurement data of the three days before the first discrete point as an example, and actually includes but is not limited to the statistics of only three days of data.

[0053] Here, the mean baseline in the embodiment of the present invention is constantly adjusted as the data is updated. Specifically, the mean baseline for the first discrete point can be updated according to the periodic variation rule of the historical data.

[0054] It can be seen that the mean baseline in the embodiment of the present invention is a dynamic mean baseline, which is different from the static baseline in the related art, so that the mean baseline can perceive the changes caused by factors such as personnel, materials, machines and processing processes, so as to be suitable for sudden personnel or workflows that change over time. Since the dynamic mean baseline can identify anomalies in the case of non-constant fluctuations (for example: regular data jitter in the system or regular growth or decline due to business characteristics), it is better adapted to data changes and can improve the accuracy of determining measurement residual data, so as to avoid false alarms or omissions of abnormal points later.

[0055] For the second process, in one example, measurement residual data is determined based on the difference between the measurement data of the business indicator time series obtained in step 410 and the mean baseline in the first process in step 420 .

[0056] Therefore, step 420 determines the measurement residual data by subtracting the average of the past three days from the measured value, so as to eliminate the abnormal false alarms caused by the periodic increase and decrease of the real-time indicator and the regular data jitter in the system.

[0057] Step 430: Perform residual prediction on the measured residual data to generate predicted residual data.

[0058] In one example, at least one of the following items is used to perform residual prediction on the measurement residual data: exponential smoothing model, autoregressive model, seasonal difference autoregressive moving average model. Wherein, each discrete point of the measurement residual data in the business indicator time series has a corresponding prediction residual data.

[0059] Step 440: Determine the abnormal points of the monitored system in the time series based on the prediction residual data and the measurement residual data.

[0060] In one example, the difference data between the measured residual data and the predicted residual data is calculated; a confidence interval is determined based on the difference data; if the measured data is not within the confidence interval, the discrete point corresponding to the measured data in the time series is determined to be an abnormal point.

[0061] Furthermore, the above-mentioned determining the confidence interval according to the difference data may specifically include: calculating the mean and standard deviation corresponding to the difference data, and determining the confidence interval using the N-sigma judgment method.

[0062] In the embodiment of the present invention, the dynamic baseline mean is calculated using historical measurement data, without manual judgment and setting of empirical thresholds, and is better adapted to data changes. In addition, the measurement residual data is determined by subtracting the mean of the past three days from the measured value, so as to eliminate abnormal false alarms caused by periodic increases and decreases in real-time indicators and regular data jitters in the system. Then, the measurement residual data is predicted and the confidence interval is calculated through the residual prediction model, which can ensure real-time response to sudden warnings and maintain a low false alarm rate.

[0063] Embodiment 2:

[0064] Based on the method shown in Example 1, the embodiment of the present invention provides another outlier detection method. The difference between this method and Example 1 is mainly in step 440, that is, in addition to determining the outlier in the business indicator time series by predicting residual data and measuring residual data, it is also possible to determine the outlier on the business indicator time series by using the month-on-month threshold, the year-on-year threshold, the predicted residual data and the measured residual data. Figure 5 Another outlier detection method is described in detail.

[0065] Figure 5 A flow chart of an outlier detection method according to another embodiment is shown.

[0066] like Figure 5 As shown, the method flow includes steps 510 to 540: first, in step 510, it is determined whether the measurement data of the business indicator time series belongs to an incremental window-type indicator; second, in step 520, the month-on-month time series and the year-on-year time series corresponding to the measurement data are calculated respectively; then, in step 530, the month-on-month threshold and the year-on-year threshold are determined according to the month-on-month time series and the year-on-year time series; then, in step 540, the abnormal points of the monitored system in the time series are determined according to the month-on-month threshold, the year-on-year threshold, the predicted residual data and the measured residual data.

[0067] The following are detailed descriptions of the above steps:

[0068] First, step 510 is involved: determining whether the measurement data of the business indicator time series belongs to an incremental window-type indicator.

[0069] In one example, if the measurement data of the business indicator time series belongs to an incremental window type indicator, then continue to execute step 520. If the measurement data of the business indicator time series does not belong to an incremental window type indicator, then convert the measurement data of the business indicator time series into an incremental window type indicator, and then execute step 520.

[0070] Among them, the incremental window type indicator refers to a type of cumulative data. The following methods can be used to determine whether the measured data belongs to the incremental window type indicator:

[0071] The difference between the measurement data in the first time window and the measurement data in the second time window is calculated. When the difference is greater than a preset threshold (such as the preset threshold is zero), it is determined that the measurement data belongs to an incremental window type indicator; conversely, when the difference is less than the preset threshold (such as the preset threshold is zero), it is determined that the measurement data does not belong to an incremental window type indicator.

[0072] When the measured data does not belong to the incremental window-type indicator, the data source of the measured data is detected, the data source is adjusted to meet the conditions of the window-type indicator, and then converted back into the incremental window-type indicator.

[0073] Therefore, in this example, the business anomaly is actually reflected through the data anomaly. Step 510 is to eliminate the possibility of errors in the data source itself, so as to obtain a higher precision month-on-month threshold and year-on-year threshold.

[0074] In another example, before step 510, the method may further include: dividing the measurement data of the business indicator time series according to data categories;

[0075] If the measurement data is divided into proportion indicator data or magnitude indicator data according to the data category, the measurement data is an incremental window indicator. Among them, proportion indicator data refers to the measurement data presented in the form of fractional values, and magnitude indicator data refers to the measurement data presented in the form of exponents.

[0076] Next, in step 520, the month-on-month time series and year-on-year time series corresponding to the measurement data are calculated respectively.

[0077] In one example, the month-on-month value and year-on-year value of the measurement data are calculated respectively; the month-on-month time series corresponding to the measurement data is determined according to the month-on-month value, and the year-on-year time series corresponding to the measurement data is determined according to the year-on-year value.

[0078] Next, in step 530, a month-on-month threshold and a year-on-year threshold are determined according to the month-on-month time series and the year-on-year time series.

[0079] In one example, a preset month-on-month threshold rule is applied to the month-on-month time series to determine a month-on-month threshold; and a preset year-on-year threshold rule is applied to the year-on-year time series to determine a year-on-year threshold.

[0080] In another example, before step 530, the process may also include: performing data cleaning on the month-on-month time series and the year-on-year time series. Data cleaning is used to review and verify the measurement data in order to delete duplicate data, correct existing errors, and provide measurement data consistency so as to improve the accuracy of the month-on-month threshold and the year-on-year threshold.

[0081] Then, in step 540, the abnormal points of the monitored system in the time series are determined according to the month-on-month threshold, the year-on-year threshold, the prediction residual data and the measurement residual data.

[0082] In one example, a confidence interval is determined based on the predicted residual data and the measured residual data; if the measured data is not within the confidence interval, the discrete point corresponding to the measured data in the time series is taken as a hypothetical outlier; the hypothesized outlier is corrected using the month-on-month threshold and the year-on-year threshold to determine whether the hypothesized outlier is an outlier.

[0083] It should be noted that the method of determining the prediction residual data and the measurement residual data is the same as step 410 to step 430 in Example 1, and will not be repeated here.

[0084] Furthermore, the assumed abnormal point is corrected by the month-on-month threshold and the year-on-year threshold. If the assumed abnormal point is still determined to be abnormal after correction, the discrete point corresponding to the measured data in the time series is determined to be an abnormal point; conversely, if the assumed abnormal point is determined to be normal after correction, the discrete point corresponding to the measured data in the time series is determined to be a normal point.

[0085] In one example, when it is determined that a discrete point corresponding to the measurement data in the time series is an outlier, the outlier needs to be processed digitally. The specific digital processing method can be to determine the degree of discreteness of the outlier through an isolation forest model or a local anomaly factor algorithm.

[0086] For example, determine that the first discrete point is an outlier, and use the measurement data of the business indicator time series where the first discrete point is located through an isolation forest model or a local anomaly factor algorithm to output the anomaly score of the first discrete point; perform weighted sum of the anomaly score and the probability score after the acquired statistical distribution probability value is standardized to determine the anomaly score of the first discrete point, and the anomaly score is used to indicate the degree of discreteness of the first discrete point.

[0087] In the embodiment of the present invention, the month-on-month threshold and the year-on-year threshold are determined by measuring the time series of the business indicator to avoid the error of the data source itself causing the business abnormality false alarm. The confidence interval is corrected by the month-on-month threshold and the year-on-year threshold, which helps to further improve the monitoring accuracy of the business indicator compared with embodiment 1, avoiding false alarms or omissions.

[0088] For ease of understanding, the following takes an example in which an outlier detection system performs outlier detection on a data center to illustrate the outlier detection method provided by the embodiment of the present invention.

[0089] Figure 6 A flow chart showing an outlier detection system implementing outlier detection for a data center according to an embodiment is shown.

[0090] like Figure 6 As shown, the detection process may include steps 601 to 610, which are specifically as follows:

[0091] First, receive an outlier detection task request.

[0092] The task request may be determined based on a query instruction issued by a user during human-computer interaction, or may be a task request issued by a data center itself.

[0093] Step 601: Respond to an outlier detection task request and obtain measurement data of a business indicator time series of a monitored system in a data center.

[0094] For example: record the measurement data of the business indicator time series as {X1, X2, X3...X10}.

[0095] From here, it is divided into two processes, one is the process of determining the confidence interval, and the other is the process of determining the month-on-month threshold and year-on-year threshold. Then, the confidence interval, month-on-month threshold and year-on-year threshold are used to determine the abnormal points in the business indicator time series.

[0096] Step 602: Divide the measurement data of the business indicator time series according to data categories.

[0097] If the measurement data is divided into proportion indicator data or magnitude indicator data according to the data category, the measurement data is an incremental window indicator.

[0098] Step 603: Determine whether the measurement data belongs to an incremental window type indicator.

[0099] When the measurement data belongs to an incremental window type indicator, step 604 is executed.

[0100] In the case that the measured data does not belong to an incremental window type indicator, the non-incremental window type indicator is converted into an incremental window type indicator, and then step 604 is executed.

[0101] For example: calculate whether X2-X1 is greater than or equal to 0. If it is greater than 0, the measured data is an incremental window indicator. If it is equal to 0, calculate whether X3-X2 is greater than 0, and calculate in sequence until it is determined that the difference between the current and the previous one is greater than 0, then it is determined to be an incremental window indicator. If it is less than 0, it is necessary to determine whether the data source of X2 itself is correct. If the data source itself is wrong, X2 can be deleted or a new data source for X2 can be obtained.

[0102] Step 604: Calculate the month-on-month time series and year-on-year time series corresponding to the measurement data respectively.

[0103] The month-on-month value and year-on-year value of the measured data are calculated, and then the month-on-month time series and year-on-year time series are determined based on the month-on-month value and year-on-year value.

[0104] For example, the year-on-year value of X2 is calculated using formula (1):

[0105] (X2-X1) / X1*100% (1);

[0106] The year-on-year value of X2 is calculated by formula (2):

[0107] (X2-N) / N*100% (2);

[0108] Among them, N is the measurement data at the same time in the previous cycle of X2, that is, when X2 represents the measurement data at 10 am on May 4, N is the measurement data at 10 am on May 4.

[0109] According to the above method, the corresponding month-on-month value and year-on-year value of each measurement data are determined respectively, and then the month-on-month time series corresponding to the measurement data is determined according to multiple month-on-month values, and the year-on-year time series corresponding to the measurement data is determined according to multiple year-on-year values.

[0110] Step 605: Clean the data of the month-on-month time series and the year-on-year time series.

[0111] Step 606: Determine a month-on-month threshold and a year-on-year threshold according to the cleaned month-on-month time series and year-on-year time series.

[0112] For example: a preset month-on-month threshold rule is applied to the month-on-month time series to determine the month-on-month threshold; a preset year-on-year threshold rule is applied to the year-on-year time series to determine the year-on-year threshold.

[0113] Step 607: Determine the mean baseline for the business indicator time series, and calculate the measurement residual data between the measurement data and the mean baseline.

[0114] The step of calculating the mean baseline can refer to the method shown in step 420, which will not be repeated here. The mean baselines are marked as {M1, M2, M3...M10}, where M1 is the mean baseline of X1, and so on.

[0115] Calculate the measurement residual data between the measurement data and the mean baseline, for example: {X1-M1, X2-M2, X3-M3, ..., X10-M10}, and record the difference between the two, i.e. the measurement residual data, as {Y1, Y2, Y3 ... Y10}.

[0116] Step 608: Perform residual prediction on the measured residual data to generate predicted residual data.

[0117] Residual prediction is performed on the measurement residual data through at least one of the following models: exponential smoothing model, autoregressive model, seasonal difference autoregressive sliding average model, and residual prediction is performed on the measurement residual data {Y1, Y2, Y3...Y10} to determine the predicted residual data {Z1, Z2, Z3...Z10}, where each measured value Y corresponds to a predicted value Z.

[0118] Step 609: Determine a confidence interval based on the prediction residual data and the measurement residual data.

[0119] Calculate the difference data between the measured residual data and the predicted residual data; determine the confidence interval based on the difference data; if the measured data is not within the confidence interval, the discrete point corresponding to the measured data in the time series is taken as the hypothetical abnormal point.

[0120] For example: the sequence (Y1-Z1), (Y2-Z2), ... (Y10-Z10) is recorded as {C1, C2, C3, ... C10}, the mean and standard deviation are calculated for {C1, C2, C3, ... C10}, and the confidence interval is determined based on the mean and standard deviation combined with N-sigma (that is, mean + N*standard deviation as the upper limit interval, mean - N*standard deviation as the lower limit interval).

[0121] Step 610: Determine the abnormal points of the monitored system in the time series according to the month-on-month threshold, the year-on-year threshold, the prediction residual data and the measurement residual data.

[0122] The assumed abnormal point is corrected by the month-on-month threshold and the year-on-year threshold determined in step 606 to determine the correction result. If the correction result indicates that the assumed abnormal point remains abnormal, the assumed abnormal point is determined to be an abnormal point; otherwise, if the correction result indicates that the assumed abnormal point is misjudged, the assumed abnormal point is marked as a normal point.

[0123] It should be noted that steps 602 to 606 and steps 607 to 609 in the embodiment of the present invention can be processed sequentially or simultaneously. This order is only a schematic representation. The embodiment of the present invention does not limit the order of execution of steps 602-606 and steps 607-609.

[0124] Here, the measurement data of the business indicator time series where the outlier is located is used to output the outlier score of the outlier through the isolation forest model or the local outlier factor algorithm; the outlier score and the probability score after the obtained statistical distribution probability value is standardized are weighted and summed to determine the outlier score of the outlier, and the outlier score is used to indicate the degree of discreteness of the outlier.

[0125] In an embodiment of the present invention, the dynamic baseline mean is calculated using historical measurement data, without manual judgment and setting of empirical thresholds, and is better adapted to data changes. In addition, the measurement residual data is determined by subtracting the mean of the past three days from the measured value, so as to eliminate abnormal false alarms caused by periodic increases and decreases in real-time indicators and regular data jitters in the system. Then, the measurement residual data is predicted and the confidence interval is calculated through the residual prediction model, which can not only ensure real-time response to sudden warnings, but also maintain a low false alarm rate. Then, the month-on-month threshold and the year-on-year threshold are determined by the measurement data of the business indicator time series to avoid business abnormal false alarms caused by errors in the data source itself. Correcting the confidence interval based on the month-on-month threshold and the year-on-year threshold helps to further improve the monitoring accuracy of business indicators and avoid false alarms or omissions.

[0126] Figure 7 A block diagram of an outlier detection device according to an embodiment is shown.

[0127] like Figure 7 As shown, the device 70 may include:

[0128] The acquisition module 701 is used to obtain the measurement data of the business indicator time series of the monitored system;

[0129] Processing module 702, used to determine the mean baseline for the business indicator time series, and calculate the measurement residual data between the measurement data and the mean baseline;

[0130] The prediction module 703 is used to perform residual prediction on the measurement residual data to generate predicted residual data;

[0131] The generating module 704 is used to determine the abnormal points of the monitored system in the time series according to the prediction residual data and the measurement residual data.

[0132] The business indicator time series includes multiple discrete points; the multiple discrete points include a first discrete point, and the measurement data of the first discrete point is used to represent the statistical data within the first time window to which the first discrete point belongs. The business indicator of the monitored system includes at least one of the following: access volume, human-computer dialogue rounds, human-computer dialogue volume, transfer rate to manual, and single-round dialogue resolution rate.

[0133] Here, at least one of the following is used to perform residual prediction on the measurement residual data: exponential smoothing model, autoregressive model, seasonal difference autoregressive moving average model.

[0134] In one example, the processing module 702 in the embodiment of the present invention can be specifically used to obtain historical sub-data of multiple discrete points from the historical data of the business indicator according to a time period; wherein, the measurement data of a preset number of discrete points before each discrete point are selected respectively, and the mean calculation is performed to determine the mean baseline for the first discrete point.

[0135] In one example, the processing module 704 in the embodiment of the present invention can be specifically used to calculate the difference data between the measured residual data and the predicted residual data; determine the confidence interval based on the difference data; if the measured data is not within the confidence interval, the discrete point corresponding to the measured data in the time series is an abnormal point.

[0136] In one example, the processing module 702 can also be used to, if the measurement data of the business indicator time series belongs to an incremental window type indicator, respectively calculate the month-on-month time series and the year-on-year time series corresponding to the measurement data; determine the month-on-month threshold and the year-on-year threshold according to the month-on-month time series and the year-on-year time series. Further, if the measurement data of the business indicator time series does not belong to an incremental window type indicator, the measurement data of the business indicator time series is converted into an incremental window type indicator. The processing module 702 can also be used to perform data cleaning on the month-on-month time series and the year-on-year time series.

[0137] Here, the processing module 702 may also be used to, if it is determined according to the data category that the measurement data is divided into proportion indicator data or magnitude indicator data, then the measurement data is an incremental window-type indicator.

[0138] In one example, the generation module 704 can be specifically used to determine the abnormal point of the monitored system in the time series according to the month-on-month threshold, the year-on-year threshold, the predicted residual data and the measured residual data. Further, the generation module 704 determines the confidence interval according to the predicted residual data and the measured residual data; if the measured data is not within the confidence interval, the discrete point corresponding to the measured data in the time series is used as the hypothesized abnormal point; and the month-on-month threshold and the year-on-year threshold are used to determine whether the hypothesized abnormal point is an abnormal point.

[0139] In one example, the generating module 704 may also be used to determine the degree of dispersion of the outliers by using an isolation forest model or a local outlier factor algorithm.

[0140] In the embodiment of the present invention, the dynamic baseline mean is calculated using historical measurement data, without manual judgment and setting of empirical thresholds, and is better adapted to data changes. In addition, by determining the predicted residual data and calculating the confidence interval through the residual prediction model, it is possible to ensure real-time response to sudden warnings and maintain a low false alarm rate. In addition, the month-on-month threshold and year-on-year threshold are determined by measuring the business indicator time series data to avoid business anomaly false alarms caused by errors in the data source itself. Correcting the confidence interval through the month-on-month threshold and year-on-year threshold helps to improve the monitoring accuracy of business indicators and avoid false alarms or omissions.

[0141] Figure 8 A schematic diagram of the structure of a computing device according to an embodiment is shown.

[0142] like Figure 8 , which is a structural diagram of an exemplary hardware architecture of a computing device capable of implementing the outlier detection method and apparatus according to an embodiment of the present invention.

[0143] The device may include a processor 801 and a memory 802 storing computer program instructions.

[0144] Specifically, the processor 801 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0145] The memory 802 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 802 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 802 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 802 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 802 is a non-volatile solid-state memory. In a particular embodiment, the memory 802 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory, or a combination of two or more of these.

[0146] The processor 801 implements any one of the outlier detection methods in the above embodiments by reading and executing computer program instructions stored in the memory 802 .

[0147] The transceiver 803 is mainly used to implement the communication between the devices in the embodiments of the present invention or with other devices.

[0148] In one example, the device may further include a bus 804. Figure 8 As shown, the processor 801, the memory 802 and the transceiver 803 are connected via a bus 804 and communicate with each other.

[0149] Bus 804 includes hardware, software or both.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In suitable cases, bus 803 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0150] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the steps of the outlier detection method of the embodiment of the present invention.

[0151] It should be clear that the present invention is not limited to the specific configurations and processes described in the above embodiments and shown in the figures. For the convenience and brevity of description, a detailed description of the known methods is omitted here, and the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, which will not be repeated here.

[0152] Those skilled in the art can clearly understand that the method process of the present invention is not limited to the specific steps described and shown. Any technician familiar with the technical field, after understanding the spirit of the present invention, can make various changes, modifications and additions, or equivalent substitutions and change the order of steps within the technical scope disclosed by the present invention. These modifications or substitutions should be included in the protection scope of the present invention.

Claims

1. A method for detecting anomalies, wherein: include: Obtain the measurement data of the business indicator time series of the monitored system; The business indicator time series includes multiple discrete points; The multiple discrete points include a first discrete point, and measurement data of the first discrete point is used to represent statistical data within a first time window to which the first discrete point belongs; Determine a mean baseline for the business indicator time series, and calculate measurement residual data between the measurement data and the mean baseline; Performing residual prediction on the measurement residual data to generate predicted residual data; Determining abnormal points of the monitored system in a time series according to the prediction residual data and the measurement residual data; Determine the mean baseline for the business indicator time series, including: Acquire historical sub-data of a plurality of discrete points from the historical data of the business indicator according to a time period; The measurement data of a preset number of discrete points before each discrete point are respectively selected to perform mean calculation, so as to determine the mean baseline for the first discrete point.

2. The method according to claim 1, wherein: Determining the abnormal point of the monitored system in the time series according to the prediction residual data and the measurement residual data includes: Calculating difference data between the measurement residual data and the prediction residual data; determining a confidence interval based on the difference data; If the measurement data is not within the confidence interval, the discrete point corresponding to the measurement data in the time series is determined to be an abnormal point.

3. The method according to claim 1, wherein: At least one of the following items is used to perform residual prediction on the measurement residual data: Exponential smoothing model, autoregressive model, seasonal difference autoregressive moving average model.

4. The method according to claim 1, wherein: Also includes: If the measurement data belongs to an incremental window type indicator, the month-on-month time series and year-on-year time series corresponding to the measurement data are calculated respectively; A month-on-month threshold and a year-on-year threshold are determined according to the month-on-month time series and the year-on-year time series.

5. The method according to claim 4, wherein: Also includes: If the measurement data does not belong to the incremental window type indicator, the measurement data is converted into the incremental window type indicator.

6. The method according to claim 4, wherein: Also includes: If it is determined according to the data category that the measurement data is divided into proportion indicator data or magnitude indicator data, then the measurement data is an incremental window-type indicator.

7. The method according to claim 4, wherein: Before determining the month-on-month threshold and year-on-year threshold, the following steps are also included: Data cleaning is performed on the month-on-month time series and the year-on-year time series.

8. The method according to claim 7, wherein: Determining the abnormal point of the monitored system in the time series according to the prediction residual data and the measurement residual data includes: The abnormal point of the monitored system in the time series is determined according to the month-on-month threshold, the year-on-year threshold, the prediction residual data and the measurement residual data.

9. The method according to claim 8, wherein: Determining an abnormal point of the monitored system in a time series according to the month-on-month threshold, the year-on-year threshold, the prediction residual data, and the measurement residual data includes: Determining a confidence interval based on the prediction residual data and the measurement residual data; If the measurement data is not within the confidence interval, the discrete point corresponding to the measurement data in the time series is taken as a hypothetical abnormal point; The month-on-month threshold and the year-on-year threshold are used to determine whether the assumed abnormal point is an abnormal point.

10. The method according to claim 1, wherein: Also includes: The degree of dispersion of the outliers is determined by using an isolation forest model or a local outlier factor algorithm.

11. The method according to claim 1, wherein: The business indicators of the monitored system include at least one of the following: number of visits, number of human-computer dialogue rounds, amount of human-computer dialogue, transfer rate to manual work, and single-round dialogue resolution rate.

12. An abnormal point detection device, wherein: include: The acquisition module is used to obtain the measurement data of the business indicator time series of the monitored system; The business indicator time series includes multiple discrete points; The multiple discrete points include a first discrete point, and measurement data of the first discrete point is used to represent statistical data within a first time window to which the first discrete point belongs; A processing module, used to determine a mean baseline for the business indicator time series, and calculate measurement residual data between the measurement data and the mean baseline; A prediction module, used for performing residual prediction on the measurement residual data to generate predicted residual data; A generating module, used for determining an abnormal point of the monitored system in a time series according to the prediction residual data and the measurement residual data; The processing module is specifically used to obtain historical sub-data of multiple discrete points from the historical data of the business indicator according to a time period; The measurement data of a preset number of discrete points before each discrete point are respectively selected to perform mean calculation, so as to determine the mean baseline for the first discrete point.

13. A computing device, wherein: The device includes a transceiver, at least one processor and a memory, the transceiver is used to send and receive data, the memory is used to store computer program instructions, and the processor is used to execute the computer program instructions in the memory to control the computer device to implement the outlier detection method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, wherein: If the computer program is executed in a computer, the computer is caused to execute the outlier detection method according to any one of claims 1 to 11.

Images