Front-end abnormality monitoring method, system, device and medium based on ema algorithm

Abstract

The application discloses a front-end exception monitoring method, system and device based on an EMA algorithm and a medium, wherein the front-end exception monitoring method based on the EMA algorithm comprises the following steps: receiving exception information sent by a front-end page; collecting the exception information according to a time sequence to obtain a data source; regarding the number of the exception information in a preset time period as a detection point according to the time sequence, the detection point comprising a historical detection point and a target detection point, and arranging the historical detection points according to the time sequence to obtain a historical detection sequence; predicting a target standard deviation corresponding to the target detection point based on the historical detection sequence; calculating an EMA baseline of the historical detection point; and determining that the front-end page is abnormal when the deviation of at least one target detection point and the corresponding EMA baseline is greater than N times the target standard deviation. The application combines the EMA baseline and the standard deviation to determine whether the detection point is abnormal, thereby reducing the false alarm probability and improving the probability that the front-end abnormality can be learned by a developer.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet, and particularly relates to a front-end exception monitoring method, system and device based on an EMA algorithm and a medium. BACKGROUND

[0002] Front-end exceptions include JS (script) exceptions, service request exceptions, service return exceptions and white screen crashes, which have adverse effects on front-end applications and reduce the use experience of customers, so it is necessary to monitor front-end exceptions, which helps front-end developers to discover and solve front-end exceptions in time and improves the use experience of customers. The monitoring method for front-end exceptions in the prior art only sets a fixed alarm threshold, so the false positive probability is high, and the feedback cannot be performed in time when the front-end has an exception. SUMMARY

[0003] The present application aims to solve the technical problem of the prior art that the front-end cannot perform feedback in time and the false positive probability of front-end exception monitoring is high, and provides a front-end exception monitoring method, system, device and medium based on an EMA algorithm.

[0004] The present application solves the above technical problem by the following technical scheme:

[0005] The present application provides a front-end exception monitoring method based on an EMA algorithm, which comprises the following steps:

[0006] Receiving exception information sent by a front-end page;

[0007] Collecting the exception information according to time sequences to obtain a data source;

[0008] Taking the number of exception information in a preset time period as a detection point according to time sequences, the detection point comprising a historical detection point and a target detection point, arranging the historical detection point according to time sequences to obtain a historical detection sequence;

[0009] Predicting a target standard deviation corresponding to the target detection point based on the historical detection sequence;

[0010] Calculating an EMA baseline of the historical detection point;

[0011] When the deviation of at least one target detection point and the corresponding EMA baseline is greater than N1 times the target standard deviation, determining that the front-end page has an exception.

[0012] Preferably, the step of predicting the target standard deviation corresponding to the target detection point based on the historical detection sequence comprises:

[0013] The first outlier sequence is obtained by filtering out abnormal historical detection points from the historical detection sequence using a low-pass filter, and the standard deviation of the first outlier sequence is calculated to obtain the first standard deviation (std). err ;

[0014] Anomalies in the historical detection sequence are screened using a box plot IQR to obtain a second outlier sequence. The standard deviation of the second outlier sequence is then calculated to obtain the second standard deviation (std). box_err ;

[0015] The mean outlier standard deviation is calculated based on the first standard deviation and the second standard deviation, and then the mean outlier standard deviation square_std is calculated according to the following formula:

[0016]

[0017] An outlier sequence is obtained based on the first outlier sequence and the second outlier sequence, and N2×square_std outlier historical detection points are selected as discard points from the outlier sequence;

[0018] Filter out the discarded points in the historical detection sequence to obtain a new historical detection sequence;

[0019] Repeat the above steps several times using the new historical detection sequence as the historical detection sequence, and calculate the standard deviation of the new historical detection sequence obtained after several repetitions to obtain the target standard deviation.

[0020] Preferably, the step of predicting the target standard deviation corresponding to the target detection point based on the historical detection sequence includes:

[0021] The number of abnormal information within a first preset time period is taken as the first detection point according to the time sequence. The first detection point includes a first historical detection point and a first target detection point.

[0022] The first historical detection points are arranged chronologically to obtain the first historical detection sequence, and the standard deviation of the first historical detection sequence is calculated to obtain the first target standard deviation (std). 1st ;

[0023] The number of abnormal information within the second preset time period is used as the second detection point according to the time sequence. The second detection point includes the second historical detection point and the second target detection point.

[0024] The second historical detection points are arranged chronologically to obtain the second historical detection sequence. The standard deviation of the second historical detection sequence is calculated to obtain the second target standard deviation (std). 2nd ;

[0025] The target standard deviation std is predicted based on the first target standard deviation and the second target standard deviation, and the target standard deviation std is obtained according to the following formula:

[0026]

[0027] Preferably, the step of calculating the EMA baseline of the historical detection points includes:

[0028] Based on the historical detection points, the EMA baseline bl is calculated using the following formula. i :

[0029] Where i represents the order of the historical detection points in the historical detection sequence.

[0030] Preferably, the step of calculating the EMA baseline of the historical detection points includes:

[0031] Verify historical detection points x i Whether a point is an outlier is determined as follows:

[0032]

[0033] Where is_outlier represents the detection result of historical detection points, true means that the historical detection points being verified are normal points, and false means that the historical detection points being verified are abnormal points;

[0034] If historical detection point x i If it is an outlier, then the EMA baseline bl i =bl i-1 ;

[0035] If historical detection point x i If the baseline is normal, the EMA baseline value bl is calculated according to the following formula. i :

[0036]

[0037] Preferably, the step of calculating the EMA baseline of the historical detection points includes:

[0038] Verify historical detection points x i Whether a point is an outlier is determined as follows:

[0039]

[0040] Calculate historical x based on the historical detection sequence i Difference between baseline and baseline The specific formula is as follows:

[0041] wherein t is the number of days involved in the historical detection sequence, i is the order of the historical detection point in the historical detection sequence in a certain day, w is the window width;

[0042] Calculate the historical detection point x i The difference between the current and the baseline, specifically as follows:

[0043]

[0044] If the historical detection point x i is normal or the absolute value of the difference between and is less than a preset threshold, then the EMA baseline value bl i is calculated according to the following formula:

[0045]

[0046] If the current detection point is abnormal and there is no , then the EMA baseline bl i = 2*bl i-1 -bl i-2 ;

[0047] If the current detection point is abnormal and there is , then the EMA baseline bl

[0048] Preferably, when the deviation of at least one of the target detection points from the corresponding EMA baseline is greater than N1 times the target standard deviation, the step of determining that the front-end page is abnormal comprises:

[0049] Calculate the entropy of the historical detection sequence entropy = -∑p*log(p), wherein p is the probability of the existence of any value in the historical detection sequence;

[0050] When the entropy is less than a preset entropy threshold, set the value of N1 to N 11 , when the entropy is greater than the preset entropy threshold, set the value of N1 to N 12 , N 12 is greater than N 11 ;

[0051] and / or,

[0052] When the deviation of at least two consecutive target detection points from the corresponding EMA baseline is greater than N1 times the target standard deviation, it is determined that the front-end page is abnormal;

[0053] and / or,

[0054] When it is determined that the front-end page is abnormal, an alarm notification is sent.

[0055] The application further provides a front-end exception monitoring system based on an EMA algorithm, comprising an exception information receiving unit, a data storage unit, a detection point dividing unit, a target standard deviation predicting unit, an EMA baseline calculating unit, and an exception determining unit.

[0056] The exception information receiving unit is configured to receive exception information sent by a front-end page.

[0057] The data storage unit is configured to collect the exception information in time sequence to obtain a data source.

[0058] The detection point dividing unit is configured to take the number of exception information in a preset time period as a detection point in time sequence, wherein the detection point comprises a historical detection point and a target detection point, and the historical detection points are arranged in time sequence to obtain a historical detection sequence.

[0059] The target standard deviation predicting unit is configured to predict a target standard deviation corresponding to the target detection point based on the historical detection sequence.

[0060] The EMA baseline calculating unit is configured to calculate an EMA baseline of the historical detection point.

[0061] The exception determining unit is configured to determine that the front-end page has an exception when a deviation of at least one target detection point from the corresponding EMA baseline is greater than N1 times the target standard deviation.

[0062] Preferably, the target standard deviation predicting unit is further configured to screen out abnormal historical detection points in the historical detection sequence by low-pass filtering to obtain a first abnormal value sequence, calculate a standard deviation of the first abnormal value sequence to obtain a first standard deviation std err .

[0063] The target standard deviation predicting unit is further configured to screen out abnormal historical detection points in the historical detection sequence by a box plot IQR to obtain a second abnormal value sequence, calculate a standard deviation of the second abnormal value sequence to obtain a second standard deviation std box_err .

[0064] The target standard deviation predicting unit is further configured to calculate an average abnormal standard deviation based on the first standard deviation and the second standard deviation, and calculate the average abnormal standard deviation square_std according to the following formula:

[0065]

[0066] The target standard deviation prediction unit is further configured to select N2xsquare_std abnormal historical detection points from the abnormal value sequence as discarded points;

[0067] The target standard deviation prediction unit is further configured to filter the discarded points in the historical detection sequence to obtain a new historical detection sequence;

[0068] The target standard deviation prediction unit is further configured to repeatedly execute the above steps several times by taking the new historical detection sequence as the historical detection sequence, and calculate the standard deviation of the new historical detection sequence obtained after the several times of repetition to obtain the target standard deviation.

[0069] Preferably, the target standard deviation prediction unit is further configured to take the number of abnormal information in a first preset time period as a first detection point in chronological order, wherein the first detection point includes a first historical detection point and a first target detection point;

[0070] The target standard deviation prediction unit is further configured to arrange the first historical detection points in chronological order to obtain a first historical detection sequence, and calculate the standard deviation of the first historical detection sequence to obtain a first target standard deviation std 1st ;

[0071] The target standard deviation prediction unit is further configured to take the number of abnormal information in a second preset time period as a second detection point in chronological order, wherein the second detection point includes a second historical detection point and a second target detection point;

[0072] The target standard deviation prediction unit is further configured to arrange the second historical detection points in chronological order to obtain a second historical detection sequence, and calculate the standard deviation of the second historical detection sequence to obtain a second target standard deviation std 2nd ;

[0073] The target standard deviation prediction unit is further configured to predict a target standard deviation std based on the first target standard deviation and the second target standard deviation, wherein the target standard deviation std is obtained according to the following formula:

[0074]

[0075] Preferably, the EMA baseline calculation unit is further configured to calculate an EMA baseline bl i based on the historical detection points according to the following formula:

[0076] wherein i is the order of the historical detection point in the historical detection sequence.

[0077] Preferably, the EMA baseline calculation unit is further configured to check whether the historical detection point xi Whether a point is an outlier is determined as follows:

[0078]

[0079] Where is_outlier represents the detection result of historical detection points, true means that the historical detection points being verified are normal points, and false means that the historical detection points being verified are abnormal points;

[0080] The EMA baseline calculation unit is also used to calculate historical detection points x. i If it is an outlier, then the EMA baseline bl i =bl i-1 ;

[0081] The EMA baseline calculation unit is also used to calculate historical detection points x. i If the baseline is normal, the EMA baseline value bl is calculated according to the following formula. i :

[0082]

[0083] Preferably, the EMA baseline calculation unit is also used to verify historical detection points x. i Whether a point is an outlier is determined as follows:

[0084]

[0085] The EMA baseline calculation unit is also used to calculate historical x based on the historical detection sequence. i Difference between baseline and baseline The specific formula is as follows:

[0086] Where t is the number of days involved in the historical detection sequence, i is the order of the historical detection point in the historical detection sequence on a certain day, and w is the window width;

[0087] The EMA baseline calculation unit is also used to calculate historical detection points x. i The difference between the current value and the baseline is calculated using the following formula:

[0088]

[0089] The EMA baseline calculation unit is also used to calculate historical detection points x. i Normal point or and If the absolute value of the difference is less than a preset threshold, the EMA baseline value bl is calculated according to the following formula. i :

[0090]

[0091] the EMA baseline calculation unit is further configured to calculate the EMA baseline bl if the current detection point is abnormal and there is no i = 2 * bl i-1 - bl i-2 ;

[0092] the EMA baseline calculation unit is further configured to calculate the EMA baseline bl if the current detection point is abnormal and there is no

[0093] Preferably, the abnormality determination unit is further configured to calculate the entropy of the historical detection sequence, entropy = -∑p*log(p), where p is the probability of any value existing in the historical detection sequence.

[0094] When the entropy is less than a preset entropy threshold, the value of N1 is set as N 11 When the entropy is greater than the preset entropy threshold, the value of N1 is set as N 12 , N 12 is greater than N 11 ;

[0095] and / or,

[0096] When the deviation of at least two consecutive target detection points from the corresponding EMA baseline is greater than N1 times the target standard deviation, it is determined that the front-end page is abnormal.

[0097] and / or,

[0098] When it is determined that the front-end page is abnormal, an alarm notification is sent.

[0099] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the front-end abnormality monitoring method based on the EMA algorithm when executing the computer program.

[0100] The application further provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the front-end abnormality monitoring method based on the EMA algorithm.

[0101] The positive progress effect of the application is that the EMA baseline and the standard deviation are combined to determine whether the detection point is abnormal, which reduces the probability of false positives, thereby reducing the manual cost of development and maintenance personnel, improving the accuracy of the alarm, and improving the probability that the developer can learn about the front-end abnormality in time, thereby adjusting and correcting the abnormality occurring in the front-end in time, improving the experience of customers, and reducing the existence of abnormality. BRIEF DESCRIPTION OF DRAWINGS

[0102] Figure 1 This is a flowchart of the front-end anomaly monitoring method based on the EMA algorithm in Embodiment 1 of the present invention.

[0103] Figure 2 This is a schematic diagram of the front-end anomaly monitoring system based on the EMA algorithm according to Embodiment 3 of the present invention.

[0104] Figure 3 This is a schematic diagram of the electronic device according to Embodiment 5 of the present invention. Detailed Implementation

[0105] The present invention will be further illustrated by way of embodiments below, but the present invention is not limited to the scope of the embodiments described herein.

[0106] Example 1

[0107] This embodiment provides a front-end anomaly monitoring method based on the EMA algorithm. (Refer to...) Figure 1 The front-end anomaly monitoring method based on the EMA algorithm includes the following steps:

[0108] S11. Receive exception information from the front-end page. The exception information is sent by the event tracking script in the front-end page of the target detection system. When the front-end page generates exception information, the event tracking script can collect it. The configuration information of the event tracking script may include: the page ID in the front-end page of the target detection system, the exception type of the exception information, the URL corresponding to the requested resource, and the URL corresponding to the interface. Developers can also define custom exceptions, which can be reported by triggering the event tracking script.

[0109] S12. Arrange the exception information in chronological order to obtain the data source. Push the exception information to the message queue Kafka, consume the exception information in the message queue and store it in the InfluxDB database to obtain the data source. The exception information in the database is arranged in chronological order.

[0110] S13. The number of abnormal information points within a preset time period is used as detection points according to time sequence, and arranged to obtain a historical detection sequence. The preset time period can be one day, one hour, etc. Detection points include historical detection points and target detection points. Historical detection points are those that are earlier than the target detection points in the time sequence. The historical detection points are arranged according to time sequence to obtain the historical detection sequence.

[0111] S14. Calculate the standard deviation of the historical detection sequence corresponding to the target detection point to obtain the target standard deviation.

[0112] S15. Calculate the EMA baseline for historical monitoring points. The EMA baseline is a reference value, and the formula for calculating the EMA baseline is as follows:

[0113] wherein i is the order of the detection point,

[0114] S16, determining that the front-end page is abnormal when the deviation of the at least one target detection point from the corresponding EMA baseline is greater than N times the target standard deviation. The target detection point x i The corresponding EMA baseline is bl i-1 , and N1 is determined according to the needs of the developer.

[0115] The front-end page abnormality determined in this way is different from the front-end page abnormality information collected by the burying point in that: a certain number of regular abnormalities of the front-end page in use is a normal phenomenon, but when the number of abnormalities of the front-end page exceeds the normal value and reaches a certain degree, it means that the front-end page cannot meet the use requirements and needs to be adjusted and maintained by the developer. The embodiment uses the EMA baseline to determine the regular value of the abnormality of the front-end page, and uses N1 times the standard deviation to determine the acceptable degree between the number of abnormalities and the regular value. When the deviation of a certain number of target detection points from the corresponding EMA baseline is greater than N1 times the target standard deviation, it is determined that the front-end page is abnormal, so that the developer can timely maintain the front-end page of the target detection.

[0116] Embodiment 2

[0117] On the basis of embodiment 1, the embodiment provides a front-end abnormality monitoring method based on an EMA algorithm.

[0118] S14 includes:

[0119] S141, initially screening abnormal points in the historical detection sequence.

[0120] The abnormal historical detection points in the historical detection sequence are screened by low-pass filtering to obtain a first abnormal value sequence.

[0121] The abnormal historical detection points in the historical detection sequence that are less than Q1-1.5*IQR and greater than Q3+1.5*IQR are screened by a box plot IQR to obtain a second abnormal value sequence, Q1 is the upper quartile value in the historical detection sequence, Q3 is the lower quartile value in the historical detection sequence, and IQR=Q3-Q1.

[0122] S142, calculating the standard deviation of the abnormal value sequence.

[0123] The standard deviation of the first abnormal value sequence is calculated to obtain a first standard deviation std err .

[0124] The standard deviation of the second abnormal value sequence is calculated to obtain a second standard deviation std box_err .

[0125] S143, calculating an average abnormal standard deviation based on the first standard deviation and the second standard deviation, the average abnormal standard deviation square_std being calculated according to the following formula:

[0126]

[0127] S144, discarding abnormal historical detection points in the historical detection sequence based on the average abnormal standard deviation.

[0128] Taking the union of the first abnormal value sequence and the second abnormal value sequence to obtain an abnormal value sequence, and selecting N2xsquare_std abnormal historical detection points from the abnormal value sequence as discarded points, where N2 can be set to 16.

[0129] Filtering out the discarded points in the historical detection sequence to obtain a new historical detection sequence;

[0130] S145, taking the new historical detection sequence as the historical detection sequence and repeating S141-144 several times until the developer considers that enough abnormal historical detection points have been discarded in the new historical detection sequence.

[0131] Calculating the standard deviation of the new historical detection sequence obtained after repeating several times.

[0132] S146, taking day as the first preset time period, taking the number of abnormal information in the first preset time period as the first detection point in time sequence, the first detection point including the first historical detection point and the first target detection point, arranging the first historical detection point in time sequence to obtain a first historical detection sequence, and calculating the standard deviation std of the first historical detection sequence according to S141-S145. 1st .

[0133] S147, taking day as the second preset time period, taking the number of abnormal information in the second preset time period as the second detection point in time sequence, the second detection point including the second historical detection point and the second target detection point, arranging the second historical detection point in time sequence to obtain a second historical detection sequence, and calculating the standard deviation std of the second historical detection sequence according to S141-S145. 2nd .

[0134] S148, predicting the target standard deviation std based on the first target standard deviation and the second target standard deviation, the target standard deviation std being obtained according to the following formula:

[0135]

[0136] S15 includes:

[0137] S151, judge whether the historical detection sequence is a periodic sequence, i.e. whether the historical detection points in the historical detection sequence satisfy a periodic rule, if the historical detection sequence is a non-periodic sequence, calculate the EMA baseline value bl according to S153 i , if the historical detection sequence is a periodic sequence, calculate the EMA baseline value bl according to S154 i .

[0138] S152, check whether the historical detection point x i is an outlier, the specific calculation method is as follows:

[0139]

[0140] Wherein, is_outlier represents the detection result of the historical detection point, true represents that the historical detection point being checked is a normal point, and false represents that the historical detection point being checked is an outlier.

[0141] S153, if the historical detection point x i is an outlier, the EMA baseline bl i = bl i-1 ;

[0142] If the historical detection point x i is a normal point, the EMA baseline value bl i is calculated according to the following formula:

[0143]

[0144] S154, calculate the difference between the historical x i and the baseline according to the historical detection sequence Here, the window width w = 5, the historical data day t = 7, 64 time points are taken every day, and the data is as follows:

[0145]

[0146] And the data is time-sequentially symmetrical to expand the data and improve the accuracy of the results. For example:

[0147] Change the window time sequence: [bl1, bl2, …, bl 20 , x1, x2, …, x 64 ] to:

[0148] [bl9, bl8, …, bl1] + [bl1, bl2, …, bl 20 , x1, x2, … x 64 ] + [x 64 , x 63 , …, x 56 ]

[0149] The specific calculation formula is as follows:

[0150]

[0151] Calculate historical detection points x i The difference between the current value and the baseline is calculated using the following formula:

[0152]

[0153] If historical detection point x i Normal point or and If the absolute value of the difference is less than a preset threshold, the EMA baseline value bl is calculated according to the following formula. i :

[0154]

[0155] If the current detection point is abnormal and there is no Then the EMA baseline bl i =2*bl i-1 -bl i-2 ;

[0156] If the current detection point is abnormal and exists Then EMA baseline

[0157] S16 includes:

[0158] S161. Calculate the entropy of the historical detection sequence: entropy = -∑p*log(p), where p is the probability of any value existing in the historical detection sequence;

[0159] When the entropy is less than the preset entropy threshold, the value of N1 can be set to 8. When the entropy is greater than the preset entropy threshold, the value of N1 can be set to 14 or 16. That is, the greater the difference between the entropy and the preset entropy threshold, the greater the value of N1 should be.

[0160] S162. When the deviation between three consecutive target detection points and the corresponding EMA baseline is greater than N1 times the target standard deviation, it is determined that the front-end page has an anomaly.

[0161] S163. When an anomaly is detected on the front-end page, an alert notification is sent to the developer. The notification includes an analysis of the cause of the anomaly. The analysis is based on the time point when the front-end page anomaly occurred. If the developer performed certain operations near the time point of the anomaly, the analysis will identify these operations as possible causes of the page anomaly.

[0162] The embodiment solves the standard deviation by combining low-pass filtering, IQR, and dividing different preset time periods, improves the accuracy of the standard deviation, solves the EMA baseline by dividing the historical detection sequence into periodic sequences and aperiodic sequences, improves the accuracy of the baseline, sets the N1 value according to the entropy, makes the setting of the N1 value more scientific, and accurately feeds back the anomaly, so that the anomaly can be discovered and processed by the developer in time. When the deviation of three consecutive target detection points and the corresponding EMA baseline is greater than N1 times the target standard deviation, it is determined that the front-end page has an anomaly, which can greatly reduce the false alarm situation, reduce errors, improve work efficiency, and save the time of the developer.

[0163] Embodiment 3

[0164] The embodiment provides a front-end anomaly monitoring system based on an EMA algorithm, referring to Figure 2 The front-end anomaly monitoring system based on the EMA algorithm includes an anomaly information receiving module 21, a data storage module 22, a detection point division module 23, a target standard deviation prediction module 24, an EMA baseline calculation module 25, and an anomaly determination module 26.

[0165] The anomaly information receiving module 21 is configured to receive anomaly information sent by a front-end page. The anomaly information is sent by a buried point collection script in the target detection front-end page, and the buried point collection sample can collect anomaly information when the front-end page generates anomaly information. The configuration information of the buried point collection script can include: an identifier page ID in the target detection front-end page, an anomaly type to which the anomaly information belongs, a url corresponding to a requested resource, a url corresponding to an interface, and a developer-defined anomaly reported by triggering the buried point collection script.

[0166] The data storage module 22 is configured to arrange the anomaly information in time sequence to obtain a data source. The anomaly information is pushed into a message queue Kafka, the anomaly information in the message queue is consumed and stored into an InfluxDB database to obtain a data source, and the anomaly information in the database is arranged in time sequence.

[0167] The detection point division module 23 is configured to arrange the number of anomaly information in a preset time period as a detection point in time sequence, and arrange to obtain a historical detection sequence. The preset time period can be one day, one hour, etc. The detection point includes a historical detection point and a target detection point, the historical detection point is an earlier detection point than the target detection point in the time sequence arrangement, and the historical detection point is arranged in time sequence to obtain a historical detection sequence.

[0168] The target standard deviation prediction module 24 is configured to calculate the standard deviation of the historical detection sequence corresponding to the target detection point to obtain a target standard deviation.

[0169] The EMA baseline calculation module 25 is configured to calculate the EMA baseline of the historical detection points. The EMA baseline is a reference value, and the calculation formula of the EMA baseline can be:

[0170] where i is the order of the detection point,

[0171] The anomaly determination module 26 is configured to determine that the front-end page is abnormal when at least one target detection point deviates from the corresponding EMA baseline by more than N times the target standard deviation. The target detection point x i corresponding EMA baseline is bl i-1 , and N1 is determined according to the needs of the developer.

[0172] The embodiment uses the EMA baseline to determine the normal value of the abnormality of the front-end page, uses N1 times the standard deviation to determine the acceptable degree between the number of abnormality and the normal value, and determines that the front-end page is abnormal when a certain number of target detection points deviate from the corresponding EMA baseline by more than N1 times the target standard deviation, so that the developer can maintain the front-end page detected in a timely manner.

[0173] Embodiment 4

[0174] On the basis of the embodiment 3, the embodiment provides a front-end abnormality monitoring system based on the EMA algorithm.

[0175] The target standard deviation prediction module 24 is further configured to: preliminarily screen out abnormal points in the historical detection sequence; calculate the standard deviation of the abnormal value sequence; calculate the average abnormal standard deviation based on the first standard deviation and the second standard deviation; discard the abnormal historical detection points in the historical detection sequence based on the average abnormal standard deviation; repeatedly use the foregoing functions and calculate the standard deviation of the new historical detection sequence; take day as the first preset time period, and calculate the standard deviation std 1st of the first historical detection sequence according to the foregoing functions; take hour as the second preset time period, and calculate the standard deviation std 2nd of the second historical detection sequence according to the foregoing functions; and predict the target standard deviation std based on the first target standard deviation and the second target standard deviation.

[0176] The EMA baseline calculation module is further configured to 25: determine whether the historical detection sequence is a periodic sequence; check whether the historical detection point x i is an abnormal point; calculate the EMA baseline of the non-periodic historical detection sequence; and calculate the EMA baseline of the periodic historical detection sequence.

[0177] The anomaly determination module 26 is further configured to: set the N1 value; determine that the front-end page is abnormal; and send an alarm notification.

[0178] In implementation, the abnormal information receiving module 21 is used to receive the abnormal information sent by the front-end page. The abnormal information is sent by the embedded collection script in the target detection front-end page. When the front-end page generates abnormal information, the embedded collection sample can collect the abnormal information.

[0179] The data storage module 22 is used to push the abnormal information into the message queue Kafka, consume the abnormal information in the message queue, and store it into the InfluxDB database to obtain the data source. The abnormal information in the database is arranged in time sequence.

[0180] The detection point division module 23 is used to arrange the number of abnormal information in a preset time period as a detection point according to time sequence. The preset time period can be one day, one hour, etc. The detection point includes a historical detection point and a target detection point. The historical detection point is a detection point earlier than the target detection point in the time sequence arrangement. The historical detection points are arranged in time sequence to obtain a historical detection sequence.

[0181] The target standard deviation prediction module 24 is used to preliminarily screen out abnormal points in the historical detection sequence. The abnormal historical detection points in the historical detection sequence are screened out by low-pass filtering to obtain a first abnormal value sequence; the abnormal historical detection points in the historical detection sequence that are less than Q1-1.5*IQR and more than Q3+1.5*IQR are screened out by the box plot IQR to obtain a second abnormal value sequence, Q1 is the upper quartile value in the historical detection sequence, Q3 is the lower quartile value in the historical detection sequence, and IQR=Q3-Q1.

[0182] The standard deviation of the abnormal value sequence is calculated. The standard deviation of the first abnormal value sequence is calculated to obtain a first standard deviation std err ; the standard deviation of the second abnormal value sequence is calculated to obtain a second standard deviation std box_err .

[0183] The average abnormal standard deviation is calculated based on the first standard deviation and the second standard deviation. The average abnormal standard deviation square_std is calculated according to the following formula:

[0184]

[0185] The abnormal historical detection points in the historical detection sequence are discarded based on the average abnormal standard deviation. The union of the first abnormal value sequence and the second abnormal value sequence is taken to obtain an abnormal value sequence. N2×square_std abnormal historical detection points are selected from the abnormal value sequence as discarded points, where N2 can be set to 16. The discarded points in the historical detection sequence are filtered out to obtain a new historical detection sequence.

[0186] Repeat the aforementioned function and calculate the standard deviation of the new historical detection sequence. Use the new historical detection sequence as the historical detection sequence and repeat the aforementioned function several times in sequence until the developer believes that enough outlier historical detection points have been discarded in the new historical detection sequence. Calculate the standard deviation of the new historical detection sequence obtained after several repetitions.

[0187] Using days as the first preset time period, the number of abnormal information within the first preset time period is taken as the first detection point according to the time sequence. The first detection point includes the first historical detection point and the first target detection point. The first historical detection points are arranged in time sequence to obtain the first historical detection sequence. The standard deviation (std) of the first historical detection sequence is calculated using the aforementioned function. 1st .

[0188] Using hours as the second preset time period, the number of abnormal information within the second preset time period is used as the second detection point according to the time sequence. The second detection point includes the second historical detection point and the second target detection point. The second historical detection points are arranged in time sequence to obtain the second historical detection sequence. The standard deviation (std) of the second historical detection sequence is calculated using the aforementioned function. 2nd .

[0189] The target standard deviation (std) is predicted based on the first and second target standard deviations. The target standard deviation (std) is obtained using the following formula:

[0190]

[0191] Using EMA baseline calculation module 25: Determine whether the historical detection sequence is a periodic sequence. That is, whether the historical detection points in the historical detection sequence satisfy a periodic pattern.

[0192] Verify historical detection points x i Is it an outlier? The specific calculation method is as follows:

[0193]

[0194] Here, is_outlier represents the detection result of historical detection points, true means that the historical detection points being verified are normal points, and false means that the historical detection points being verified are abnormal points.

[0195] Calculate the EMA baseline for the non-periodic historical detection sequence. If the historical detection point x... i If it is an outlier, then the EMA baseline bl i =bl i-1 ;

[0196] If historical detection point x i If the baseline is normal, the EMA baseline value bl is calculated according to the following formula. i :

[0197]

[0198] EMA baseline of periodic historical detection sequence is calculated. Historical x i is calculated based on historical detection sequence Here, window width w=5, historical data day t=7, 64 time points are taken every day, and the data is as follows:

[0199]

[0200] The data is symmetrical in time sequence to expand the data and improve the accuracy of the results. For example:

[0201] The window time sequence: [bl1, bl2, …, bl 20 , x1, x2, …, x 64 ] is changed to:

[0202] [bl9, bl8, …, bl1]+[bl1, bl2, …, bl 20 , x1, x2, …, x 64 ]+[x 64 , x 63 , …, x 56 ]

[0203] The specific calculation formula is as follows:

[0204]

[0205] The difference between the current detection point x i and the baseline is calculated, and the specific formula is as follows:

[0206]

[0207] If the historical detection point x i is a normal point or the absolute value of the difference between is less than a preset threshold, then the EMA baseline value bl i is calculated according to the following formula:

[0208]

[0209] If the current detection point is abnormal and there is , then the EMA baseline bl i = 2*bl i-1 -bl i-2 ;

[0210] If the current detection point is abnormal and there is , then the EMA baseline

[0211] Using the anomaly determination module 26: setting the value of N1. Calculate the entropy of the historical detection sequence entropy = -∑p*log(p), where p is the probability of any value existing in the historical detection sequence; when the entropy is less than the preset entropy threshold, the value of N1 can be set to 8, and when the entropy is greater than the preset entropy threshold, the value of N1 can be set to 14 or 16, that is, the greater the difference between the entropy and the preset entropy threshold, the greater the value of N1 should be set.

[0212] Determine that the front-end page has an anomaly. When the deviation of three consecutive target detection points from the corresponding EMA baseline is greater than N times the target standard deviation, it is determined that the front-end page has an anomaly.

[0213] When the deviation of three consecutive target detection points from the corresponding EMA baseline is greater than N1 times the target standard deviation, it is determined that the front-end page has an anomaly.

[0214] When it is determined that the front-end page has an anomaly, an alarm notification is sent to the developer, and the notification is accompanied by an analysis of the cause of the anomaly. The analysis is based on the time node at which the front-end page has an anomaly, and if the developer performs some operation near the anomaly time node, the operation performed will be analyzed as a possible cause of the page anomaly.

[0215] The embodiment improves the accuracy of the standard deviation by combining low-pass filtering, IQR, and dividing different preset time periods to solve the standard deviation. By dividing the historical detection sequence into periodic and non-periodic sequences to solve the EMA baseline, the accuracy of the baseline is improved. By setting the value of N1 according to the entropy, the setting of N1 is more scientific, and the anomaly can be accurately fed back, so that the anomaly can be discovered and processed by the developer in time. By determining that the front-end page has an anomaly when the deviation of three consecutive target detection points from the corresponding EMA baseline is greater than N1 times the target standard deviation, the false alarm situation can be greatly reduced, the error can be reduced, the work efficiency can be improved, and the developer's time can be saved.

[0216] Embodiment 5

[0217] Figure 3 A structural schematic diagram of an electronic device is provided for the embodiment. The electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the EMA algorithm-based front-end anomaly monitoring method of embodiment 1 or embodiment 2 when executing the program. Figure 3 The displayed electronic device 30 is only an example and should not limit the function and use range of the embodiments of the present application.

[0218] As Figure 3As shown, the electronic device 30 can be in the form of a general computing device, for example, it can be a server device. The components of the electronic device 30 can include, but are not limited to, the at least one processor 31 described above, the at least one memory 32 described above, and a bus 33 that connects the different system components, including the memory 32 and the processor 31.

[0219] The bus 33 includes a data bus, an address bus, and a control bus.

[0220] The memory 32 can include volatile memory, such as random access memory (RAM) 321 and / or cache memory 322, and can further include non-volatile memory, such as read-only memory (ROM) 323.

[0221] The memory 32 can also include a program / utility 325 having a set (at least one) of program modules 324, including but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which can include implementation of a network environment, individually or in some combination.

[0222] The processor 31, through the operating system, executes the various functional applications and data processing, such as the EMA algorithm-based front-end anomaly monitoring method of Embodiment 1 or Embodiment 2, by running the computer programs stored in the memory 32.

[0223] The electronic device 30 can also communicate with one or more external devices 34, such as a keyboard or a pointing device, through an input / output (I / O) interface 35. Further, the model generation device 30 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet, through a network adapter 36. As depicted, the network adapter 36 communicates with the other modules of the model generation device 30 through the bus 33. It should be appreciated that although not shown, other hardware and / or software modules could be used in conjunction with the model generation device 30, including but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0224] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the foregoing detailed description, such a division is merely exemplary and not mandatory. Indeed, according to the embodiments of the present application, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into a plurality of units / modules.

[0225] Embodiment 6

[0226] The embodiment provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement steps of the front-end exception monitoring method based on the EMA algorithm in the embodiment 1 or the embodiment 2.

[0227] More specifically, the readable storage medium can include, but is not limited to, a portable disc, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0228] In possible implementation manners, the present application can also be implemented in the form of a program product, which includes program codes for causing a terminal device to execute steps of the front-end exception monitoring method based on the EMA algorithm in the embodiment 1 or the embodiment 2 when the program product is run on the terminal device.

[0229] The program codes for executing the present application can be written in any combination of one or more programming languages, and can be executed completely on a user device, partially on a user device, as a separate software package, partially on a user device and partially on a remote device, or completely on a remote device.

[0230] Although the specific implementation manners of the present application are described above, those skilled in the art should understand that this is only an example, and the protection scope of the present application is defined by the appended claims. Those skilled in the art can make various changes or modifications to these implementation manners without departing from the principles and essence of the present application, and these changes and modifications all fall within the protection scope of the present application.

Claims

1. A front-end exception monitoring method based on an EMA algorithm, characterized in that, The method comprises the following steps: receiving the exception information sent by the front-end page; collecting the exception information in time sequence to obtain a data source; taking the number of exception information in a preset time period as a detection point in time sequence, the detection point comprising a historical detection point and a target detection point, and arranging the historical detection points in time sequence to obtain a historical detection sequence; predicting a target standard deviation corresponding to the target detection point based on the historical detection sequence; calculating an EMA baseline of the historical detection point; When the deviation of at least one of the target detection points from the corresponding EMA baseline is greater than a multiple of the target standard deviation, it is determined that the front-end page is abnormal. The step of predicting the target standard deviation corresponding to the target detection point based on the historical detection sequence comprises: The abnormal historical detection points in the historical detection sequence are screened through low-pass filtering to obtain a first abnormal value sequence, and a standard deviation of the first abnormal value sequence is calculated to obtain a first standard deviation ; abnormal historical detection points in the historical detection sequence through box plot IQR screening to obtain a second abnormal value sequence, calculating the standard deviation of the second abnormal value sequence to obtain a second standard deviation ; calculating an average anomaly standard deviation based on the first standard deviation and the second standard deviation, the average anomaly standard deviation being calculated according to the following formula : An outlier sequence is obtained based on the first outlier sequence and the second outlier sequence, and an outlier sequence is selected from the outlier sequence. Anomaly historical detection points are used as discard points; filtering out discarded points in the historical detection sequence to obtain a new historical detection sequence; repeating the above steps several times by taking the new historical detection sequence as the historical detection sequence, and performing standard deviation calculation on the new historical detection sequence obtained after repeating for several times to obtain the target standard deviation.

2. The EMA algorithm based front-end anomaly monitoring method of claim 1, wherein, The step of predicting the target standard deviation corresponding to the target detection point based on the historical detection sequence comprises: taking the number of exception information in a first preset time period as a first detection point in time sequence, the first detection point comprising a first historical detection point and a first target detection point; arranging the first historical detection points in time sequence to obtain a first historical detection sequence, and calculating a standard deviation of the first historical detection sequence to obtain a first target standard deviation ; taking the number of exception information in a second preset time period as a second detection point in time sequence, the second detection point comprising a second historical detection point and a second target detection point; arranging the second historical detection points in time sequence to obtain a second historical detection sequence, and calculating a standard deviation of the second historical detection sequence to obtain a second target standard deviation ; predicting a target standard deviation based on the first target standard deviation and the second target standard deviation , the target standard deviation is obtained according to the following formula: 。 3. The EMA algorithm based front-end anomaly monitoring method of claim 1, wherein, The step of calculating the EMA baseline of the historical detection point comprises: Based on the historical detection points, the EMA baseline is calculated using the following formula : where i is the ordering of the historical detection point among the historical detection sequence.

4. The EMA algorithm based front-end anomaly monitoring method of claim 3, wherein, The step of calculating the EMA baseline of the historical detection point comprises: Check history detection point whether it is an abnormal point, and the specific calculation is as follows: ， wherein, represents a detection result of a historical detection point, represents that the historical detection point being checked is a normal point, represents that the historical detection point being checked is an abnormal point; If the historical detection point is an abnormal point, then the EMA baseline ; If the historical detection point is normal, then the EMA baseline value is calculated according to the following formula : 。 5. The EMA algorithm based front-end anomaly monitoring method of claim 3, wherein, The step of calculating the EMA baseline of the historical detection point comprises: Check history detection point whether it is an abnormal point, which is calculated as follows: ； calculating history based on the historical detection sequences difference from baseline , specifically as follows: where t is the number of days involved in the historical detection sequence, i is the ordering of the historical detection point in the historical detection sequence for a day, and w is the window width. Computing historical checkpoints The difference from baseline, specifically formulated as follows: ； If the historical detection point is a normal point or the absolute value of the difference between the current detection point and is less than a preset threshold, the EMA baseline value is calculated according to the following formula : ； If the current detection point is abnormal and there is no , the EMA baseline ; If the current detection point is abnormal and there is , then the EMA baseline .

6. The EMA algorithm based front-end anomaly monitoring method of claim 1, wherein, when the deviation of at least one of the target detection points from the corresponding EMA baseline is greater than a multiple of the target standard deviation, the step of determining that the front-end page is abnormal includes: calculating an entropy of the history of detections sequence wherein is a probability of existence of any value in the history of detections sequence; When the entropy is less than the preset entropy threshold, the value of the parameter is set as , When the entropy is greater than the preset entropy threshold, the value of the parameter is set as , , is greater than . and / or, When the deviation of at least two continuous target detection points from the corresponding EMA baseline is greater than times the target standard deviation, it is determined that the front-end page is abnormal. and / or, when it is determined that the front-end page has an exception, sending an alarm notification.

7. An EMA algorithm based front-end anomaly monitoring system, characterized in that, The method comprises the following steps: The method comprises the following steps: The exception information receiving unit is configured to receive the exception information sent by the front-end page; The data storage unit is configured to collect the exception information in time sequence to obtain a data source; The detection point division unit is configured to take the number of exception information in a preset time period as a detection point in time sequence, the detection point comprising a historical detection point and a target detection point, and arrange the historical detection points in time sequence to obtain a historical detection sequence; The target standard deviation prediction unit is configured to predict a target standard deviation corresponding to the target detection point based on the historical detection sequence; The abnormality determining unit is configured to determine that the front-end page is abnormal when the deviation of at least one of the target detection points from the corresponding EMA baseline is greater than a plurality of target standard deviations. The target standard deviation prediction unit is further configured to: filter out abnormal historical detection points in the historical detection sequence by low-pass filtering to obtain a first abnormal value sequence; calculate a standard deviation of the first abnormal value sequence to obtain a first standard deviation ; The target standard deviation prediction unit is further configured to screen out abnormal historical detection points in the historical detection sequence by a box plot IQR to obtain a second abnormal value sequence, calculate a standard deviation of the second abnormal value sequence to obtain a second standard deviation ; The target standard deviation prediction unit is further configured to calculate an average anomaly standard deviation based on the first standard deviation and the second standard deviation, the average anomaly standard deviation being calculated according to the following equation : The target standard deviation prediction unit is further configured to obtain an abnormal value sequence based on the first abnormal value sequence and the second abnormal value sequence, and select one abnormal history detection point from the abnormal value sequence as a discard point. The target standard deviation prediction unit is further configured to obtain an abnormal value sequence based on the first abnormal value sequence and the second abnormal value sequence, and select one abnormal history detection point from the abnormal value sequence as a discard point. The EMA baseline calculation unit is configured to calculate an EMA baseline of the historical detection point; The target standard deviation prediction unit is further configured to filter out discarded points in the historical detection sequence to obtain a new historical detection sequence; 8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The target standard deviation prediction unit is further configured to repeat the above steps several times by taking the new historical detection sequence as the historical detection sequence, and perform standard deviation calculation on the new historical detection sequence obtained after repeating for several times to obtain the target standard deviation.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The processor executes the computer program to realize the EMA algorithm-based front-end exception monitoring method in any one of claims 1-6. The computer program is executed by the processor to realize the EMA algorithm-based front-end exception monitoring method in any one of claims 1-6.

Images