An edge service anomaly detection method based on concept drift

By combining a randomly paired reservoir sampling algorithm with the Distribution Change Metric (DCM), the problem of uniformity and representativeness of sampled data in edge services is solved, enabling more accurate detection of concept drift anomalies and improving the accuracy and sensitivity of detection.

CN119989231BActive Publication Date: 2025-11-04INNER MONGOLIA UNIVERSITY
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510150250.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-02-11
Publication Date
2025-11-04
Estimated Expiration
2045-02-11

AI Technical Summary

Technical Problem

In existing edge service concept drift anomaly detection methods, the sampled data cannot meet the requirements of uniformity and representativeness, and the measurement methods have room for improvement in detection sensitivity and accuracy.

Method used

By employing a reservoir sampling algorithm based on random pairing combined with the degree of difference, and by calculating the reliability score of the data stream and the distribution change index (DCM), representative extraction and anomaly detection of historical data are achieved.

Benefits of technology

Under storage cost constraints, the system effectively extracts features from a representative historical sample library, improving the accuracy and sensitivity of concept drift anomaly detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119989231B_ABST
    Figure CN119989231B_ABST
Patent Text Reader

Abstract

The application discloses an edge service anomaly detection method based on concept drift, comprising the following steps: S1, calculating the reliability score of a data stream to obtain a data stream sequence; S2, sampling the data stream sequence through a reservoir sampling method based on random pairing, and storing the sampling result in a sample library; S3, calculating the distribution change index of historical data in the sample library and current data in a sliding window; and S4, performing concept drift anomaly detection according to the distribution change index. The method effectively utilizes the sliding window, improves the traditional reservoir sampling algorithm, introduces the difference degree, realizes the extraction of representative historical sample library features from historical data under the constraint of storage cost, introduces the concept of random pairing to realize the extraction of uniform historical sample library features from historical data, and realizes more accurate concept drift anomaly detection by calculating the minimum cost required for the probability distribution conversion between current reliability data streams and historical reliability data streams in a service environment.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of edge service, and particularly relates to an edge service anomaly detection method based on concept drift. BACKGROUND

[0002] In order to cope with concept drift, researchers have proposed a variety of detection methods. These methods include statistical-based drift detection methods, prediction-based drift detection methods, sliding window-based drift detection methods, and clustering-based drift detection methods.

[0003] Statistical-based drift detection methods usually monitor the changes of data streams using statistical indicators such as mean, variance, or more complex statistical tests. Baidari et al. developed a novel concept drift detection mechanism that relies on the Bhattacharyya distance, supplemented by the calculation of mean and standard deviation to identify gradual or sudden changes in data distribution. The essence of this method is to measure the center position and dispersion of consecutive data blocks and compare them with the data of the historical reference window to assess the degree of change in data distribution. When the observed changes in mean and standard deviation exceed the established limits, the system will diagnose the occurrence of concept drift. Statistical-based drift detection methods usually rely on simple statistics such as mean and variance, so they are not sensitive enough to changes in data distribution, especially when the data has nonlinear or complex patterns. In addition, such methods can be very sensitive to noise, leading to false drift detection. Prediction-based drift detection methods focus on training a prediction model using historical data and monitoring changes in model prediction accuracy. When prediction performance declines, it indicates that the data stream has concept drift. Yang designed an integrated prediction framework called Performance Weighted Probability Averaging Ensemble (PWPAE) to predict and identify concept drift. This framework integrates multiple anomaly prediction models and adjusts the performance weight and prediction probability of each model to adaptively adjust the drift. This method relies on the performance of the prediction model, and if the model itself has overfitting or insufficient conditions (such as insufficient training data, improper feature selection, etc.), it may lead to inaccurate drift detection. In addition, this method usually requires a long history of data, so it may have a lag when the data flow is large, affecting timeliness. Sliding window-based drift detection methods use sliding windows on data streams to capture the latest data trends and compare the statistical properties of the data within the window with the statistical properties of historical data to detect distribution shifts. Liu developed a novel concept drift detection framework called Nearest Neighbor Density Variation Identification (NN-DVI) for drift detection tasks under the sliding window mechanism. NN-DVI uses a k-nearest neighbor-based spatial partitioning pattern (NNPS) to convert discrete and difficult-to-directly-measure data instances into a set of shared subspaces, then accumulates the density differences in these subspaces and quantifies the overall difference to determine the concept drift confidence interval. Sliding window methods require defining window size when processing data streams, and a large window size may not be able to capture rapid changes in drift in a timely manner, while a small window size may result in insufficient information and increase the inaccuracy of detection. In addition, sliding window methods usually assume that the data distribution is locally stable, which may not be true in practical applications. Clustering-based drift detection methods analyze the internal structural changes of data by clustering data points, and when the clustering structure changes significantly, it indicates that concept drift has occurred. Jain et al. proposed a concept drift detection method that combines error rate and data distribution information.Their approach is based on analyzing drift using a sliding window of captured data and combined with a K-Means clustering algorithm, aiming to reduce the data size and update the training dataset. Additionally, they employ a Support Vector Machine (SVM) classifier for anomaly detection and trigger model retraining upon confirming the occurrence of drift. The clustering method relies on the selection and parameter tuning of the clustering algorithm, such as the selection of K in K-Means, and different clustering methods can have significant differences in their adaptability to data, leading to unstable results. Especially for high-dimensional data, the effectiveness of clustering may not be ideal, resulting in inaccurate detected drift information. Furthermore, the computational complexity of clustering algorithms is high, and they may face performance bottlenecks when dealing with large-scale data.

[0004] In edge computing environments, the problem of concept drift is particularly pronounced. Although research in this area is still relatively limited, some researchers have begun to explore this challenge. Wang et al. developed an advanced detection mechanism called A-Detection, which integrates reservoir sampling and singular value decomposition (SVD) techniques for efficient sampling and feature extraction of large-scale data streams. They adopted the Jensen-Shannon (JS) divergence to create a new metric, Fractional Distribution Change (FDC), to measure the inconsistency of data stream distribution. FDC provides a new tool for real-time monitoring of edge service operational anomalies by quantifying the distribution changes of edge service reliability data streams. Although A-Detection demonstrates its advancement in detecting concept drift, it does not consider the representativeness of samples in the design of the sampling algorithm, which to some extent affects the accuracy of the detection mechanism, and the FDC metric cannot accurately detect anomalies under certain drift patterns. Wang et al. further proposed two methods, B-Detection and L-Detection. B-Detection uses LSTM auto-encoders for anomaly detection of MEC service reliability data by learning the distribution characteristics of normal data and comparing reconstruction errors to identify data points significantly different from normal patterns. B-Detection also incorporates weighted reservoir sampling and Boosting strategies to adapt to the dynamic changes in data stream distribution in MEC environments. L-Detection combines sliding window technology and local sensitive hashing (LSH)-based sampling methods to extract features from recent and historical data, and uses JS divergence to evaluate the degree of reliability anomaly in edge service runtime. Both B-Detection and L-Detection methods consider the representativeness of the sampling algorithm, but their improvements are mainly achieved through weight allocation. Although this method improves the representativeness of the sampled data to some extent, weight allocation is very sensitive to the selection of initial weights, which is difficult to achieve without sufficient prior knowledge. SUMMARY

[0005] To overcome the above deficiencies in the prior art, the edge service anomaly detection method based on concept drift provided by the present application solves the problem that the sampled data of the existing edge service concept drift anomaly detection method cannot meet the uniformity and representativeness, and the existing measurement method also has room for improvement in detection sensitivity and accuracy. The method realizes the extraction of representative historical sample library features from historical data under the constraint of storage cost, and more accurate edge service concept drift anomaly detection.

[0006] In order to achieve the above object, the technical scheme adopted by the present application is as follows: a concept drift-based edge service anomaly detection method, comprising the following steps:

[0007] S1, calculating the reliability score of the data stream to obtain a data stream sequence;

[0008] S2, sampling the data stream sequence through a reservoir sampling method based on random pairing, and storing the sampling result in a sample library;

[0009] S3, calculating the distribution change index of the historical data in the sample library and the current data in the sliding window;

[0010] S4, performing concept drift anomaly detection according to the distribution change index.

[0011] Further, in S1, the reliability score of the data stream is specifically:

[0012] The probability r of the edge server processing user requests to have an error response, partial response failure or timeout Δt , and the expression is specifically:

[0013] r Δt =e - λ ×len(Δt)

[0014] In the formula, Δt is a fixed time period, len(.) is the length of the fixed time period interval Δt, λ is the failure rate of the edge service within the maximum response time, and the expression is specifically:

[0015] λ=1-[1-f(Δt)] n

[0016] In the formula, n is the number of call requests, f(Δt) is the failure response probability of the edge server within a limited time, and the expression is specifically:

[0017] f(Δt)=L0 / n

[0018] In the formula, L0 is the number of failed calls in n call requests.

[0019] Further, S2 comprises the following steps:

[0020] S21, initializing a deletion counter, a prohibition counter and a total exclusion counter;

[0021] S22, sampling the elements in the data stream sequence in turn through a sliding window, and calculating the difference degree of the sampled elements and the first sequence;

[0022] S23, judging whether the calculated difference degree is greater than the forbidden threshold value, if yes, allowing the element to enter the sample library, entering S24; if no, prohibiting the element from entering the sample library, entering S26;

[0023] S24, judging whether the total exclusion counter is 0, if yes, performing the normal reservoir sampling method, entering S27; if no, entering S25;

[0024] S25, generating a random decimal number through the random pairing idea, calculating the probability of the new data joining the sample library according to the values of the deletion counter and the forbidden counter, judging whether the probability is greater than the random decimal number; if yes, adding the element to the sample library and reducing the value of the deletion counter by 1; if no, prohibiting the element from entering the sample library and reducing the value of the forbidden counter by 1;

[0025] entering S27;

[0026] S26, prohibiting the sampled element from entering the sample library, adding the value of the forbidden counter by 1, traversing the sample library to calculate the Euclidean distance between the sampled element and each element in the sample library, if there is an element in the sample library whose Euclidean distance is less than the deletion threshold value, deleting the element whose Euclidean distance is less than the deletion threshold value from the library and adding the value of the deletion counter by 1, entering S27;

[0027] S27, judging whether all elements in the data stream sequence are sampled, if yes, obtaining the final sample library, if no, returning to S21 to sample the next element in the data stream sequence.

[0028] Further, in S21, the deletion counter records the number of deleted elements in the sample, the forbidden counter records the number of forbidden sampling elements in the window, and the total exclusion counter records the total exclusion number, which is the sum of the values of the deletion counter and the forbidden counter.

[0029] Further, in S22, for the element D[i] at position i in the sampled data stream sequence D, the expression of the difference degree z of D[i] and the first sequence D[i+1:i-1+W] is specifically:

[0030] z = Z(D[i], D[i+1:i-1+W])

[0031] In the formula, W is the size of the sliding window, and Z(.) is the Z-Score formula.

[0032] Further, in S25, the expression of the probability P1 of the new data joining the sample library is specifically:

[0033] P1 = C1 / (C1+C2)

[0034] In the formula, C1 is the value of the deletion counter, and C2 is the value of the forbidden counter.

[0035] Further, the expression of the distribution change indicator DCM in S3 is specifically:

[0036]

[0037] In the formula, D history is historical data in the sample library, D current is current data in the sliding window, P current is the probability distribution of the current data, P history is the probability distribution of the historical data, and γ is a set of all possible joint distributions. history , current is a set of joint distributions γ that satisfy the edge distribution conditions of P history and P current , (x, y) represents a sample pair drawn from the joint distribution γ, wherein x comes from the probability distribution of the historical data, y comes from the probability distribution of the current data, ||x-y|| represents the distance between the sample pair (x, y), and Δ(·) is a difference measure between two distributions.

[0038] Further, S4 is specifically:

[0039] It is judged whether the distribution change indicator is greater than the abnormal threshold, if yes, an abnormality occurs, and if not, no abnormality occurs.

[0040] The present application has the following advantages: the present application provides an edge service anomaly detection method based on concept drift, combines the difference degree, the distribution change indicator based on the reservoir sampling algorithm of random pairing, solves the technical problem that the sampling data of the existing edge service concept drift anomaly detection method cannot meet the uniformity and representativeness, and also solves the problem that the existing measurement method also has room for improvement in detection sensitivity and accuracy, realizes the extraction of representative historical sample library features from historical data under the constraint of storage cost, and more accurate edge service concept drift anomaly detection, compared with the prior art, has the following effects:

[0041] (1) The present application effectively utilizes the sliding window and improves the traditional reservoir sampling algorithm, introduces the difference degree, realizes the extraction of representative historical sample library features from historical data under the constraint of storage cost.

[0042] (2) The present application introduces the concept of random pairing, realizes the extraction of uniform historical sample library features from historical data.

[0043] (3) The application innovatively proposes a data flow distribution change index DCM as an index for judging abnormalities in terms of metric values, and realizes more accurate concept drift anomaly detection by calculating the minimum cost required for probability distribution conversion between current reliability data flow and historical reliability data flow in a service environment. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 A concept drift-based edge service anomaly detection method flowchart of the application. DETAILED DESCRIPTION

[0045] The specific embodiments of the application are described below to facilitate understanding of the application by those skilled in the art, but it should be clear that the application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that various changes are within the spirit and scope of the application defined and determined by the appended claims, and all applications utilizing the concept of the application are within the scope of protection.

[0046] As shown in the figure, in one embodiment of the application, a concept drift-based edge service anomaly detection method comprises the following steps: Figure 1

[0047] S1, calculating the reliability score of the data flow to obtain a data flow sequence;

[0048] S2, sampling the data flow sequence by a reservoir sampling method based on random pairing, and storing the sampling result in a sample library;

[0049] S3, calculating the distribution change index of the historical data in the sample library and the current data in the sliding window;

[0050] S4, performing concept drift anomaly detection according to the distribution change index.

[0051] In this embodiment, the application adds the concept of difference degree on the basis of the reservoir sampling algorithm, uses the difference degree to judge whether the element is allowed to be sampled into the sample library, so as to ensure that representative data can be stored in the fixed-size sample library. In order to ensure that the uniformity of the sampling data can be met after adding the difference degree to the sampling method, the concept of random pairing is introduced, and after obtaining more uniform and more representative sampling data, a suitable distribution change index DCM is selected based on the bulldozer distance idea, to realize more accurate concept drift anomaly detection.

[0052] The application calculates the reliability score of the data flow, which is used for quantifying the reliability data flow. The application defines that under certain operating conditions and time limits, the probability that the edge server successfully processes user requests and avoids error responses, partial response failures or timeouts is represented by r Δt .​

[0053] The reliability score of the data stream in S1 is specifically:

[0054] The probability r of the edge server processing the user request to have an error response, partial response failure or timeout Δt , and the expression is specifically:

[0055] r Δt = e - λ ×len(Δt)

[0056] In the formula, Δt is a fixed time period, len(.) is the length of the fixed time period interval Δt, λ is the failure rate of the edge service within the maximum response time, and the expression is specifically:

[0057] λ = 1 - [1 - f(Δt)] n

[0058] In the formula, n is the number of call requests, f(Δt) is the failure response probability of the edge server within a limited time, and the expression is specifically:

[0059] f(Δt) = L0 / n

[0060] In the formula, L0 is the number of failed calls in n call requests.

[0061] In this embodiment, in the fixed time period Δt, one call request is sent to the edge server every fixed time, and a total of n call requests are performed, and each call request is regarded as a Bernoulli test, wherein only when the request gets a complete response within the longest time T max , the test is considered successful; otherwise, it is considered failed. Through n consecutive Bernoulli tests, a rate value can be obtained to estimate the ability of the edge server to perform the required function within the fixed time period Δt, and the rate value can be represented as [1-f(Δt)] n .

[0062] The S2 includes the following steps:

[0063] S21, initializing a deletion counter, a prohibition counter and a total exclusion counter;

[0064] S22, sampling the elements in the data stream sequence in order through a sliding window, and calculating the difference degree of the sampled elements and the first sequence;

[0065] S23, judging whether the calculated difference degree is greater than the prohibition threshold, if yes, allowing the element to enter the sample library, entering S24; if not, prohibiting the element from entering the sample library, entering S26;

[0066] S24, judging whether the total exclusion counter is 0, if yes, executing the normal reservoir sampling method, entering S27; if no, entering S25;

[0067] In the embodiment, the normal reservoir sampling algorithm is specifically: requiring initializing a sample reservoir with size k during the sliding of the sliding window. The first k data in the sliding window are directly stored in the sample reservoir, and for the i th (i>k) data, it is determined whether to replace the data in the reservoir according to the probability k / i. If yes, a data in the reservoir is randomly selected for replacement. In this way, the probability of each data point being selected is equal.

[0068] S25, generating a random decimal number through the random pairing idea, calculating the probability of the new data joining the sample reservoir according to the values of the deletion counter and the prohibition counter, judging whether the probability is greater than the random decimal number; if yes, the element is added to the sample reservoir, and the value of the deletion counter is reduced by 1; if no, the element is prohibited from entering the sample reservoir, and the value of the prohibition counter is reduced by 1;

[0069] entering S27;

[0070] S26, prohibiting the sampled element from entering the sample reservoir, adding 1 to the value of the prohibition counter, traversing the sample reservoir to calculate the Euclidean distance between the sampled element and each element in the sample reservoir, if there is an element with an Euclidean distance less than the deletion threshold in the sample reservoir, the element with the Euclidean distance less than the deletion threshold is deleted from the reservoir, and the value of the deletion counter is added by 1, entering S27;

[0071] S27, judging whether all elements in the data stream sequence are sampled, if yes, the final sample reservoir is obtained, if no, returning to S21 to sample the next element in the data stream sequence.

[0072] In S21, the deletion counter records the number of deleted elements in the sample, the prohibition counter records the number of prohibited sampling elements in the window, and the total exclusion counter records the total exclusion number, which is the sum of the values of the deletion counter and the prohibition counter.

[0073] In S22, for the element D[i] at position i in the sampled data stream sequence D, the difference degree z of D[i] and the first sequence D[i+1:i-1+W] is specifically expressed as:

[0074] z=Z(D[i],D[i+1:i-1+W])

[0075] In the formula, W is the size of the sliding window, and Z(.) is the Z-Score formula.

[0076] In the embodiment, the sliding window slides on the data stream sequence, the window size is W, and the sliding step is 1.

[0077] The expression of the probability P1 of the new data joining the sample library in the S25 is specifically:

[0078] P1=C1 / (C1+C2)

[0079] In the formula, C1 is the value of the deletion counter, C2 is the value of the prohibition counter, and the generated random number ranges from 0 to 1.

[0080] In the embodiment, the application introduces the concept of random pairing. Pairing refers to that element A forms a “partnership” with element B, and the operation of element A needs to depend on certain operations of element B, that is, certain operations of element B are the premise of certain operations of element A. Random pairing refers to that when the current data D[i] is allowed to be sampled into the sample library, the data D[i] randomly selects an element D[j] that is prohibited from sampling and has not been paired in the elements of the window D[0] to D[i-1] to form a pair, and the element D[j] is called a pairing item. If the pairing item D[j] is prohibited from sampling, the current data is added to the sample library; otherwise, the current data D[i] will not be added to the sample library even if it is allowed to be sampled.

[0081] The expression of the distribution change indicator DCM in the S3 is specifically:

[0082]

[0083] In the formula, D history is the historical data in the sample library, D current is the current data in the sliding window, P current is the probability distribution of the current data, P history is the probability distribution of the historical data, γ is a set of all possible joint distributions, which describe a transition from P history to P current , Π(P history , P current ) is a set of joint distributions γ that satisfy the conditions of the marginal distributions of P history and P current , (x, y) represents a sample pair drawn from the joint distribution γ, where x comes from the probability distribution of the historical data and y comes from the probability distribution of the current data, ||x-y|| represents the distance between the sample pair (x, y), the Euclidean distance is used, Δ(·) is the difference measure between two distributions, and ∫||x-y||dγ(x, y) represents the weighted sum of the distances between the sample pairs (x, y) under all possible joint distributions γ, which can be regarded as the “cost” required for the transition from P history to P current , that is, the transition from P history to P currentThe minimum value of this weighted sum among all possible joint distributions, that is, the minimum energy consumption under the optimal path planning, is the DCM value.

[0084] In the embodiment, the application defines a distribution change metric (DCM) by referring to the Wasserstein Distance. history The difference between the distribution of the current data D current in the sliding window and the distribution of the historical data D history is represented as Delta (D current ), and Delta represents the difference between the two distributions.

[0085] The S4 is specifically:

[0086] It is judged whether the distribution change metric is greater than the abnormal threshold value, if yes, an abnormality occurs, and if not, no abnormality occurs.

[0087] The application has the following advantages: the application provides an edge service anomaly detection method based on concept drift, combines the difference degree, the distribution change metric based on random pairing reservoir sampling algorithm, solves the technical problem that the sampling data of the existing edge service concept drift anomaly detection method cannot meet the uniformity and representativeness, also solves the problem that the existing measurement method has room for improvement in detection sensitivity and accuracy, realizes the extraction of representative historical sample library features from historical data under the constraint of storage cost, and more accurate edge service concept drift anomaly detection, and has the following effects compared with the prior art:

[0088] (1) The application effectively utilizes the sliding window, improves the traditional reservoir sampling algorithm, introduces the difference degree, and realizes the extraction of representative historical sample library features from historical data under the constraint of storage cost.

[0089] (2) The application introduces the concept of random pairing, realizes the extraction of uniform historical sample library features from historical data.

[0090] (3) In terms of measurement value, the application innovatively proposes a data stream distribution change metric DCM as an index for judging abnormality, calculates the minimum cost required for the probability distribution conversion between the current reliable data stream and the historical reliable data stream in the service environment, and realizes more accurate concept drift anomaly detection.

[0091] In the description of the application, it needs to be understood that the terms "center", "thickness", "upper", "lower", "horizontal", "top", "bottom", "inner", "outer", "radial" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the application. In addition, the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implying the number of technical features indicated. Therefore, the features defined by "first", "second", "third" can explicitly or implicitly include one or more of the features.

Claims

1.A concept drift based edge service anomaly detection method, characterized in that, The method comprises the following steps: S1, calculating the reliability score of the data stream to obtain a data stream sequence; S2, sampling the data stream sequence through a reservoir sampling method based on random pairing, and storing the sampling result in a sample library; S3, calculating a distribution change index of historical data in the sample library and current data in a sliding window; S4, detecting concept drift anomaly according to the distribution change index; The S2 comprises the following steps: S21, initializing a deletion counter, a forbidden counter and a total exclusion counter; S22, sampling the elements in the data stream sequence in sequence through the sliding window, and calculating the difference degree of the sampled elements and the first sequence; S23, judging whether the calculated difference degree is greater than the forbidden threshold, if yes, allowing the element to enter the sample library, and entering S24; If not, the element is forbidden to enter the sample library, and S26 is entered; S24, judging whether the total exclusion counter is 0, if yes, executing the normal reservoir sampling method, and entering S27; if not, S25 is entered; S25, generating a random decimal number through the random pairing idea, calculating the probability of the new data entering the sample library according to the values of the deletion counter and the forbidden counter, judging whether the probability is greater than the random decimal number; if yes, the element is added to the sample library, and the value of the deletion counter is reduced by 1; If not, the element is forbidden to enter the sample library, and the value of the forbidden counter is reduced by 1; S27 is entered; S26, the forbidden element is sampled to enter the sample library, the value of the forbidden counter is added by 1, the Euclidean distance between the sampled element and each element in the sample library is calculated, if there is an element in the sample library with an Euclidean distance less than the deletion threshold, the element with the Euclidean distance less than the deletion threshold is deleted from the library, and the value of the deletion counter is added by 1, and S27 is entered; S27, judging whether all elements in the data stream sequence are sampled, if yes, the final sample library is obtained, if not, the next element in the data stream sequence is sampled by returning to S21; In the S3, the distribution change index DCM The expression of the distribution change index is specifically: wherein D history is the historical data in the sample library, D current is the current data in the sliding window, P current is the probability distribution of the current data, P history is the probability distribution of the historical data, γ is the set of all possible joint distributions, P history , P current is the set of joint distributions γ satisfying P history and P current is the set of joint distributions γ satisfying the marginal distribution condition, (x, y) represents a sample pair drawn from the joint distribution γ, wherein x is the probability distribution from the historical data, y is the probability distribution from the current data, ||x-y|| represents the distance between the sample pair (x, y), is the difference measure between the two distributions, represents the weighted sum of the distances between the sample pairs under all possible joint distributions . 2.The concept drift based edge service anomaly detection method according to claim 1, characterized in that, In the S1, the reliability score of the data stream is specifically: The probability of an error response, partial response failure, or timeout occurring in the edge server processing a user request r Δt The expression is specifically: where ∆ t is the fixed time period, len (.) is the length of the fixed time period interval ∆ t , λ is the failure rate of the edge service within the maximum response time, which is expressed as: wherein n is the number of invocation requests, f (Δt) is the probability of failed response of the edge server in a limited time, which is expressed as: In the formula, L 0 is The number of call failures in the sub-call request. 3.The concept drift based edge service anomaly detection method of claim 1, wherein, In the S21, the deletion counter records the number of deleted elements in the sample, the forbidden counter records the number of forbidden sampling elements in the window, and the total exclusion counter records the total exclusion number, which is the sum of the values of the deletion counter and the forbidden counter. 4.The concept drift based edge service anomaly detection method of claim 1, wherein, In the S22, for the sampled data stream sequence D The middle position i The difference degree of the element D[i] and the first sequence D[i+1:i-1+W] z The expression is specifically: wherein W is the size of the sliding window, Z (.) is the Z-Score formula. 5.The concept-drift-based edge service anomaly detection method of claim 1, wherein, In the S25, the probability that new data is added to the sample library P The expression of the formula 1 is specifically: In the formula, C 1 is the value of the deletion counter, C 2 is the value of the prohibition counter. 6.The concept-drift-based edge service anomaly detection method of claim 1, wherein, The S4 is specifically: judging whether the distribution change index is greater than the anomaly threshold, if yes, an anomaly occurs, if not, no anomaly occurs.

Citation Information

Patent Citations

  • Concept drift detection method and system based on weighted sampling and electronic equipment

    CN113033643A

  • Cardinality estimation method and system based on dynamic sample recommendation

    CN117909363A