Abnormal value detection method based on two new abnormal factors

Abstract

The invention provides an abnormal value detection method based on two new abnormal factors, and belongs to the technical field of data processing and management. Comprising the steps of 1, constructing a de-duplication data set; 2, calculating a k neighborhood local centroid offset density factor of each point in the deduplication data set; 3, calculating a global feature fitting offset distance factor of each point in the deduplication data set; 4, fusing the two types of factors; and step 5, comparing the size of the fusion factor of each point in the deduplication data set, and identifying an abnormal value. According to the method, the local centroid offset density factor and the global feature fitting offset distance factor are designed, and the two types of factors are fused, so that accurate identification of data abnormal values with the characteristics that the number of data repetition times is large and fitting relations exist between data features is realized.

Description

Technical Field

[0001] The invention relates to an outlier detection method based on two new outlier factors, and belongs to the technical field of data processing and management. Background Art

[0002] Outliers are unexpected objects in a data group that have significantly different attribute characteristics from other objects. By detecting outliers, we can discover abnormal data activities, thereby improving data quality and providing a basis for further data mining or data processing.

[0003] Mainstream outlier detection methods can be divided into statistical-based methods, clustering-based methods, density-based methods, and distance-based methods. (1) Statistical-based methods assume that normal data follows a certain distribution, and use this as a benchmark to identify outliers by measuring the relationship between the data and the distribution model; (2) Clustering-based methods also assume that the data distribution is known and group similar types of data objects; (3) Density-based methods check the neighborhood density of each data object and measure the degree of abnormality of the data object by the density; (4) Distance-based methods calculate the distance between a data object and its neighbors, and identify objects that are too far away as outliers.

[0004] Through analysis and verification, the existing methods are suitable for outlier detection problems when the data is independently distributed and the influence of the number of data repetitions is ignored. However, it is difficult to achieve effective outlier detection when there are many repeated data points and fitting requirements between data features. Summary of the invention

[0005] The present invention aims to solve the problem that the existing outlier detection methods are difficult to achieve effective outlier detection when detecting data with many repeated data points and fitting requirements between data features. The existing methods are only applicable to data sets with independent data distribution and ignore the influence of data repetition times.

[0006] The present invention proposes an outlier detection method based on two new anomaly factors, designs a local centroid offset density factor and a global feature fitting offset distance factor, and fuses the two types of factors to achieve accurate identification of outliers in data with many data repetitions and fitting relationships between data features.

[0007] The technical solution adopted by the present invention to solve the above problems is:

[0008] The steps of an outlier detection method based on two new outlier factors include:

[0009] Step 1: Construct a deduplicated data set;

[0010] The specific method is:

[0011] Suppose there is a data set Represents U Class characteristics. p∈U is a point in U. t1, t2, ...t m is the eigenvalue of the corresponding feature, that is

[0012] If and only if When q=p. In this case, p and q are coincident points. p and q are repeated times n p =n q =2.

[0013] The number of repetitions of a point is used as the basis for calculating the weight of the data point, finding the repeated points of each point in the data set, determining the number of repetitions of the point, and reconstructing a deduplicated data set

[0014] R(p)={q∈U\{p}|q=p,n p ≥2}

[0015] S(p′)={p′∈U|n p′ =1}

[0016]

[0017] Among them, U represents the data set, Represents the dataset U Class features, p, q and p′ represent points on the dataset U, n p represents the number of repetitions of point p, R(p) represents the set of repeated points constructed by repeated points, and S(p′) represents the set of single points constructed by non-repeated points. Represents a deduplicated dataset, which contains all the points at different locations in the original dataset, but there is only one point at each location.

[0018] Step 2: Calculate the k-neighborhood local centroid offset density factor of each point in the deduplication data set; the specific sub-steps include:

[0019] Step 201: In the data set In the above example, we determine the k-neighborhood of each de-emphasized point.

[0020] Step 202: Normalize the number of repetitions of point p and all points in its k-neighborhood and calculate the weight of p And the weight of each point in the neighborhood

[0021] Step 203: Use the centroid formula to calculate the point The centroid position under the influence of other points in its k-neighborhood

[0022] Step 204: Calculate the local k-neighborhood centroid offset density lcsd of point p k (p);

[0023] Step 205: Calculate the local k-neighborhood centroid offset density factor LCOD of point p k (p).

[0024] Further, in step 201, in the data set In the above example, we determine the k-neighborhood of each de-emphasized point.

[0025]

[0026] in, Indicates the distance between two points. dist k (p) represents the k distance of p, which is the distance between p and the kth point closest to p.

[0027] Furthermore, in step 202, the number of repetitions of point p and all points in its k-neighborhood is normalized to calculate the weight of p And the weight of each point in the neighborhood

[0028] The weight of p and the weights of all points in its k-neighborhood interact with each other to determine the stability of the point.

[0029]

[0030] Among them, maxn p represents the maximum number of repetitions of point p and its k-neighborhood point o, minn p represents the minimum number of repetitions of point p and point o in its k neighborhood, represents the weight of point p, Represents the weight of point o in the k-neighborhood of point p.

[0031] Furthermore, in step 203, the centroid formula is used to calculate the point The centroid position under the influence of other points in its k-neighborhood

[0032]

[0033] in, Indicates that point p is in the feature The eigenvalues ​​of Indicates that point o is in the feature The eigenvalues ​​below.

[0034] Further, in step 204, the local k-neighborhood centroid offset density lcsd of point p is calculated k (p):

[0035] lcsd k (p) is p and C (p) k times the offset distance and C (p) The ratio of the sum of the reachable distances to all points in the neighborhood.

[0036]

[0037] in, reach-dist k (C (p) , o) represents point C (p) The reachable distance between and point o. d(p, C (p) ) represents point p and point C (p) The distance between k (o) represents the k distance of point o.

[0038] Further, in step 205, the local k-neighborhood centroid offset density factor LCOD of point p is calculated. k (p):

[0039] LCOD k (p) is a relative concept, expressed as the average value of the ratio of the local centroid offset density of p to the sum of the local centroid offset densities of all points in its k-neighborhood.

[0040]

[0041] Among them, lcsd k (o) is the centroid offset density factor calculated within the k-neighborhood of p.

[0042]

[0043] in, is a point in the k-neighborhood of point p, is a point in the k-neighborhood of point o, lcsd k (o) represents the centroid offset density of the local k-neighborhood of point o, d(o, C (o) ) represents point o and point C (o) distance between k (C(o), o′) represents point C (o) and the reachable distance between point o′.

[0044] Step 3: Calculate the global feature fitting offset distance factor of each point in the deduplication data set; the specific sub-steps include:

[0045] Step 301: Determine the fitting function that satisfies the characteristics of the data set

[0046] Step 302: Calculate the distance between point p and feature fitting line L

[0047] Step 303, normalizing the feature fitting offset distance;

[0048] Step 304: Calculate the global feature fitting offset distance factor GOD of the data point off (p).

[0049] Further, in step 301, the fitting function satisfied between the features of the data set is determined

[0050] Represents U Class characteristics.

[0051] feature There is a close correlation between them. For any point in U, its characteristics satisfy

[0052] in, Representation characteristics With features The functional relationship satisfied between them.

[0053] Furthermore, in step 302, the distance between point p and the characteristic fitting line L is calculated.

[0054] Furthermore, in step 303, the feature fitting offset distance is normalized:

[0055] The feature fitting stability of p is a global concept. In order to facilitate the comparative analysis of the abnormal degree of feature fitting offset of data points, the feature fitting offset distance is normalized.

[0056]

[0057] Among them, maxoff p Represents a deduplicated dataset The maximum value of the distance of all points offset from the feature fitting line L, minoff p Represents a deduplicated dataset The minimum value of the distance between all points in the feature fitting line L.

[0058] Further, in step 304, the global feature fitting offset distance factor GOD of the data point is calculated. off (p):

[0059]

[0060] Step 4: Fusion of two types of factors;

[0061] Shift the density factor LCOD based on the local centroid of the data point k (p) and global feature fitting offset distance factor GOD off (p) Calculate the fusion anomaly factor LCGOD(p) of data point p.

[0062]

[0063] Among them, ω1+ω2=2, ω1, ω2∈(0,1) represent the abnormal factor weights.

[0064] Step 5: Compare the fusion factor of each point in the deduplicated data set to identify outliers;

[0065] In the complete data set U, the abnormal factors of the data points that coincide with point p are all LCGOD(p). The value of LCGOD(p) reflects the degree of abnormality of the data point. The larger the value of LCGOD(p), the higher the degree of abnormality of point p.

[0066] The beneficial effects of the present invention are:

[0067] The present invention solves the problem that the existing outlier detection method is suitable for outlier detection of data that is independently distributed and ignores the influence of the number of data repetitions. The existing method is difficult to achieve effective outlier detection when detecting data with many data point repetitions and data features with fitting requirements. The present invention proposes an outlier detection method based on two new outlier factors, which realizes accurate identification of outliers of data with many data repetitions and fitting relationship characteristics between data features by designing a local centroid offset density factor and a global feature fitting offset distance factor and fusing the two types of factors. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 is a flow chart of the present invention;

[0069] Figure 2 is the relationship between the different types of distances used in the present invention;

[0070] Figure 3 An example of the fitting straight line used in the present invention to calculate the feature offset distance;

[0071] Figure 4 An example of the fitting curve used in the present invention to calculate the feature offset distance;

[0072] Figure 5It is the result of the present invention performing outlier detection on the task working time data set;

[0073] Figure 6 It is the evaluation index result of outlier detection of task working time data set by the present invention and other methods. DETAILED DESCRIPTION

[0074] Combination Figure 1 To illustrate this embodiment, the steps of an outlier detection method based on two new outlier factors described in this embodiment include:

[0075] Step 1: Construct a deduplicated data set;

[0076] Suppose there is a data set Represents U Class characteristics. p∈U is a point in U. t1, t2, ...t m is the eigenvalue of the corresponding feature, that is

[0077] If and only if When q=p. In this case, p and q are coincident points. p and q are repeated times n p =n q =2.

[0078] The number of repetitions of a point is used as the basis for calculating the weight of the data point, finding the repeated points of each point in the data set, determining the number of repetitions of the point, and reconstructing a deduplicated data set

[0079] R(p)={q∈U\{p}|q=p,n p ≥2}

[0080] S(p')={p'∈U|n p′ =1}

[0081]

[0082] Among them, U represents the data set, Represents the dataset U Class features, p, q and p′ represent points on the dataset U, n p represents the number of repetitions of point p, R(p) represents the set of repeated points constructed by repeated points, and S(p′) represents the set of single points constructed by non-repeated points. Represents a deduplicated dataset, which contains all the points at different locations in the original dataset, but there is only one point at each location.

[0083] Step 2: Calculate the k-neighborhood local centroid offset density factor of each point in the deduplicated data set;

[0084] Step 201: In the data set In the above example, we determine the k-neighborhood of each de-emphasized point.

[0085]

[0086] in, Indicates the distance between two points. dist k (p) represents the k distance of p, which is the distance between p and the kth point closest to p.

[0087] Step 202: Normalize the number of repetitions of point p and all points in its k-neighborhood and calculate the weight of p And the weight of each point in the neighborhood

[0088] The weight of p and the weights of all points in its k-neighborhood interact with each other to determine the stability of the point.

[0089]

[0090] Among them, maxn p represents the maximum number of repetitions of point p and its k-neighborhood point o, minn p represents the minimum number of repetitions of point p and point o in its k neighborhood, represents the weight of point p, Represents the weight of point o in the k-neighborhood of point p.

[0091] Step 203: Use the centroid formula to calculate the point The centroid position under the influence of other points in its k-neighborhood

[0092]

[0093] in, Indicates that point p is in the feature The eigenvalues ​​of Indicates that point o is in the feature The eigenvalues ​​below.

[0094] Step 204: Calculate the local k-neighborhood centroid offset density lcsd of point p k (p);

[0095] lcsd k (p) is p and C (p) k times the offset distance and C (p) The ratio of the sum of the reachable distances to all points in the neighborhood.

[0096]

[0097] in, reach-dist k (C (p) , o) represents point C (p) The reachable distance between and point o. d(p, C (p) ) represents point p and point C (p) The distance between k (o) represents the k distance of point o.

[0098] The relationship between the distance between two points, k distance, and reachable distance is as follows Figure 2 shown.

[0099] Step 205: Calculate the local k-neighborhood centroid offset density factor LCOD of point p k (p).

[0100] LCOD k (p) is a relative concept, expressed as the average value of the ratio of the local centroid offset density of p to the sum of the local centroid offset densities of all points in its k-neighborhood.

[0101]

[0102] Among them, lcsd k (o) is the centroid offset density factor calculated within the k-neighborhood of p.

[0103]

[0104] in, is a point in the k-neighborhood of point p, is a point in the k-neighborhood of point o, lcsd k (o) represents the centroid offset density of the local k-neighborhood of point o, d(o, C (o) ) represents point o and point C (o) distance between k (C(o), o′) represents point C (o) and the reachable distance between point o′.

[0105] The local centroid offset density factor LCOD of the k-neighborhood of point p k (p) reflects the abnormality of data point p.

[0106] If the number of repetitions of point p is much greater than that of other points in its neighborhood, according to the centroid calculation formula, the neighborhood centroid C of point p (p) The offset p is smaller.(p) Closer to point p and farther away from other points in the neighborhood, the lcsd obtained by solving the formula is k (p) is smaller than the same period last year. When the local centroid offset density Lcsd k (p) is small, while the local centroid offset density Lcsd of other points in its neighborhood k (o) When the ratio is large, the calculated local centroid offset density factor LCOD k (p) Relative LCOD k (o) is smaller, indicating that when the number of repetitions of point p is larger than that of its surrounding neighbors, the abnormality of point p is lower.

[0107] Similarly, when p is repeated less times than other points in its neighborhood, p is more abnormal.

[0108] Step 3, calculate the global feature fitting offset distance factor of each point in the deduplicated data set;

[0109] Step 301: Determine the fitting function that satisfies the characteristics of the data set

[0110] Represents U Class characteristics.

[0111] feature There is a close correlation between them. For any point in U, its characteristics satisfy

[0112] in, Representation characteristics With features The functional relationship satisfied between them.

[0113] Step 302: Calculate the distance between point p and feature fitting line L

[0114]

[0115] The characteristic fitting line includes the fitting straight line and the fitting curve; if the characteristic fitting line is a straight line or satisfies a single value relationship, such as Figure 3 As shown,

[0116] If the characteristic fitting line is a curve and satisfies the multi-valued relationship in extreme cases, such as Figure 4 , then take the shortest distance from the point to the characteristic line as Right now

[0117] in, Indicates the minimum offset distance obtained.

[0118] Step 303, normalizing the feature fitting offset distance;

[0119] The feature fitting stability of p is a global concept. In order to facilitate the comparative analysis of the abnormal degree of feature fitting offset of data points, the feature fitting offset distance is normalized.

[0120]

[0121] Among them, maxoff p Represents a deduplicated dataset The maximum value of the distance of all points offset from the feature fitting line L, minoff p Represents a deduplicated dataset The minimum value of the distance between all points in the feature fitting line L.

[0122] Step 304: Calculate the global feature fitting offset distance factor GOD of the data point off (p).

[0123]

[0124] GOD off The size of (p) reflects the global abnormality of point p.

[0125] 1) Compared with other points o in the data set, if the distance of p offset from the characteristic fitting curve is smaller than that of other points o in the data set, the GOD calculated by the formula is off (o) YoY GOD off (o) is smaller, indicating that from a global perspective, the abnormality of p is lower.

[0126] 2) If the distance of p from the characteristic fitting curve is larger than that of other points o in the data set, the GOD calculated by the formula is off (p) YoY GOD off (o) is larger, indicating that from a global perspective, the degree of abnormality of p is higher.

[0127] 3) If the distance between p and the offset characteristic fitting curve of other points in the data set is relatively balanced, the GOD calculated by the formula off (p)Compared to GOD off (o) are roughly the same, indicating that from a global perspective, the degree of abnormality of p is similar to that of other data points.

[0128] Step 4: Fusion of two types of factors;

[0129] Shift the density factor LCOD based on the local centroid of the data point k (p) and global feature fitting offset distance factor GOD off(p) Calculate the fusion anomaly factor LCGOD(p) of data point p.

[0130]

[0131] Among them, ω1+ω2=1, ω1, ω2∈(0,1) represent the abnormal factor weights.

[0132] Step 5: Compare the fusion factor size of each point in the deduplicated data set to identify outliers.

[0133] In the complete data set U, the abnormal factors of the data points that coincide with point p are all LCGOD(p). The value of LCGOD(p) reflects the degree of abnormality of the data point. The larger the value of LCGOD(p), the higher the degree of abnormality of point p.

[0134] In this implementation, after the design of the outlier detection method based on two new anomaly factors is completed, the outlier detection method based on two new anomaly factors is used to perform data anomaly detection with the characteristics of many data point repetitions and fitting requirements between data features. The specific steps are as follows:

[0135] Step 1: Based on the original data set, find the duplicate points of each point in the data set, determine the number of repetitions of the point, and construct a deduplicated data set;

[0136] Step 2: Set k and determine the k-neighborhood of each de-emphasis point

[0137] Step 3: Calculate the k-neighborhood centroid offset density factor of each removed point p;

[0138] Step 4: Determine the fitting function that satisfies the characteristics of the data set

[0139] Step 5: Calculate the global feature fitting offset distance factor GOD for each removed point p off (p);

[0140] Step 6: Fusion of two types of abnormal factors;

[0141] Step 7: Compare the fusion factor size of each point in the deduplicated data set to identify outliers;

[0142] After the above steps, the outlier size of each point in the data set can be obtained, and outliers can be filtered according to the outlier size. The larger the outlier factor, the higher the degree of abnormality of the object. Therefore, the objects are sorted in descending order according to their outlier factors and numbered starting from 1. Given a sequence number t, select objects with sequence numbers less than or equal to t and mark them as outliers, that is, samples predicted as positive by the model. Obviously, if t is too small, the model may ignore the real outliers. On the contrary, if t is too large, the model may mistakenly judge more normal values ​​as outliers.

[0143] The size of the data set U is |U|, OU(t) represents the set of outliers obtained by selecting t and sorting them in descending order according to the outlier factor. true Represents the set of true outliers in U. The indicators for evaluating the performance of outlier detection methods are precision P(t), recall R(t), ROC curve, and AUC value. P(t) represents the probability of samples predicted as positive being actually positive. R(t) represents the probability of samples predicted as positive being actually positive.

[0144]

[0145] Among them, TP represents the positive sample predicted by the model as positive; FN represents the positive sample predicted by the model as negative; FP represents the negative sample predicted by the model as positive; TN represents the negative sample predicted by the model as negative. When the t value is given, the larger the P(t) and R(t), the better the outlier detection result of the algorithm. When P(t) and R(t) are given, the smaller the t value, the better the detection effect of the algorithm.

[0146] ROC is the receiver operating characteristic curve, which is used to show the relationship between the true positive rate TPR and the false positive rate FPR in the model. TPR is expressed as the proportion of positive samples predicted as positive to all positive samples. FPR is expressed as the proportion of negative samples predicted as positive to all negative samples.

[0147]

[0148] The ROC curve uses the true positive rate TPR as the ordinate and the false positive rate FPR as the abscissa. When comparing algorithm performance through the ROC curve, the closer the ROC curve is to the upper left corner of the coordinate axis and the more ROC curves of other algorithms are included inside, the better the algorithm performance. AUC is the area under the ROC curve. When the AUC value is closer to 1, the algorithm performance is better.

[0149] One embodiment of the present invention is to detect abnormal values ​​in a task working time data set. Figure 5As shown in FIG. 1 , an embodiment of the present invention is a ROC curve evaluation index result of the present invention LC-GOD and the local outlier factor (LOF), the connection-based outlier factor (COF), and the K nearest neighbor (KNN) method in the task working time data set outlier detection. Figure 6 One embodiment of the present invention is the precision rate P(t) and recall rate R(t) evaluation index results of the LC-GOD and LOF, COF, and KNN methods of the present invention in detecting outliers in task working hours data set, as shown in Table 1 below.

[0150] Table 1 Evaluation index results

[0151]

[0152] One embodiment of the present invention is that the AUC evaluation index results of the LC-GOD of the present invention and the LOF, COF, and KNN methods in the task working time data set outlier detection are shown in Table 2 below.

[0153] Table 2 AUC evaluation index results

[0154]

[0155] The effectiveness of the present invention in detecting outliers in a data set with many data repetitions and fitting requirements between features is verified through examples.

[0156] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. An outlier detection method based on two new anomaly factors, characterized in that: The following steps are involved: Step 1: Construct a deduplicated data set; Step 2: Calculate the k-neighborhood local centroid offset density factor of each point in the deduplicated data set; Step 3, calculate the global feature fitting offset distance factor of each point in the deduplicated data set; Step 4: Fusion of two types of factors; Step 5: Compare the fusion factor size of each point in the deduplicated data set to identify outliers.

2. The outlier detection method based on two new outlier factors according to claim 1, characterized in that: The method for constructing a deduplicated dataset in step 1 is: Suppose there is a data set Represents U Class characteristics; p∈U is a point in U; ​​t1, t2, ...t m is the eigenvalue of the corresponding feature, that is q∈U; if and only if When q = p, in this case, p and q are coincident points; p and q are repeated times n p =n q =2; The number of repetitions of a point is used as the basis for calculating the weight of the data point, finding the repeated points of each point in the data set, determining the number of repetitions of the point, and reconstructing a deduplicated data set R(p)={q∈U\{p}|q=p,n p ≥2} S(p′)={p'∈U|n p′ =1} Among them, U represents the data set, Represents the dataset U Class features, p, q and p′ represent points on the dataset U, n p represents the number of repetitions of point p, R(p) represents the set of repeated points constructed by repeated points, and S(p′) represents the set of single points constructed by non-repeated points; Represents a deduplicated dataset, which contains all the points at different locations in the original dataset, but there is only one point at each location.

3. The outlier detection method based on two new outlier factors according to claim 1, characterized in that: Step 2 includes the following sub-steps: Step 201: In the data set In the above example, we determine the k-neighborhood of each de-emphasized point. Step 202: Normalize the number of repetitions of point p and all points in its k-neighborhood and calculate the weight of p And the weight of each point in the neighborhood Step 203: Use the centroid formula to calculate the point The centroid position under the influence of other points in its k-neighborhood Step 204: Calculate the local k-neighborhood centroid offset density lcsd of point p k (p); Step 205: Calculate the local k-neighborhood centroid offset density factor LCOD of point p k (p).

4. The outlier detection method based on two new outlier factors according to claim 3 is characterized in that: Each k-neighborhood of the removed key points in step 201 for: in, Indicates the distance between two points; dist k (p) represents the k distance of p, which is the distance between p and the kth point closest to p.

5. The outlier detection method based on two new outlier factors according to claim 3 is characterized in that: In step 202, the weight of p and the weights of all points in its k-neighborhood interact with each other to determine the stability of the point: Among them, maxn p represents the maximum number of repetitions of point p and its k-neighborhood point o, minn p represents the minimum number of repetitions of point p and point o in its k neighborhood, represents the weight of point p, Represents the weight of point o in the k-neighborhood of point p.

6. The outlier detection method based on two new outlier factors according to claim 3 is characterized in that: The centroid position in step 203 for: in, Indicates that point p is in the feature The eigenvalues ​​of Indicates that point o is in the feature The eigenvalues ​​below.

7. The outlier detection method based on two new outlier factors according to claim 3 is characterized in that: In step 204, lcsd k (p) is p and C (p) k times the offset distance and C (p) The ratio of the reachable distance to all points in the neighborhood: in, reach-dist k (C (p) , o) represents point C (p) and the reachable distance between point o; d(p, C (p) ) represents point p and point C (p) The distance between k (o) represents the k distance of point o.

8. The outlier detection method based on two new outlier factors according to claim 3 is characterized in that: In step 205, LCOD k (p) is a relative concept, expressed as the average value of the ratio of the local centroid offset density of p to the sum of the local centroid offset densities of all points in its k-neighborhood: Among them, lcsd k (o) is the centroid offset density factor calculated for a point in the k-neighborhood of p: in, is a point in the k-neighborhood of point p, is a point in the k-neighborhood of point o, lcsd k (o) represents the centroid offset density of the local k-neighborhood of point o, d(o, C (o) ) represents point o and point C (o) distance between k (C (o) , o′) represents point C (o) and the reachable distance between point o′.

9. The outlier detection method based on two new outlier factors according to claim 1, characterized in that: Step 3 includes the following sub-steps: Step 301: Determine the fitting function that satisfies the characteristics of the data set Represents U Class characteristics; feature There is a close correlation between them. For any point in U, its characteristics satisfy in, Representation characteristics With features The functional relationship satisfied between them; Step 302: Calculate the distance between point p and feature fitting line L Step 303, normalizing the feature fitting offset distance; The feature fitting stability of p is a global concept. To facilitate the comparative analysis of the abnormal degree of feature fitting offset of data points, the feature fitting offset distance is normalized; Among them, maxoff p Represents a deduplicated dataset The maximum value of the distance of all points offset from the feature fitting line L, minoff p Represents a deduplicated dataset The minimum value of the distance between all points in the feature fitting line L; Step 304: Calculate the global feature fitting offset distance factor GOD of the data point off (p):

10. The outlier detection method based on two new outlier factors according to claim 1, characterized in that: Step 4: Shift the density factor LCOD based on the local centroid of the data point k (p) and global feature fitting offset distance factor GOD off (p) Calculate the fusion anomaly factor LCGOD(p) of data point p: Among them, ω1+ω2=1, ω1, ω2∈(0,1) represent the abnormal factor weight; In step 5, in the complete data set U, the abnormal factors of the data points that coincide with point p are all LCGOD(p); the size of the LCGOD(p) value reflects the degree of abnormality of the data point. The larger the LCGOD(p) value, the higher the degree of abnormality of point p.

Images